7.16

13 Nov 2017

Version 7.16

New features

Knowledge Base Alerting service
AF_PACKET v3 packet acquisition support on sensor
Improved appliance selection
Improved visual consistency of Lastline Portal
Flexible export of blacklisted IoCs in intelligence tab
Integration with McAfee TIE using OpenDXL
Display DNS resolution timeline in intelligence tab
New ICAP and explicit proxy blocking capabilities
ICAP and explicit proxy availability improvements
Explicit proxy performance improvements
Support shifting the selected time range with one click
Show warning when downloading analyzed file
Improvements to ICAP and explicit proxy
Improvements to Active Directory integration
Ability to configure Engine appliances as HA
File analysis improvements
URL analysis improvements
Traffic sniffing improvements
Bug fixes and improvements

Knowledge Base Alerting service

Users with a Knowledge Base license now have access to a new alerting service under the Knowledgebase interface of the Lastline Enterprise Portal Intelligence Page. With this version, licensed users can create matching rules to match on Lastline analysis results while they are being indexed within the Knowledge Base. Rules are based on the same language as queries, offering the same rich set of terms, with the additional support of regular expressions.

The alerting service enriches the capabilities of the Knowledge Base with the support of new use cases. With alerting, users can monitor for company assets (e.g. domains, mail addresses, clients) and understand if their company is being targeted by recent threats. Users can also generate feeds of samples satisfying certain criteria or exhibiting specific IoCs (e.g. hunting for samples using bit coin wallets).

Users can directly access rule matches from the Lastline Enterprise Portal. For proactive notification, users can also configure different types of notifications to automatically receive these matches (e.g. email notification, syslog notification).

The alerting service, the creation of matching rules, the access to matches from the portal, or the configuration of notification based on matches are described in details within the Lastline Portal Guide.

This improvement was tracked internally with tickets FEAT-1414 and FEAT-1735.

AF_PACKET v3 packet acquisition support on sensor

This release includes experimental support for a new packet acquisition technology on sensors meant to replace PF_RING in the long term. While tested at length on a number of supported hardware configurations, AF_PACKET support is less mature than PF_RING and should at this stage be considered experimental. It is our plan to phase out the use of PF_RING in favor of AF_PACKET: future sensor releases will default to using AF_PACKET upon first install.

Main improvements associated with AF_PACKET:

Better performance at multi-gigabit packet rates (on 10g appliances)
Possibility to enable AF_PACKET on any NIC driver. This should lead to performance benefits on NIC drivers not supported by PF_RING (e.g. virtual sensors).
Support for recent NICs. The current release includes support for Intel X550 NICs, additional NICs will be supported in future releases exclusively via AF_PACKET.

To enable AF_PACKET on a sensor appliance, a new option is available in the sensor configuration page. The option can be used to either directly switch from PF_RING to AF_PACKET on appliances currently configured to use PF_RING, or to enable AF_PACKET on appliances that do not support PF_RING. The switch to AF_PACKET requires a NIC driver reload, but in the majority of cases should not require a reboot.

This feature was tracked internally as FEAT-2155.

Improved appliance selection

This release improves the modal dialog that is used when selecting multiple appliances in the Appliance monitoring logs and Appliance metrics views. The existing selection dialog could be cumbersome to use for customers with many appliances. The appliance selection dialog now includes options to clear the selection, as well as select all appliances of a given type.

This improvement was tracked internally as USER-2568.

Improved visual consistency of Lastline Portal

In this release we have made a number of small changes that improve the visual consistency of the Lastline Portal, by improving conformance with our visual style guide. These improvements include:

more consistent use of icons for remove vs delete operations
fix missing titles for links that open in new tab
more consistent display of icons within links
more consistent options for closing modal dialogs

This improvement was tracked internally as USER-2502.

Flexible export of blacklisted IoCs in intelligence tab

When accessing the domains and IPs resulting from a search in the Intelligence Page, the Network IoCs tab now offers users the choice to select the entries to be exported for blacklisting. Recommended entries to be exported are selected by default.

This feature was tracked as FEAT-2030

Integration with McAfee TIE using OpenDXL

Lastline Enterprise now supports integrating with McAfee's Threat Intelligence Exchange (TIE) server using the Open Data Exchange Layer (OpenDXL). When configured for this integration, Lastline Enterprise will push to the TIE server information on files captured in the protected network. This includes downloaded files as well as files attached to emails.

To configure this integration, refer to the dedicated integration guide that is available for download from the Manuals page.

This feature was tracked as FEAT-1355

Display DNS resolution timeline in intelligence tab

When performing a search for IPs or domains contacted during analysis in the Intelligence Page, the DNS tab will now display a timeline of the DNS resolutions related to the query terms that Lastline observed during analysis. This timeline also provides some additional information about the IPs and domains involved. This feature was tracked as FEAT-1740

New ICAP and explicit proxy blocking capabilities

New blocking modes are available to customers leveraging ICAP or explicit proxy for HTTP monitoring. The different blocking modes allow a trade-off between security and delay in serving potentially suspicious content.

passive: content is extracted from analysis, but no blocking is ever attempted.
sensor-known: blocking decisions are taken by leveraging exclusively a cache local to the specific sensor appliance. Only artifacts that have already been downloaded by the specific sensor appliance will be blocked.
manager-known: the currently default blocking mode, it leverages scoring information collected by the hosted backend or by the manager to perform blocking decisions. If a file is observed for the very first time (patient zero), the file will not be blocked.
full: provide protection against patient zero by stalling a file download for a file that is deemed suspicious by the prefilter and that is not yet known by Lastline. The file transfer will be stalled until its analysis will be completed by the backend. This blocking mode can lead to delay of several minutes when attempting to download suspicious unknown files.
full with feedback: particularly indicated for usage in the explicit proxy implementation, this blocking mode provides feedback to the user about the fact that a given file is currently being held for analysis.

The different blocking modes can be activated from the sensor appliance configuration in the user website. Different blocking modes can be enabled for different filetypes.

ICAP and explicit proxy availability improvements

This release improves sensor reliability when running the ICAP or the explicit proxy service. More specifically, the sensor continuously monitors the state of the ICAP service and its responsiveness. In case the ICAP service was affected by any issue affecting its availability, the appliance status is updated to reflect the issue and an attempt is made to automatically recover from the issue by triggering a restart of the relevant services.

When the ICAP service is active, statistics are also shown in the monitoring logs on the number of parallel connections currently handled by the service and the number of currently active workers.

This improvement was tracked as FEAT-1692.

Support shifting the selected time range with one click

The time range selection widget used throughout the Lastline Portal now supports shifting to the previous or following interval with a single click on the < and > buttons. Using these buttons shifts the selected interval while preserving the number of days that are selected. This feature was tracked as FEAT-2083.

Show warning when downloading analyzed file

Portal will now show a warning the first time a user attempts to download a file that was submitted from analysis. The warning will advise the user that file is potentially malicious and should be handled with care.

This improvement was tracked as FEAT-2022.

Improvements to ICAP and explicit proxy

FEAT-1601, FEAT-1602: revamped analysis pipeline for ICAP and explicit proxy blocking. This includes the ability to block on threat intelligence data, and the ability to analyze artifacts being uploaded in POST messages. For ICAP deployments, the additional features require the configuration to be set to send also REQMOD requests. Both REQMOD and RESPMOD can be configured at the same time if supported by the proxy appliance.
SQUID-13: support for the installation of custom CA certificates in explicit proxy mode. It is now possible to place a custom CA certificate in /etc/puppet/private/squid/certificate.pem, to be used for signing during TLS inspection.
SQUID-14: if an upstream proxy is configured to set the X-Forwarded-For header, the header information will be taken into account. It should be noted however that this is possible exclusively for HTTP exchanges, as HTTPS ones will not contain that information.
SENT-585: experimental support C&C detection and drive-by download detection in explicit proxy mode. This release introduces experimental signature matching and on-the-wire webpage inspection on the traffic analyzed by the explicit proxy, including analysis of the content of encrypted TLS flows.
FEAT-2153 ability to analyze scripts. It is now possible to analyze and block malicious scripts. Blocking mode for scripts can be configured from the sensor configuration UI (scripts belong to the file category "other" in the ICAP blocking configuration). Support includes files with the following extensions: .bat, .ps1, .psm1, psd1, .vba, .vbs, .js, .jse, .hta
SURI-593 improved support for network detections (explicit proxy only). When configured to run in explicit proxy, the sensor will inspect the decrypted traffic with capabilities that are similar to those used when sniffing traffic.

Improvements to Active Directory integration

The Active Directory integration has seen a number of improvements in this release.

Support for explicitly specifying a domain name when configuring the user credentials to be used for fetching login events from a domain controller. This can be done by providing a username in the form DOMAINNAME\USERNAME
Official support for domain controllers running Windows Server 2016
Enforce use of NTLMv2 for authentication to domain controller, instead of weaker authentication such as NTLMv1

These improvements were tracked as FEAT-1962.

Ability to configure Engine appliances as HA

In certain deployment scenarios it can be useful to disable a subset of Engine appliances from processing analysis tasks. For this purpose, the system provides a utility for marking individual Engine appliances as inactive, meaning that they will not be assigned any work.

For this purpose, this release includes the lastline_configure_engine_availability tool can be used to obtain a list of Engine appliances, to mark them as inactive, and to re-enable appliances that have been previously disabled. For detailed steps on how to configure an Engine, refer to the installation manual.

This improvement was tracked as FEAT-1584.

File analysis improvements

MALS-2307: accept Nuget archives in Analyst API.
MALS-2254 MALS-2273: improved analysis environment selection for XPS submissions.
LLFILE-367: accept PCAP-ng files in Analyst API.
MALS-2274 MALS-2196: more aggressively use static document features to determine the analysis environment used for dynamic analysis.

We have made enhancements to the detection of

LLADOC-458 exploits using SOAP Moniker in Microsoft Office documents.
LLADOC-450 LLADOC-451 LLADOC-454 exploits using external commands, external OLE data, or DDE Links in Microsoft Office documents.
LLADOC-457 LNK files embedded in Microsoft Office documents.
LLADOC-453 encrypted documents embedded in documents or emails.
LLADOC-336 compressed streams embedded in Hangul documents.
LLADOC-459 evasive code using mouse movement.
LLADOC-462 position independent shellcode.
LLADOC-463 suspicious OLE objects embedded in RTF documents.
LLADOC-464 encoded commands embedded in Powerpoint presentations.
LLAM-3046 LLAM-3014 sleep-based evasions.
SIGLOGSCAN-205 x86 shellcode embedded in legitimate tools.
LLADOC-469 malicious DDE commands embedded in Microsoft Office documents.

and improved the reliability of

LLAM-2978 LLAM-2981 LLAM-2997 extracting process snapshot metadata and cover more memory regions.
LLAM-3030 tracking behavior spawned from MSI packages.
LLADOC-467 extracting non-ASCII code snippets from PDF documents.
MALS-2324 identifying email messages in RFC2822 format.
LLFILE-381 analyzing OLE streams embedded in Microsoft Office documents.

URL analysis improvements

LLWEB-1777: improve whitelisting of newly spawned benign processes observed during a URL analysis.

Bug fixes and improvements

MALS-2299: more robust handling of hash-lookups with MD5 collisions.
MALS-2280: improved handling of Analyst API get_completed for returning completion information for partially completed seconds.
MALS-2298: more robust handling of corrupted archives in Analyst API.
MALS-2320: better handling of large archives in Analyst API.
MALS-1947: better validation of report_uuid in calls to Analyst API.
MALS-2345 MALS-2280 MALS-2309: improve handling of queries for completion information in the Analyst API using long time windows.
LLADOC-461: better extraction of long strings in Analyst API results (e.g., subject command line or Microsoft Office macro code).
MALS-2338: support optimized API call for retrieving UTC timestamp.
MALS-2326: more flexible support of sessions in Analyst API - allow using latest API clients against server versions that do not support sessions.
ENG-2169: downgrade impact of lack of pf ring support in lastline_test_appliance
SENT-637: fixes to an irqbalance bug in trusty appliances
SURI-591: performance improvements to sniffing performance in trusty appliances
SENT-638: robustness improvements to on-the-wire webpage inspection
LLMAIL-372: allow customization of hostname presented when receiving emails in in-line MTA mode
LLMAIL-368: Tolerate (without reporting) one-time transient IMAP/POP3 connection errors
LLMAIL-367: for SSL/TLS SMTP receiver, load all certificates in user-provided certificate, instead of only the first one.
CC-1953: lastline_register/lastline_test_appliances: no longer use OPTIONS method for testing connectivity to a proxy server
LLUPL-538: no longer incorrectly refer to "Internal Host" in notification emails
USER-2448: fix issue where built-in dashboard configurations were not added to history of recently used dashboards
USER-2476: allow non-administrator accounts with the "can view appliances" permission to view appliance configuration
USER-2517/USER-2520: fix issue that caused link to child tasks to not be shown in some analysis reports
USER-2529: fix incorrect link to "view traffic capture in new tab"
USER-2530: fix bugs in "Label/Whitelist Hosts" dialog
USER-2539/USER-2541: fix bug that prevented deleting a home network configuration
CC-1970/CC-2015: improve error reporting during appliance installation
CC-2026: diagnostic checks: support case where multiple virtual drives are defined on an LSI RAID controller
LLUPL-525: Improve scoring of network events associated with UDP traffic. UDP events were previously downgraded in impact if there was no response from server. This logic has now been disabled because for many UDP events no server response is expected.
LLPSV-117: The llpsv service now runs with reduced privileges.
SURI-507: Support for blocking network events based on network signature detections.
LLMAIL-375: Improved error checking and handling when fetching emails from an IMAP inbox.
LLMAIL-371: In the X-Lastline header (in-line email analysis), indicate when analysis failed because of too many upload errors.
FEAT-1872: Support for management interface different from eth0

Deprecation of API methods

All methods of the legacy API (/ll_api/ll_api) are now deprecated. The Lastline API documentation includes a deprecation schedule for methods in the legacy API, as well as information on how to replace usage of these deprecated methods with supported methods.

Released appliance versions

As part of this release, we are making available the following versions of Lastline appliances for use with Lastline Enterprise On-Premises:

Lastline Manager version 800.4
Lastline Engine version 800.4
Lastline Sensor version 725.1
Lastline All-in-one (pinbox) version 800.4

Released sandbox images versions

With this release, the sandbox images version is updated to 2017-07-17-01.

Deprecation of appliance versions

With On-premises release 7.16, we are dropping support for sensor versions before 720. Versions before 720 were deprecated with release 7.14.

Distribution Upgrade

As of this version, support for Ubuntu Precise as the underlying operating system distribution has been discontinued. Before upgrading to version 7.16, appliances that are still on Ubuntu Precise will need to be upgraded to Ubuntu Trusty.

Users can check the distribution in use by an appliance in the Appliance Status view of the portal. The "Base Distribution" listed should be "trusty". If it is "precise", the appliance distribution needs to be upgraded.

For complete information regarding the upgrade process please refer to the Lastline Support Knowledge Base