Lastline Network and Email Defender On-Premises Release Notes

Version 9.4

Distribution Upgrade

Version 9.4 will be the final version that supports Ubuntu Xenial as the operating system distribution. In all future releases, Ubuntu Bionic will be required. To support this distribution upgrade, 9.4 will support both Ubuntu Xenial and Ubuntu Bionic. Before upgrading to any future version, appliances on Ubuntu Xenial must be upgraded to Ubuntu Bionic while running version 9.4. The upgrade of the distribution will require a reboot and may take up to an hour to complete.

You can check the distribution in use by an appliance in the Appliance Status view of the portal. The "Base Distribution" listed should be "bionic". If it is "xenial", the appliance distribution needs to be upgraded.

For help on the upgrade process, please refer to the following instructions. This update is not done automatically to prevent unexpected downtime.

New Features

  • Installation-specific feed of Network IoCs
  • License check during Engine installation
  • Improved support for sniffing on virtual appliances
  • Added capability to block traffic based on URL reputation events
  • Added new file processing pipeline on sensor
  • Added integration of RAPID in ICAP
  • Removed processing of events outside the home network
  • Added support for collecting cloud asset data
  • Added new sensor blocking options
  • Added integration to provide a feed of Network IoCs to Carbon Black Enterprise EDR
  • Added indexing of detection data in Elasticsearch/Kibana

INSTALLATION-SPECIFIC FEED OF NETWORK IOCS

Lastline can now generate an installation specific feed of Network Indicators of Compromise (IoC) that are relevant to a customer's network. The Network IoC feed currently consists of:

  • IP addresses
  • domain names

These are suspicious IPs and domains that have been observed in network detections in the customer network, or that have been observed during detonation of samples captured in the customer network.

This feed of Network IoCs can be exported using the existing notification backends:

  • syslog (in CEF or LEEF formats)
  • email
  • streaming API
  • HTTP post

This new feature was tracked internally as FEAT-4738

LICENSE CHECK DURING ENGINE INSTALLATION

Starting with this release, the appliance registration tool requires you to enter the Product Keys for the Microsoft Windows and Microsoft Office software allocated for running the dynamic analysis sandbox on Analyst, Engine, and All-in-one (Pinbox) appliances.

As per the EULA, these third-party licenses were already required for previous installations, but the installation wizard in lastline_register did not require entering them as part of the installation.

Under Appliance Status, the web-portal now shows which appliances have product keys registered, and the lastline_register configuration utility was extended to allow you to update the Product Keys for previously installed appliances. For details, refer to the appliance installation manuals.

This new feature was tracked internally as FEAT-6547

IMPROVED SUPPORT FOR SNIFFING ON VIRTUAL APPLIANCES

When installing a new sensor, the AF_PACKET acquisition driver is now recommended for all NIC types, and enabled by default. For existing appliances installed before this change, you can enable AF_PACKET from the Admin -> Appliances -> Configuration page in the UI.

This new feature was tracked internally as FEAT-6100

ADDED CAPABILITY TO BLOCK TRAFFIC BASED ON URL REPUTATION EVENTS

The sensor has now the capability to perform blocking based on URL reputation events. URL reputation events that are considered suspicious enough to trigger a block event will attempt to do so on sensors where a blocking methodology compatible with the HTTP protocol has been configured. It should be noted that, due to the characteristics of the pipeline, we cannot guarantee blocking of the first interaction with a given URL.

This new feature was tracked internally as SENT-2832

ADDED NEW FILE PROCESSING PIPELINE ON SENSOR

The sensor now enables by default the new file processing pipeline based on RAPID for the sniffing and mail analysis pipelines. RAPID is a static analysis module that integrates a variety of fast techniques to produce verdicts on analysed files without relying on dynamic analysis in the manager or cloud. In this first release, the RAPID capabilities are mostly equivalent to the previous standalone Lastline prefilter.

This new feature was tracked internally as FEAT-5985

ADDED INTEGRATION OF RAPID IN ICAP

The ICAP daemon has been updated to be able to take advantage of the RAPID file processing pipeline similarly to mail processing and sniffing.

This new feature was tracked internally as FEAT-6090

REMOVED PROCESSING OF EVENTS OUTSIDE THE HOME NETWORK

The definition of a home network in the network settings now has an impact on the content being analyzed by the sensor. The sensor will no longer analyze artifacts, produce alerts or attempt blocking for traffic where both endpoints are outside the configured home network.

This new feature was tracked internally as FEAT-5894

ADDED SUPPORT FOR COLLECTING CLOUD ASSET DATA

Defender now supports collecting information about assets present in AWS. Sensors can be configured to query the assets available in the cloud (for example, EC2 instances, S3 buckets); the discovered assets are listed in the Network Explorer page.

This new feature was tracked internally as FEAT-5885

ADDED NEW SENSOR BLOCKING OPTIONS

New blocking location functionality allows choosing where blocking should be performed: inbound; outbound; or within the home network. Additionally, the following blocking options have been added: Block connections via ICMP port unreachable; Block connections via DNS sinkholing; Block connections via HTTP redirection; Blocking location settings

This new feature was tracked internally as FEAT-5870

ADDED INTEGRATION TO PROVIDE A FEED OF NETWORK IOCS TO CARBON BLACK ENTERPRISE EDR

This feature enables customers to share Network IOCs identified within the environment to a watchlist within Carbon Black EDR. From there, administrators can decide what actions to take when connections to these known-malicious sites occur. To set up this integration, please read the instructions here.

This new feature was tracked internally as FEAT-6040

ADDED INDEXING OF DETECTION DATA IN ELASTICSEARCH/KIBANA

The detection data generated in a Defender installation is now accessible and queryable in Kibana.

This new feature was tracked internally as FEAT-5159

Detection Improvements

  • TRES-1176: Improved detection for malware that uses conditional command-line execution.
  • TRES-1684: Improved detection of VBA macros and documents that abuse regsvr32.
  • TRES-1586: Improved detection for malware built in Python detecting virtual environments.
  • TRES-1583: Improved detection of Snake ransomware.
  • TRES-1526: Improved static detection of archive-bombs.
  • TRES-1199: Improved detection of malicious encrypted Excel document attachments in email.
  • FEAT-5968: Added two correlation rules that correlate events for transfers of malicious files into intrusions. One rule correlates events based on the files' SHA-1 hashes, while the other rule correlates events based on the malware and antivirus family labels associated to the files' analysis tasks.
  • FEAT-5865: Improved sensor's ability to perform reputation decisions based on the query arguments of a URL.
  • TRES-1884: Improved detection of Cobalt Strike implant.
  • TRES-1683: Improved detection of documents that leverage Document_Close to trigger their malicious behavior.
  • TRES-1627: Improved detection of malicious XL4 weaponized XLS documents.
  • TRES-1590: Improved detection of ZLoader.
  • TRES-1572: Improved detection of documents accessing geolocation services.
  • TRES-1521: Added detection of malicious Excel documents weaponized with XL4 macro with DConn records.
  • TRES-1279: Improved detection of Shell.Explorer Objects in OLEs.

Bug Fixes and Improvements

  • FEAT-3900: The number of nodes comprising the Elasticsearch cluster on the Data Node is now reported through the Monitoring logs.
  • SENT-2885: Fixed a bug where the registration of a sensor with no sniffing interfaces defined (e.g. MTA, ICAP) would lead to a failure.
  • USER-4687: Fixed an issue where inappropriate permissions were being set while creating multiple user accounts.
  • USER-4689: Fixed a bug where having a large number of sensors would cause the appliance metric graph legends to compress and obscure the graphs themselves.
  • USER-4620: Fixed a bug where Quarantine configuration was not visible if DROP EMAILS WITH MALICIOUS ATTACHMENTS is disabled
  • USER-4517: Fixed a bug where changing the time units of min interval in syslog trigger was not working as expected.
  • TRES-1915: Fixed a bug related to process creation through WMI.
  • TRES-1594: Added support of unicode characters in file names inside archives.
  • SENT-2952: Prior sensors were affected by an issue where non-inline sensor processing modes (such as passive sniffing) would still enforce a default maximum processing time of 1 hour on messages. This could cause messages extracted by sniffing to not be processed correctly under high load situations. This problem is now fixed and the maximum processing time is enforced only in MTA mode.
  • SENT-2943: Improved robustness of the TLS NTA processing in case of issues at extracting a full ja3s fingerprint.
  • SENT-2922: Improved file extraction logic for sniffing sensors by implementing additional checks to minimize the likelihood of over-extraction of partial or irrelevant files, which led in the past to excessive load on large installations.
  • SENT-2918: Fixed an issue where the installation of a sensor using a Silicom adapter would require additional steps such as disabling the support before performing the registration. Installing a new sensor using a Silicom adapter now does not require any special additional step and no content is required in /etc/appliance-config/override.yaml. It is now sufficient to proceed with the registration, and once the sensor is correctly registered execute lastline_setup to define the inline interface pairs.
  • SENT-2883: Fixed an issue on the sensor where defining an unusually high number of sniffing interfaces could cause the IDS service to fail to initialise.
  • SENT-2843: Fixed a problem where attempting to limit the IPs allowed to interact with an MTA sensor from the UI would not have the desired effect, allowing any host capable to interact with the sensor.
  • PLTF-1468: When clicking on the 'Logs' tab of the 'Files downloaded' page, the file download logs are now properly sorted by timestamp.
  • PLTF-1361: Fixed issue that could lead to excessive memory usage in the "session-tracker-daemon" causing it to no longer be able to retrieve user login events from Active Directory servers.
  • FEAT-6152: Sensor appliances now allow you to configure the maximum size of an extracted artifact via lastline_setup. It is now possible to customise this value by acting on the "sensor_max_upload_filesize_mb" setting, with a default value of 20MB.
  • USER-3220: Fixed a bug where global search icon in analysis report view redirects user to 404 error.
  • TRES-1373: Fixed a problem in document application bundle analysis. When a document file is submitted with a password and incorrect extension, we allow renaming the extension to a proper one.
  • TRES-581: Fixed a bug in dynamic analysis when an unknown process "sample.exe" appeared in the report.
  • SENT-2921: Fixed an issue where reputation events would be reported by the sensor as successful even when a firewall had blocked the handshake by injecting a RST/ACK.
  • SENT-2919: Fixed a problem where inline sensors may incorrectly add more delay than the expected 10ms in the forwarding plane.
  • SENT-2856: Improved the handling of timeouts in the MTA processing in conjunction with the integration with RAPID. If a message analysis times out while waiting for a verdict from dynamic analysis a warning will be reported in the monitoring logs.
  • SENT-2809: Improved the memory utilization of some components of the mail processing pipeline.
  • MALS-3294: Removed SSDeep hash information extracted during static analysis of applications.
  • FEAT-6089: The suricata daemon in charge of the sensor IDS capabilities is now running as a docker container. The associated logs on the sensor have consequently changed slightly. The main suricata log is now located at /var/log/suricata/suricata-lastline-daemon.log.
  • FEAT-6002: Elasticsearch and Kibana have been upgraded to version 6.8.9.
  • FEAT-5992: The ICAP service now has the capability to analyze EML files being transferred on top of the HTTP protocol.
  • FEAT-5961: Network analysis records now include the name of the NIC where the network activity associated to a record was observed.
  • FEAT-5905: A list of password candidates may be provided when submitting a URL using the Analyst API. This list will be used if the URL is pointing to an encrypted file (for example an encrypted archive).
  • FEAT-5391: The classification of devices from their observed network traffic has been improved by integrating data provided by Fingerbank.

Changes to MacOS and Android Support

In the next release, 9.5, we will be changing the way in which we analyze macOS and Android files. We still continue to analyze the macOS files that are likely to compromise systems, as well as PDF and Word documents that can impact both macOS and Windows operating systems, however Android and some macOS file types are no longer analyzed, and llama-macos service will no longer be used or installed. For additional details please contact Lastline Support.

Deprecation of API Methods

The following KnowledgeBase features, announced for deprecation in On-Premise 9.2, will be effectively decommissioned, both at the UI and API levels, in the Lastline Analyst and Detonator On-Premise 9.4 release:

  • To improve performance, the KnowledgeBase clustering service will be discontinued. This only changes some Intelligence search capabilities: the clustering tab is no longer available and you cannot search by cluster. Static clustering-based detection remains operational.
  • All strings will remain indexed across the entire process space. You retain the ability to search by string independently of location using the existing top-level keyword. However you cannot search for strings by sub-key location (heap, stack, memory block or executable section).
  • The KnowledgeBase will no longer provide the analysis subject location within its search results. In the Intelligence page, under the Report tab, the subjects column is removed from the table of results.

The Lastline API documentation includes a deprecation schedule for methods in the legacy API, as well as information on how to replace usage of these deprecated methods with supported methods.

Released Appliance Versions

As part of this release, we are making available the following versions of Lastline appliances for use with Lastline Defender On-Premises:

  • Lastline Manager version 1110
  • Lastline Engine version 1110
  • Lastline Data Node version 1110
  • Lastline Sensor version 1221
  • Lastline All-in-one (Pinbox) version 1110

Released Sandbox Images Versions

The sandbox images version will remain at 2020-03-13-01.

9.3.2 9.4.1