Lastline Enterprise On-Premises Release Notes

Version 8.4

Image Download Time

The overall size of our sandbox images has increased, impacting the time it may take to download these images during the install or upgrade compared to time it took in our most recent 8.3.4 release. The install and upgrade time can be reduced by pre-loading the Lastline sandbox images before you upgrade or install. Instructions on how to pre-load sandbox images can be found here. Additionally, to mitigate this risk upon installation we strongly suggest you enable downloading from a CDN, which is documented in the Lastline Manager Installation Guide. For customers upgrading from a previous version who are concerned about the download speed may also contact support to enable the use of CDNs. As part of 8.4, this option will be exposed to customers via the lastline_register utility.

New Features

  • Move Admin > Integrations items to new Notifications and Data sources sections
  • Integrate Kibana on-prem
  • UI to display log of all email messages
  • New minimum impact filter in network events view
  • Network and security summary widget
  • Integrate Antimalware Scan Interface (AMSI) for MS Office document analysis in Lastline sandbox
  • Updated host profile - new summary and tabs
  • Monitored network hosts widget
  • Ingest TLS logs
  • New alert suppression wizard
  • Use ICAP to remove body of malicious POST requests
  • New "what else is interesting" widget for intrusions listing page
  • Display documentation for detectors
  • Display the state of processed mail messages
  • Detection of URL Link Chains
  • Extract Powershell/VBS/JS executed code to Llama report using Windows 10 AMSI interface

MOVE ADMIN > INTEGRATIONS ITEMS TO NEW NOTIFICATIONS AND DATA SOURCES SECTIONS

Two new sections have been added under Admin: Notifications and Data sources. Items have been moved from Integrations into the new sections.

This new feature was tracked internally as USER-3248

INTEGRATE KIBANA ON-PREM

The Kibana visualization tool is now integrated in the portal, giving access to the data records stored on the data nodes (webrequests, passive DNS, and netflow records). This new functionality enables analysts to easily explore data records, for example, as part of an investigation.

The Kibana interface is accessible via the new menu item Investigation -> Network explorer for accounts with the Can Access Kibana permission.

This new feature was tracked internally as FEAT-4301

UI TO DISPLAY LOG OF ALL EMAIL MESSAGES

Lastline mail sensor now provides a status update of each message during processing.

A new section has been added to the Email tab, called Messages log. The new section displays a table with all mail messages processed. Details about each message are available with a single click. Additionally, several new filters have been introduced, including mail message log id, message action, and content action.

Additionally, while previously only messages containing suspicious artifacts were surfaced in the Email Tab, the new Message tab reports the full log of all messages processed by the sensor, including messages with no urls or attachments or messages whose artifacts were considered benign during analysis.

This new feature was tracked internally as FEAT-4091

NEW MINIMUM IMPACT FILTER IN NETWORK EVENTS VIEW

A new minimum impact filter has been added to the network events view.

This new feature was tracked internally as FEAT-4080

NETWORK AND SECURITY SUMMARY WIDGET

A new visualization widget has been introduced to show how network traffic is processed and analyzed by the product.

This new feature was tracked internally as FEAT-4072

INTEGRATE ANTIMALWARE SCAN INTERFACE (AMSI) FOR MS OFFICE DOCUMENT ANALYSIS IN LASTLINE SANDBOX

The Windows Antimalware Scan Interface (AMSI) was integrated into Lastline Sandbox for MS Office document analysis. The AMSI increases visibility into execution of VBA code, which allow the sandbox to observe not only system level events, but also VBA code specific events.

This new feature was tracked internally as FEAT-4043

UPDATED HOST PROFILE - NEW SUMMARY AND TABS

Host Profile Security teams can now investigate threats on hosts more efficiently. A newly designed Summary section for the hosts lists various common attributes that enables security teams to identify the hosts and provides actionable context. The new design also clearly identifies all incidents and events for the hosts. Additionally, the host profiles also provides context of applications observed on the hosts based on network data. This provides context when investigating threats on a host.

There is a known issue the the Host Profile view, where hosts on the "home network" do not show on the host profile page. This will be fixed in a coming release.

This new feature was tracked internally as FEAT-4027

MONITORED NETWORK HOSTS WIDGET

A new widget has been introduced on the Network dashboard that displays an overview of the number of hosts on a network, and information about the devices, OSs, services, and applications running on each host.

This new feature was tracked internally as FEAT-4025

INGEST TLS LOGS

Defender now supports ingesting TLS records generated by the Lastline sensor. The records are stored and can queried together with other existing NTA records (passive DNS, netflow, web requests).

The Defender rules language has been extended to support creating rules matching on TLS records (including SNI and JA3 values).

This new feature was tracked internally as FEAT-4008

NEW ALERT SUPPRESSION WIZARD

When triaging events, security analysts may find that certain events are not malicious or not interesting to them based on their network environment. The analysts can use the suppress event capability to create rules based on certain criteria which when matched against any future events will take the action of either demoting the matching event to INFO status or completely removing the matching event from the system. Removing or demoting events that match the rules will ensure that correlation rules that create incidents or intrusions are not triggered on these events. The list of criterion available for rule matching is available in the user manual.

This new feature was tracked internally as FEAT-3966

USE ICAP TO REMOVE BODY OF MALICIOUS POST REQUESTS

ICAP integration now supports the sanitation of malicious content from POST requests rather than blocking the request. POST requests then will be forwarded to the server without the malicious content.

This new feature was tracked internally as FEAT-3930

NEW "WHAT ELSE IS INTERESTING" WIDGET FOR INTRUSIONS LISTING PAGE

The intrusion listing page now includes a new widget containing custom-tailored facts and destinations that were prepared based on activity in your network.

This new feature was tracked internally as FEAT-3925

DISPLAY DOCUMENTATION FOR DETECTORS

You can now click on a detector in the Portal to learn about the detector goal, a high-level overview of how the detection works and well-known causes for true and false positives.

This new feature was tracked internally as FEAT-3709

DISPLAY THE STATE OF PROCESSED MAIL MESSAGES

A user can now see up-to-date information for both emails that have fully resolved and emails that are still in process. Additionally, a user can filter emails based on such state information.

This new feature was tracked internally as FEAT-3677

DETECTION OF URL LINK CHAINS

Lastline's URL analysis engine extracts and analyzes URLs found in Google Docs submitted for analysis. This allows the engine to follow the URL link chain and detect malicious payloads or phishing pages at the end of the chain.

This new feature was tracked internally as FEAT-3578

EXTRACT POWERSHELL/VBS/JS EXECUTED CODE TO LLAMA REPORT USING WINDOWS 10 AMSI INTERFACE

Lastline Sandbox analysis on Windows 10 is now integrated with AMSI (Antimalware Scan Interface) for Powershell, VBS, JS scripts and Macro code analysis. The new feature increases visibility into script/macro code execution and improves detection capabilities of the sandbox.

This new feature was tracked internally as FEAT-3515

Detection Improvements

  • TRES-647: Improved prefilter detection for documents with XL4 macro code
  • TRES-641: Improved detection of OSX/Pirrit malware family.
  • TRES-584: Improved detection of compiled python scripts.
  • TRES-569: Improved detection of PUA/Softcnap malware family.
  • TRES-551: Improved prefilter detection for documents with XL4 macro code.
  • TRES-490: Reduced false positive rate for executables.
  • TRES-478: Improved detection of POWRUNER and BONDUPDATER malware families.
  • TRES-460: Improved detection rate of compressed SWF files.
  • TRES-436: Improved analysis of malicious executable and document files targeting Mac OS.
  • TRES-417: Improved detection of OSX/Callisto malware family.
  • TRES-397: Improved detection of Shadow Hammer malware family.
  • TRES-387: Improved detection of Flashback, Crisis, XSLCmd, Calisto, Coldroot, Dummy, CreativeUpdate and DarthMiner OSX malware families.
  • TRES-377: Improved detection of malicious URL embedded into PDF.
  • TRES-371: Improved detection of XSLCmd malware family.
  • TRES-370: Improved detection of OSX/Komplex malware family.
  • TRES-324: Improved detection of ASLR bypass in Microsoft Office documents
  • TRES-301: Improved detection of evasive Microsoft Office documents which use country-specific checks to bypass analysis systems.
  • TRES-295: Improved detection of malware exploiting ACE format vulnerability (CVE-2018-20250).
  • TRES-197: Improve analysis of encrypted XLS documents.
  • TRES-177: Improved detection of LazyMeerkat malware family.
  • TRES-163: Improved detection of Chches malware family (APT10).
  • TRES-150: Improved detection of embedded API names in OLE streams of XLS files.
  • TRES-148: Improved detection of Vflooder malware family.
  • TRES-134: Improved detection of exploits targeting Microsoft Equation Editor.
  • LLMAIL-498: Support for detection of a new redirection attack observed in recent spam waves, where the URLs leverage benign third party redirection services such as google.dm
  • LLMAIL-489: Added support for processing URLs in emails that have been analyzed by Zix.
  • FEAT-4010: A new detector raises an alert upon observing TLS traffic with a new, never-seen-before JA3 hash. JA3 hashes characterize applications generating TLS traffic, so this detector can be used to identify new and potentially unauthorized applications generating encrypted traffic.

Bug Fixes and Improvements

  • USER-3394: The link from Admin / Appliances / Configuration / Integrations / Active Directory to Admin / Data Sources / Active Directory now works as expected.
  • USER-3297: Fixed incorrect URLs linking to the AWS documentation.
  • TRES-419: Improved extraction and parsing of long encoded Powershell command line.
  • SENT-1650: Fixed a suricata bug where an invalid certificate in the TLS handshake could cause the component to crash.
  • SENT-1175: Fix to an issue caused by a race condition where the SHA1 hash for certain file downloads processed by the sensor would be reported incorrectly in the UI.
  • SENT-1173: Fixed a major issue in the ICAP/Explicit proxy implementation of the malicious progress mode that was impacting correct functionality when processing certain downloads. The issue would often manifest by having continuously updating MD5 hashes for the file under analysis in the progress page served to users.
  • SENT-1162: Added monitoring of the file processing pipeline used by the sensor to process artifacts in sniffing and ICAP mode. Queue utilization for the file processing pipeline is now reported in the monitoring logs and the appliance status will warn in case of anomalous increases in the backlog.
  • SENT-1148: Ensured that all sensor components honor customization of the sensor::max_file_size value.
  • SENT-1146: Fixed a problem where updating the sensor hostname by means of lastline_register --change-local-fqdn= would lead to an error in applying the configuration in xenial.
  • SENT-1136: Now allow the selection of port 8080 as listening port for the explicit proxy component.
  • SENT-1089: Improved the ICAP component ability to handle HTTP POST requests containing large payloads.
  • LLMAIL-487: It is now possible to set up the sensor MTA to receive emails in plaintext SMTP, dropping the requirement for encrypted communication.
  • LLMAIL-481: Fixed a bug that would cause a deadlock when the mail sensor was restarted and a very large backlog of messages was found on cold storage.
  • FEAT-4216: Sensors components making use of the Lastline prefilter (mail, sniffing file processing, ICAP) have been updated to ensure that prefilter invocations are always time-bound. This means that an issue in the prefilter logic causing processing to be stalled on a file can no longer lead to an impact on the file processing pipeline and queues.
  • FEAT-4144: The suricata component running on sensors and managers has been updated to version 4.1.4, addressing a number of stability and security issues. Full details can be found here: https://suricata-ids.org/2019/04/30/suricata-4-1-4-released/
  • FEAT-4104: Improved internal monitoring for the sensor IDS component. The status of a sensor appliance configured for packet sniffing will now be affected by issues in the operation of the component, and can lead to email notifications if appliance status notifications have been configured. The IDS component now monitors and warns on the following conditions: A) The suricata component fails to initialize successfully; B) The suricata component is restarting too frequently; C) No packets are being processed by the IDS component despite sniffing being enabled.
  • FEAT-3972: The unique and all files downloaded views under Network > Files downloaded can now be filtered by a minimum score.
  • FEAT-3932: Sniffing sensors have now the capability to produce netflow logs for UDP flows.
  • FEAT-3931: We have improved the sensor capability of annotating netflows with an application level protocol tag by adding more protocols.
  • FEAT-3749: The time to run lastline_register during installation of a sensor appliance has been reduced. More packages are installed from the ISO during installation so that less time and network traffic is required at registration time.
  • FEAT-2902: Events containing unicode character sets can now be successfully exported in XML format.

Deprecation of API Methods

No additional API methods are being deprecated or discontinued in this release.

The Lastline API documentation includes a deprecation schedule for methods in the legacy API, as well as information on how to replace usage of these deprecated methods with supported methods.

Deprecation of Legacy Web Threat Analysis System

In the previous 8.3 release, we introduced a new analysis system for the dynamic analysis of web threats. This new system is faster and covers a wider variety of attacks. Until now, the old and new analysis systems were run in parallel to evaluate the detection accuracy of the new system. This meant analysis runs for web threats were showing multiple analysis reports.

As part of this release, we are now deprecating the use of the legacy system, meaning that these duplicate reports (titled "instrumented browser" or "instrumented file-viewer") are no-longer generated as part of the analysis.

Released appliance versions

As part of this release, we are making available the following versions of Lastline appliances for use with Lastline Enterprise On-Premises:

  • Lastline Manager version 1060
  • Lastline Engine version 1060
  • Lastline Sensor version 1081
  • Lastline All-in-one (Pinbox) version 1060

Released sandbox images versions

The sandbox images version will be upgraded to 2019-04-18-01.

Distribution Upgrade

Version 8.3.2 was the final version to support Ubuntu Trusty as our operating system distribution. In order to upgrade to 8.4, you must be running Xenial as the operating system distribution.

You can check the distribution in use by an appliance in the Appliance Status view of the portal. The "Base Distribution" listed should be "xenial". If it is "trusty", the appliance distribution needs to be upgraded.

For help on the upgrade process, please refer to the following instructions. This update is not done automatically to prevent unexpected downtime.

8.3.4 8.4.1