8.0

16 Jul 2018

Version 8.0

With the onpremises v8.0 release, the Dell R440 hardware platform is fully certified. We have also made changes to the recommended hardware platform to improve the overall performance and ensure the features and functionality we provide to our customers operate as expected. For more details, please refer to Lastline Support Knowledge Base.

New features

Updated look and feel for Lastline Portal
Enforce a strong password policy for new passwords on Lastline Portal
Update sensor to xenial kernel
On-prem deployments can specify higher email threshold when using local SMTP
Local callback detection
ICAP Processing log
Improved user interface for file submission
Display labels for the classification of samples in file download and mail attachment views
Support for sensor groups
Backup to private AWS S3 storage
Upgrade to Suricata 4.0
Disable TLSv1.0 on Manager/Pinbox/Analyst appliances
Leverage malware analysis information and labels in Enterprise product
EVE-based parsing of Suricata events
Support granular permissions for viewing and managing individual appliances

Updated look and feel for Lastline Portal

This release introduces a new styling for the Lastline Portal, with the aim of reducing visual clutter and adopting a more up-to-date look-and-feel. Note that this does not change the overall structure and functionality of the Portal: while the Portal looks different, the same functionality remains available in the same places, so existing users should be able to quickly adapt to the new look.

This change was tracked internally as FEAT-2790

Enforce a strong password policy for new passwords on Lastline Portal

With this release, the Lastline Portal will begin enforcing a stronger password policy for all new account passwords. This change applies:

At account creation
When changing the password of an account
When using the password reset functionality to change the password of an account

In all these cases, if the user selects a password that is too weak, the portal will display an informative error message that should assist the user in selecting a better password.

To determine if a password is weak and suggest how it can be improved, we do not just rely on its length and on hard-coded rules on the character classes it contains. Password character composition rules are both cumbersome for users and ineffective at ensuring password strength. Instead, we adopt industry best practices for detecting weak passwords by using the zxcvbn library to estimate password strength.

This change was tracked internally as FEAT-2745

Update sensor to xenial kernel

This release installs a major kernel upgrade for sensor appliances, updating the currently used Ubuntu Trusty kernel (3.3.x) to the kernel distributed in the Ubuntu Xenial (4.4.x). The kernel upgrade brings significant improvements to the kernel network stack, which are particularly relevant on sniffing sensors.

The upgrade will be automatically installed on the sensor but will not disrupt the normal sensor functionality. Runs of lastline_test_appliance will however issue a warning to inform about the availability of a major kernel upgrade. For the upgrade to be effective, a reboot will be required. The appliance will however be fully operational even before the reboot, which can therefore be performed whenever convenient.

This change was tracked internally as FEAT-2728

On-prem deployments can specify higher email threshold when using local SMTP

While using an on-prem local email server for notifications, customers can now specify the maximum number of daily email messages to unlimited.

This change was tracked internally as FEAT-2709

Local callback detection

When a malware sample detected in the protected network is analyzed by detonating it in the Lastline sandbox, this analysis can generate network IoCs. These contain information about domains, IP addresses, or URLs that the malware uses to communicate with the outside world, upgrade itself or receive commands (Command and Control behavior).

With this release, Lastline Sensors automatically make use of these network IoCs to detect the execution of that sample on a host in the protected network.

With this functionality, once a malware sample is analyzed the network IoCs generated are shared with all sensors within the same sensor group and can immediately be used for detection. For customers who have multiple Lastline Sensors protecting their organization, we recommend configuring sensor groups so that these network IoCs can be shared across sensors.

This change was tracked internally as FEAT-2606

ICAP Processing log

An additional log is now available to detail transaction processing information for sensors set up as ICAP responders or as explicit proxies. The log, located in /var/log/c-icap/processing.log, provides detailed information on the analysis decisions taken during the analysis of each HTTP transaction.

The log format contains the following information:

<date> <tz> <icap mode> <icap response> <client IP> -> <destination IP> bytes:<body length> <file hash> <url complete of protocol> score:<score> status:<status> blocked:<blocked>

This change was tracked internally as FEAT-2530

Improved user interface for file submission

With this release, the functionality for submitting files for analysis in the Lastline Portal has been revamped, so users now can:

Select multiple files to be submitted for analysis
Drag and drop files to be analyzed into the page
View the status of multiple submissions directly in the submission page

This change was tracked internally as FEAT-2521

Display labels for the classification of samples in file download and mail attachment views

The display of mail attachments and file downloads in the Lastline Portal now includes additional information on the classification of the samples.

This change extends to a number of views: - mail attachments - mail URLS - file downloads - analysed URLs

Specifically, the tables in these views now have two additional columns:

Av class. This is the general classification of this sample according to antivirus technology, and has values such as "trojan", "ransomware", "adware".
Malware: this is the malware name attributed to this sample

This change was tracked internally as FEAT-2470

Support for sensor groups

With this release, we are introducing support for sensor groups. By configuring a number of sensors to be part of a group, users can enable all of Lastline's current and future correlation functionality to work across sensors within the group. Specifically, configuring a group of sensors has two main effects:

We can correlate detections that happened on different sensors within a group.
We can assume that local network IP addresses observed by different sensors within a group are consistent. That is, if different sensors see a local IP address such as 192.168.1.1, this IP is assumed to refer to the same host.

Users are now able to configure sensor groups in the sensor groups configuration page. Furthermore, sensor group information is now available as an additional column in the existing sensor listing.

This change was tracked internally as FEAT-2347

Backup to private AWS S3 storage

This release extends the existing functionality for storing back ups of Lastline installations to Amazon AWS S3. With this new version, customers can configure backups to be stored to their private S3-compatible storage, instead of using Amazon's cloud-based storage.

This change was tracked internally as FEAT-2308

Upgrade to Suricata 4.0

This release upgrades the version of Suricata used by the sensor to 4.0.1. The upgrade reworks and addresses a number of limitations and inefficiencies in previous versions of the Lastline sensor.

Performance improvements: the adoption of Intel Hyperscan (https://01.org/hyperscan) speeds up matching performance when loading large amounts of rules.
Improvements to SMB2 file extraction: SMB file extraction is now triggered by filestore signatures. Differently from previous releases, only artifacts of relevant filetypes will be extracted for analysis, addressing a number of previously known problems in large network deployments.
Pcap snipping improvements: the generation of network captures in case of signature hits has now become more reliable.

This change was tracked internally as FEAT-2297

Disable TLSv1.0 on Manager/Pinbox/Analyst appliances

Disable TLSv1.0 on customer appliances to enhance the security by removing weak cipher on Manager/Pinbox/Analyst.

This change was tracked internally as FEAT-2284

Leverage malware analysis information and labels in Enterprise product

This release adds a number of features that allow us to make better use of malware analysis information within the Enterprise product.

Specifically, this leverages information on:

the Antivirus class and family of analyzed files or URLs
malware label information based on the network traffic observed within analysis
the specific activities observed

This information was already available within an analysis report, but is now propagated into the Enterprise product, where it can be used for searching within the protected network, by making use of the new "analysis tag" filter.

This release introduces support for filtering based on analysis tags in:

the downloads tab of the portal
the mail attachments and mail urls views of the portal

Similarly, ability to search for these analysis tags throughout the protected network has been added to the search tab of the portal. For this, just select the "Analysis tags" type for the search. Both filtering and search support auto-completion to help users find values to search for.

This allows users to ask general questions such as "which ransomware samples were seen in my network?", as well as much more specific questions such as "which samples were seen in my network exhibiting a specific evasion behavior?".

Finally, the analysis report overview page has been extended with links to the search tab. This allows a user who is viewing an analysis report to quickly identify other samples that detected in the protected network that share a classification or a specific behavior with the sample being viewed.

This change was tracked internally as FEAT-2204

EVE-based parsing of Suricata events

The Suricata event processing pipeline has been completely reworked and now makes full use of the Extensible Event (EVE) Format. The sensor no longer relies on the llidsupload daemon for Suricata event processing, and the daemon is no longer installed on the appliances. The sniffing events archive (previously located in /var/lib/llidsupload/archive) is now moved to /var/lib/suricata-eve/archive/suricata-lastline. Also, all the archived event logs are now in json format following the Suricata EVE standard (http://suricata.readthedocs.io/en/latest/output/eve/eve-json-format.html).

This change was tracked internally as FEAT-2156

Support granular permissions for viewing and managing individual appliances

With this release, we are increasing the granularity of our permissions to support granting permissions to view and manage specific appliances. This change affects the existing permissions:

can_view_appliances: this permission allows viewing information about appliances, such as overall status, configuration, logs and metrics.
can_manage_appliances: this permission allows to perform administrative tasks on an appliance, including installation, configuration and upgrade.

Prior to this change, these two permissions could only be granted for all of a customer's appliances. With this change, they can now also be granted on individual licenses or sensors, providing fine-grained control on which appliances an account can view and manage.

This change was tracked internally as FEAT-1662

Detection Improvements

SENT-840: Sensor has now the capability to extract CSV and SLK files for analysis.
ANREV-4433: Better detection of malware persistency mechanisms installed directly from Microsft Office.
ANREV-4476: Better detection of OSX/BackTrack.
ANREV-4508, ANREV-4522: Better detection of CVE-2018-4878.
ANREV-4525, SIGREPSCAN-456: Better detection for building MSBuild projects from Microsoft Office.
LLADOC-541: Better detection of exploits against Equation Editor OLE objects embedded in Microsoft Office documents.
LLADOC-557: Better detection of remote OLE objects embedded in Microsoft Office documents.
LLAM-3466: Better detection of websites dropping HTA files.
LLAM-3519: Better detection of Cryptojacking.
LLAM-3530 Better detection for launching malicious wscript through ActiveX.
SIGLOGSCAN-239: Better detection of Ebowla payloads.
SIGLOGSCAN-253: Better detection of OSX bitcoin miners.
SIGLOGSCAN-258: Better detection of anomalous use of MMX registers.
SIGLOGSCAN-260: Better detection of samples using Tiny Shell backdoor.
SIGLOGSCAN-261: Better detection of OSX/MaMi family.
SIGLOGSCAN-264: Better detection of Samsam ransomware.
SIGLOGSCAN-266: Better detection of files packed with unregistered versions of Enigma Protector.
SIGREPSCAN-360: Better detection of malware dropping Microsoft Office Add-Ins.
SIGREPSCAN-363: Better detection of drivers dropped by malware.
SIGREPSCAN-433: Better detection for communicating with hosts via TOR proxy servers.
SIGREPSCAN-437: Better detection of malware persistency mechanisms.
SIGREPSCAN-438: Better detection of FTP credential stealing.
SIGREPSCAN-444: More robust detection of corrupted Microsoft Office files.
SIGREPSCAN-461: Better detection of files downloaded via certutil utility.
SIGREPSCAN-466: Better detection of samples using VMProtect.
SIGLOGSCAN-220: Better detection of evasions via known Sandbox mutex names.
SIGLOGSCAN-283: Better detection of evasions via Guest Addition registry key fingerprinting.
SIGREPSCAN-443: Better detection of evasions via bitsadmin utility.
SIGREPSCAN-447: Better detection of evasions via machine serial numbers.
SIGREPSCAN-450: Better detection of evasions using log-on information of the current user.
SIGREPSCAN-452: Better detection of evasions via network information collected via scutil.
SIGREPSCAN-448, SIGREPSCAN-449: Better detection of evasions via network information collected on Mac OS.
LLAM-3440: Better user emulation for Microsoft Office analysis on Mac OS.
SIGLOGSCAN-242: Better handling of evasions via system information gathered from GetLocaleInfo calls.
SIGLOGSCAN-252: Better handling of evasions using system uptime information.
SIGREPSCAN-432: Better handling of evasions via querying Win32_PnpSignedDriver WMI class.
SIGREPSCAN-422: Better handling of obfuscation via dotless IP addresses.
LLADOC-540: More robust extraction of malicious PDFs embedded in RTF documents.
LLADOC-547: Improved handling of malformed RTF documents containing binary data.
SIGREPSCAN-445: Less aggressive classification of installer programs.
ANREV-4490, LLADOC-538: Less aggressive classification of scripts modifying files on disk embedded in Microsoft Office documents.
FEAT-2868: We have introduced new heuristics for the identification of suspicious URLs in email messages. The heuristics focus on the identification of URL patterns that are common in large scale malspam campaigns such as Emotet.
LLADOC-542: More robust extraction of Ole-10-native files from Microsoft Office documents.
LLADOC-543: More robust parsing of RTF files in the prefilter module.
LLADOC-549, LLFILE-400: Improved analysis of URLs in Internet Shortcut files.
LLADOC-551: More robust parsing of invalid XML.
LLFILE-380: Improved file type classification for non-Office files using OpenXML file format.
LLFILE-393: Improved extraction of partially-corrupted 7z archives.
LLFILE-395: Improved analysis of Microsoft Office Spreadsheet-ML files.
LLFILE-399: Improved analysis of Microsoft Office Presentation-ML files.
FEAT-2829: Support the analysis of SYLK (SYmbolic LinKs) files in Lastline Sandbox, to be opened in Excel and other spreadsheet applications.
FEAT-2808: Improved handling of malicious code embedded in CSV files for Microsoft Excel.

Bug Fixes and Improvements

SURI-700: Fix to a problem in the extraction of files over FTP where the file extraction could be affected by anomalous bidirectional data transfers.
SURI-698: Fix to a bug introduced by the sensor 730 sniffing component that would prevent correct processing of certain HTTP transactions.
SENT-822: Fix to a bug that would prevent the correct operation of a sensor inline bridge in case the bridge was assigned an IP address.
SENT-821: Improvement to the sensor file analysis processing logic in case of manager downtime.
SENT-818: Improvement to the "blocking with feedback" mode in ICAP and explicit proxy. The refreshing page used to provide feedback to the customer on the analysis status is now compatible with a wider range of web browsers.
SENT-817: Improvements to CPU allocation for sniffing sensors with large amount of CPU cores.
SENT-790: A major issue was identified in the AF_PACKET support for i40e NICs (Intel X710). A bug in the kernel driver was preventing Suricata from accessing the NIC rings. The problem has been fixed by updating the i40e driver to version 2.4.6.
SENT-779: This release improves the sensor ability to correctly configure the sniffing service in presence of multiple NICs. The required network configuration is now being inferred by checking the NIC model associated to each sniffing interface. The presence of an unused NIC on the appliance will no longer affect the correct configuration of the sensor. This also increases the ability of the sensor to cope with unsupported NICs when AF_PACKET is enabled.
PLTF-201: Improved handling of non-ASCII character encodings.
PLTF-190: Fix bug that could cause HTTP 500 error when filtering mails view with non-ascii email subject
MALS-2591: Better detection of exploits launched from infected websites.
MALS-2368: Extend tool for analyzing submission volume to the analysis system.
LLSHED-48: Improved error handling in sensor upload processing.
LLMAIL-441: Fix to an issue that would cause llmail to generate an unusual amount of segfaults. The segfaults were associated to parsing subprocesses that were designed to fail in case of excessive memory utilization, causing no impact on the product functionality. This fix minimizes the occurrence of such segfaults.
LLMAIL-420: More robust handling of email URL extraction (better handling of non-lowercase schemas)
LLFILE-406: More robust detection of archives containing Mac OS applications/bundles.
LLFILE-405: More robust content-based detection of macro-enabled OpenXML documents.
LLFILE-402: More robust content-based detection of macro-enabled Microsoft Excel spreadsheets.
LLAM-3613: Better analysis of websites hosting exploits and using TLS1.1/TLS1.2.
LLADOC-568: Improve reliability of parsing large XML files in the Lastline document prefilter.
LLADOC-564: More robust handling of large analysis reports for structural document analysis.
FEAT-2828: Fix bug where reports would be empty for mail-only sensors.
FEAT-2827: Daily OS security updates are now scheduled in a way that ensure they will not fail in case an appliance reconfiguration is running at the same time.
FEAT-2703: MTA email analysis: it is now possible to configure via the web UI a list of regular expressions for recipient domains for which the Sensor should accept mails. If the list is non-empty, emails will be accepted only if the recipient domain matches at least one of the regular expressions. If the recipient domain does not match any of the regular expressions, the email will be rejected with a 521 SMTP error. If the domain list is empty, all emails will be accepted for analysis/forwarding.
FEAT-2649: Trigger analysis of documents with Mac-OS-specific macros in Microsoft Office for Mac.
FEAT-2597: The analysis report overview for a file that was analyzed by Lastline now includes a link to search for that specific file being detected in the protected network.
FEAT-2196: Lastline Managers no longer have TCP port 25 open, as this is no longer needed for our architecture, and unnecessarily increased the potential attack surface of our appliances.
FEAT-1823: While sending out the event notifications via e-mail, URLs are now obfuscated using hxxp and not spaces. This has been done to simplify the parsing of event notifications automatically.
FEAT-1714: Extend Lastline Analyst API report to show more information on files inside archive/container files submitted for analysis.

Deprecation of API methods

All methods of the legacy API (/ll_api/ll_api) have been deprecated.

The Lastline API documentation includes a deprecation schedule for methods in the legacy API, as well as information on how to replace usage of these deprecated methods with supported methods.

Released appliance versions

As part of this release, we are making available the following versions of Lastline appliances for use with Lastline Enterprise On-Premises:

Lastline Manager version 900
Lastline Engine version 900
Lastline Sensor version 1001.1
Lastline All-in-one (Pinbox) version 900

Released sandbox images versions

The sandbox images version is now updated to 2018-03-02-01.

Distribution Upgrade

As of version 7.16, support for Ubuntu Precise as the underlying operating system distribution has been discontinued. Before upgrading to version 7.16 or later, appliances that are still on Ubuntu Precise will need to be upgraded to Ubuntu Trusty.

Users can check the distribution in use by an appliance in the Appliance Status view of the portal. The "Base Distribution" listed should be "trusty". If it is "precise", the appliance distribution needs to be upgraded.

For complete information regarding the upgrade process please refer to the Lastline Support Knowledge Base.