Data Node Installation and Administration
This document describes the installation and administration of the Data Node in an On-Premises environment.
About Data Node
The Data Node component receives data records (such as netflow, passive DNS, and webrequest records) collected by the Sensor as well as from third party devices. It stores these data and analyzes it. It returns analysis results to the Manager, which then displays those results.
Supported Hardware
Refer to Hardware Specifications for details about the hardware certified for use with VMware NSX Network Detection and Response appliances.
Deployment Considerations
Data Node may exchange significant volumes of data with other Data Node appliances and with Manager as part of their data storage and analysis functionality. You must consider the following when deploying these appliances in your installation:
-
Data Node and Manager must be able to communicate with each other directly. Using intermediate proxies or NAT devices is not supported.
-
Data Node and Manager should be physically located in close proximity, for example placed on the same rack or at a minimum in the same data center.
-
For resiliency, it is recommended to have three Data Nodes in the cluster. If a Data Node fails, the cluster should remain green with the two remaining Data Nodes and it is recommended to restore functionality as soon as possible for the Data Node in the failed state.
If three Data Nodes for a cluster is not possible or wanted, then the following setups can be used for limited resiliency or no resiliency:
-
Two Data Node cluster: If a Data Node fails, the cluster goes into a degraded state, but it should be possible to reconstruct the data from the remaining node.
-
One Data Node: Provides no resiliency and is suggested only for Proof of Concept (POC) setups or environments with constrained resources.
-
Network Connectivity
Domain Names
Assuming that lastline.example.com
is the
FQDN for the Manager, the server hosting the Data Node needs to be able to connect to:
-
user.lastline.example.com
on TCP port 443. -
log.lastline.example.com
on TCP port 443. To obtain data records from the RabbitMQ broker, access to port 5671 (encrypted channel) and port 5672 (non-encrypted channel) is required. -
update.lastline.example.com
on TCP port 443 and 8443,. -
ntp.lastline.com
on UDP port 123 for time synchronization. It can be replaced with a local NTP server. -
The Data Node needs to access TCP port 9200 and 9300 on every other Data Node appliance in order to create an Elasticsearch cluster. TCP port 9200 is used for REST traffic and TCP port 9300 is used for nodes communication. The Manager must also be able to communicate with the Data Node on TCP port 9200.
You can add FQDNs such as the CDN domain for Google. For further details and information about VMware NSX Network Detection and Response CDN operation, see VMware Knowledge Base article NSX Lastline CDN Usage (900006).
Acquire the Data Node ISO
To install the Data Node, you must download the ISO from VMware.
Install Data Node
The installation process for the Data Node consists of three steps. In the first step, the base system is installed. In the second step, basic configuration information is collected and the configuration is applied to the system. In the final step, required data is retrieved from the VMware backend servers.
Base System Installation
The Data Node uses Ubuntu Server 18.04 (Bionic distribution) as its underlying operating system. Therefore, many of the steps of the installation are similar to the ones required to install Ubuntu Server. Refer to the Ubuntu guide, Installing Ubuntu 18.04.
Many of the steps involved in a standard Ubuntu installation have been automated and hidden from the Data Node Installer.
If you are running an existing installation with appliances based on an earlier Ubuntu release, you should upgrade to a version based on Bionic. To upgrade to Bionic from Xenial, you must first update the Data Node to the last version that supports Xenial (see the release notes for your specific version, and then follow the instructions on the linked support article).
Install on VMware ESXi
Before you install the Data Node on VMware ESXi, you must ensure the VM meets the minimum hardware specifications for the class of appliance. See Hardware Specifications for details. Ensure that the base hardware runs on an Intel CPU.
Using the VMware ESXi vSphere client 7.0 update 3, create a new virtual machine and configure it to meet the requirements of the Data Node.
Registration and Configuration
Before you can configure Data Node for an On-Premises installation, you must have previously installed and configured the Manager. The Manager must be on-line and reachable.
For a hosted installation using the NSX Cloud, the User Portal must be accessible at https://user.lastline.com/ (for EMEA customers https://user.emea.lastline.com/).
To register and apply the software configuration to the Data Node, you must login to the server console.
Register the Data Node
The registration process runs some tests to check hardware compatibility. The configuration is then applied to the machine. This process may take a while (20-40 minutes) depending on your network connectivity and system characteristics.
After the completed prompt is displayed, select <Ok>
or press
Enter to
exit from the registration process.
Re-registration
If the Data Node needs to be replaced or reinstalled, the existing appliance needs to be deregistered first before your new registration will succeed.
Delete the Data Node
Before you can successfully delete the Data Node from the User Portal it must be offline. The easiest way to do this is to login to the appliance and shut it down.
To delete the Data Node, it needs to be offline and deregistered.
Administer the Data Node
The Data Node was developed to require as little maintenance and administration as possible.
The following topics describe how to customize and configure some of the advanced features of the Data Node.
Configuration Tool
Use the VMware
NSX Network Detection and
Response configuration tool, lastline_setup
, to administer and manage the Data Node.
If you encounter an error running any of the lastline_setup
command options, make a note of
the error message returned and contact VMware Support.
Network Configuration
You can easily change the network configuration of the Data Node. This may be needed if its assigned IP address changes (for example, upon a reconfiguration of the network).
Reconfigure for DHCP
To enable a network configuration using DHCP, use the network
option of
the lastline_setup
command.
Reconfigure for Static Addressing
To enable a network configuration using a static IP, you must provide values for the
address, netmask, gateway, and
dns_nameservers parameters. Use the network
options of
the lastline_setup
command to make these changes.
Update On-Premises Manager FQDN
If you had selected "Use On-Premises Manager" during the registration and configuration of the Data Node and the FQDN of the Manager changes, you must update the FQDN information to ensure your appliances can continue to successfully communicate.
This process does not allow you to move appliances from one Manager to another.
If the Manager is deployed in an active-standby configuration, you must use the configured virtual IP address, either taken from DNS or using the address directly.
Enable the monitoring user
The Data Node has a
monitoring user who can access the system using console or via SSH
(password only without using the SSH key). To enable the monitoring user, use the
monitoring_user_password
option of the lastline_setup
command.
Once the monitoring user is enabled, you can SSH to the Data Node using that account:
server# ssh monitoring@ip_appliance
monitoring@ip_appliance's password:
...
monitoring@lastline-manager:~$
Enable Password-Based SSH Authentication
The Data Node supports specifying
users who can access the system using console or via SSH (password only without using the
SSH key). To enable existing users to authenticate with password-based SSH use the
enable_additional_password_auth_ssh_usernames
option of the
lastline_setup
command.
Once the user has been added, you can SSH to the Data Node using that account:
server# ssh ghopper@ip_appliance
ghopperg@ip_appliance's password:
...
ghopper@lastline-manager:~$
Enable the Home Network
The Home network enables additional correlation rules and simplifies distinguishing internal devices from external ones. We recommend you configure the Home network for your installation.
If you do not configure a home network, the system defaults to RFC1918
ranges (private ranges such as 10.x.x.x
and
192.168.x.x
).
Disable Automatic Updates
VMware periodically releases appliance updates or hotfixes. By default, automatic updates are enabled on newly installed appliances. As long as the appliance has automatic updates enabled, these updates and fixes will transparently be applied to the system.
If you prefer to manually update the Data Node, follow these steps to disable automatic updates.
Manual Updates
If you have disabled automatic updates for your appliances you must apply updates and hotfixes manually.
Follow these steps to manually update an appliance.
About Hardening
During the development process, steps were taken to lock down the Data Node by default to help reduce any attack surfaces. These include:
-
Default Applications — All unnecessary applications included in the base Ubuntu server build have been removed from the system. What remains are the libraries and applications necessary for the normal functioning, routine maintenance, and troubleshooting of the Data Node.
-
Default Firewall — The Data Node image comes with Uncomplicated FIrewall (UFW) installed and configured to restrict inbound access to the system.
-
Security Patches — The system will install daily OS security updates by default. You can disable automatic updates.
-
Least privilege — VMware has taken care to ensure a paradigm of least privilege regarding the permissions of services and file system access.
-
Secure SSH — SSH is configured to use certificate-based authentication by default.
-
TLS encryption — Communications between the appliances are TLS encrypted.
Harden the Data Node
We recommend the following guidelines for hardening the Data Node after installation. These steps are not required, but they will allow you to further restrict access to your VMware NSX Network Detection and Response appliances.
Hardware Specifications
The hardware certified for use with VMware NSX Network Detection and Response appliances is listed below:
Dell Hardware
Supported Dell Hardware
Manager | |
---|---|
Server Model | Dell PowerEdge R450 |
CPU Type |
|
CPU Quantity | 1 CPU |
Minimum RAM | 96 GB |
RAID Controller |
Dell EMC PowerEdge RAID Controller (PERC) H745/H755 (with flash-backed cache) |
RAID Configuration |
RAID 10 Note: If the Dell website does not allow RAID 10 configuration from
factory, purchase the server with RAID unconfigured and then manually create a RAID 10
virtual volume before software installation.
|
Persistent Storage | Recommended: 4 × 4 TB HDDs |
Additional Network Card | None |
Redundant Power Supply | Recommended for reliability |
iDRAC9 Enterprise | Recommended for remote management and installation |
Data Node | |
---|---|
Server Model | Dell PowerEdge R450 |
CPU Type |
|
CPU Quantity | 1 CPU |
Minimum RAM | 96 GB |
RAID Controller |
Dell EMC PowerEdge RAID Controller (PERC) H745/H755 (with flash-backed cache) |
RAID Configuration |
RAID 10 Note: If the Dell website does not allow RAID 10 configuration from
factory, purchase the server with RAID unconfigured and then manually create a RAID 10
virtual volume before software installation.
|
Persistent Storage | Recommended: 4 × 2 TB 10k RPM HDDs |
Additional Network Card | None |
Redundant Power Supply | Recommended for reliability |
iDRAC9 Enterprise | Recommended for remote management and installation |
Engine | |
---|---|
Server Model | Dell PowerEdge R450 |
CPU Type |
|
CPU Quantity | 1 CPU |
Minimum RAM |
128 GB Recommended: 4 GB per CPU virtual core |
RAID Controller |
Dell EMC PowerEdge RAID Controller (PERC) H745/H755 (with flash-backed cache) |
RAID Configuration | RAID 1 |
Persistent Storage | Minimum: 2 × 1 TB HDDs |
Additional Network Card | None |
Redundant Power Supply | Recommended for reliability |
iDRAC9 Enterprise | Recommended for remote management and installation |
Sensor — 1G Networks | |
---|---|
Server Model | Dell PowerEdge R450 |
CPU Type |
|
CPU Quantity | 1 CPU |
Minimum RAM | 64 GB |
RAID Controller |
Dell EMC PowerEdge RAID Controller (PERC) H745/H755 (with flash-backed cache) |
RAID Configuration | RAID 1 |
Persistent Storage | Minimum: 2 × 1 TB HDDs |
Additional Network Card | Intel i350 Quad Port 1GbE |
Redundant Power Supply | Recommended for reliability |
iDRAC9 Enterprise | Recommended for remote management and installation |
Sensor — 10G Networks | |
---|---|
Server Model | Dell PowerEdge R450 |
CPU Type |
|
CPU Quantity | 2 CPUs |
Minimum RAM | 128 GB |
RAID Controller |
Dell EMC PowerEdge RAID Controller (PERC) H745/H755 (with flash-backed cache) |
RAID Configuration | RAID 1 |
Persistent Storage | Minimum: 2 × 1 TB HDDs |
Additional Network Card | Intel X710 Dual Port 10GbE |
Redundant Power Supply | Recommended for reliability |
iDRAC9 Enterprise | Recommended for remote management and installation |
Previously Supported Dell Hardware
The following Dell hardware are no longer supported.
Manager | |
---|---|
Server Model | Dell PowerEdge R440 |
Chassis Type | Chassis with Hot-plug Hard Drives |
CPU Type | Intel® Xeon® Silver 4114 — or better (minimum 12 threads/cores) |
CPU Quantity | 1 CPU |
Minimum RAM | 64 GB ECC RAM |
RAID Controller | HW RAID10 |
RAID Configuration |
|
Minimum Persistent Storage | 4 × 2 TB 7.2K RPM SATA 6Gbps 3.5in |
Power Supply | Dual Hot-plug Power — Optional |
iDRAC9 Enterprise | Optional |
ProSupport Service Plan | Optional |
Data Node | |
---|---|
Server Model | Dell PowerEdge R440 |
Chassis Type | Chassis with Hot-plug Hard Drives |
CPU Type | Intel® Xeon® Silver 4114 — or better (minimum 24 threads/cores) |
CPU Quantity | 1 CPU |
Minimum RAM | 64 GB ECC RAM |
RAID Controller | HW RAID10 |
RAID Configuration |
|
Minimum Persistent Storage | 2 × 1 TB SATA HDD |
Power Supply | Dual Hot-plug Power — Optional |
iDRAC9 Enterprise | Optional |
ProSupport Service Plan | Optional |
Engine | |
---|---|
Server Model | Dell PowerEdge R440 |
Chassis Type | Chassis with Hot-plug Hard Drives |
CPU Type | Intel® Xeon® Silver 4114 — or better (minimum 20 threads/cores) |
CPU Quantity | 1 CPU |
Minimum RAM | 96 GB ECC RAM |
RAID Controller | HW RAID10 |
RAID Configuration |
|
Minimum Persistent Storage | 2 × 1 TB SATA HDD |
Power Supply | Dual Hot-plug Power — Optional |
iDRAC9 Enterprise | Optional |
ProSupport Service Plan | Optional |
Sensor — 1G Networks | |
---|---|
Server Model | Dell PowerEdge R440 |
Chassis Type | Chassis with Hot-plug Hard Drives |
CPU Type | Intel® Xeon® Silver 4114 — or better (minimum 20 threads/cores) |
CPU Quantity | 1 CPU |
Minimum RAM | 32 GB ECC RAM |
RAID Controller | HW RAID10 |
RAID Configuration |
|
Minimum Persistent Storage | 2 × 1 TB SATA (7.2K RPM) HDD |
Power Supply | Dual Hot-plug Power — Optional |
Network Card | Intel Ethernet I350 Quad-Port 1Gb Server Adapter |
iDRAC9 Enterprise | Optional |
ProSupport Service Plan | Optional |
Sensor — 10G Networks | |
---|---|
Server Model | Dell PowerEdge R440 |
Chassis Type | Chassis with Hot-plug Hard Drives |
CPU Type | Intel® Xeon® Silver 4114 — or better (minimum 20 threads/cores) |
CPU Quantity | 2 CPUs |
Minimum RAM | 128 GB ECC RAM |
RAID Controller | HW RAID10 |
RAID Configuration |
|
Minimum Persistent Storage | 2 × 1 TB SATA (7.2K RPM) HDD |
Power Supply | Dual Hot-plug Power — Optional |
Network Card | Intel Ethernet X710-DA2 10Gbps network card |
iDRAC9 Enterprise | Optional |
ProSupport Service Plan | Optional |
All-In-One | |
---|---|
Server Model | Dell PowerEdge R440 |
Chassis Type | Chassis with Hot-plug Hard Drives |
CPU Type | Intel® Xeon® Silver 4114 — or better (minimum 20 threads/cores) |
CPU Quantity | 2 CPUs |
Minimum RAM | 128 GB ECC RAM |
RAID Controller | HW RAID10 |
RAID Configuration |
|
Minimum Persistent Storage | 4 × 2 TB 7.2K RPM SATA 6Gbps 3.5in |
Power Supply | Dual Hot-plug Power — Optional |
Network Card | Intel Ethernet X710-DA2 10Gbps network card |
iDRAC9 Enterprise | Optional |
ProSupport Service Plan | Optional |
Analyst | |
---|---|
Server Model | Dell PowerEdge R440 |
Chassis Type | Chassis with Hot-plug Hard Drives |
CPU Type | Intel® Xeon® Silver 4114 — or better (minimum 12 threads/cores) |
CPU Quantity | 1 CPU |
Minimum RAM | 96 GB ECC RAM |
RAID Controller | HW RAID10 |
RAID Configuration |
|
Minimum Persistent Storage | 4 × 2 TB 7.2K RPM SATA 6Gbps 3.5in |
Power Supply | Dual Hot-plug Power — Optional |
iDRAC9 Enterprise | Optional |
ProSupport Service Plan | Optional |
HPE Hardware
- Manager
-
Intel® Xeon® Silver 4114 2.2GHZ
64 GB RAM
4 × 2 TB in RAID 10 (6 Gbps SATA)
On-board NIC
- Data Node
-
Intel® Xeon® Silver 4114 2.2GHZ
64 GB RAM
4 × 2 TB in RAID 10 (SAS 10K RPM)
On-board NIC
- Engine
-
Intel® Xeon® Silver 4114 2.2GHZ
96 GB RAM
2 × 2 TB HDDs in RAID 1 (6 Gbps SATA)
On-board NIC
- Sensor — 1G Networks
-
Intel® Xeon® Silver 4114 2.2GHZ
32 GB RAM
2 × 2 TB HDDs in RAID 1 (6 Gbps SATA)
Intel I350 Quad port (or HPE 366T)
- Sensor — 10G Networks
-
2 × Intel® Xeon® Silver 4114 2.2GHZ
128 GB RAM
2 × 2 TB HDDs in RAID 1 (6 Gbps SATA)
Intel X710-DA2
- Analyst
-
2 × Intel® Xeon® Silver 4114 2.2GHZ
128 GB RAM
2 × 2 TB HDDs in RAID 1 (6 Gbps SATA)
On-board NIC