Kibana data structures

Kibana is an Elastic product that visualizes data stored in Elasticsearch. The VMware NSX Network Detection and Response currently uses Kibana version 6.8.4. Refer to the Kibana User Guide for detailed information. This page documents the Kibana data structures that are specific to the VMware NSX Network Detection and Response.

Indexes

Data is stored in Elasticsearch in per-day and per-record-type indexes. An index-pattern in Kibana allows users to query one or multiple indexes. The following index patterns are predefined:

  • dhcp-* — Includes indexes that store DHCP records.

  • krb-* — Includes indexes that store Kerberos records.

  • netflow-* — Includes indexes that store netflow records.

  • pdns-* — Includes indexes that store passive DNS records.

  • rdp-* — Includes indexes that store RDP records.

  • smb-* — Includes indexes that store SMB records.

  • tls-* — Includes indexes that store TLS records.

  • webrequest-* — Includes indexes that store web request records.

  • all — Includes all indexes (default index pattern).

The default data retention value for these indexes is 30 days. It is not configurable.

DHCP record types

A DHCP document represents a DHCP message.

Field name Type Description
dhcp.assigned_ip ip The IP address currently assigned to the client.
dhcp.client_id string The identifier of the client.
dhcp.client_ip ip The IP address currently assigned to the client.
dhcp.client_mac string The MAC address of the client.
dhcp.dhcp_type string The DHCP operations phase. The phases are offer, ack, request, or discover.
dhcp.id string The transaction cookie.
dhcp.lease_time string The length of the offered lease in seconds.
dhcp.next_server_ip string The IP address of the server containing the client bootstrap file.
dhcp.params parameter request list A list of configuration parameters the client requests from the server. These are strings: dns_server, ntp_server, router, and subnet_mask.
dhcp.rebinding_time date The timeout for the client to request a lease rebinding.
dhcp.relay_ip ip The IP address of the relay agent.
dhcp.renewal_time date The timeout for the client to request a lease renewal.
dhcp.requested_ip ip The IP address the client accepted from the server's offer.
dhcp.subnet_mask ip The subnet mask.
dhcp.type string Indicates if the transaction is a request or a reply.
direction string The direction of the record. to_server is a client request/reply. The client may be responding to a server's offer. to_client is the server response.
dst_hostname string The host name for dst_ip.
dst_ip ip The destination IP address.
dst_port number The destination port.
dst.homenet boolean Indicates if the destination IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
dst.host_tags string Contains the list of host tags that are assigned to the destination IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
network.community_id string Stores a hash of the source and destination IPs and ports, as well as the protocol used in the communication. This is a tool-agnostic standard to identify flows. In this implementation, the (default) value of 0 is used as the seed in the hashing algorithm.
observer.ingress.interface.name string Stores the name of the sensor interface where the activity corresponding to the NTA record was observed.
observer.name string The custom name of the sensor that generated the record. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
proto string The protocol used for the record: UDP.
source string The license of the sensor that uploaded the passive DNS record.
src_hostname string The hostname for src_ip.
src_ip ip The source IP address.
src_port number The source port.
src.homenet boolean Indicates if the source IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
src.host_tags string Contains the list of host tags that are assigned to the source IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
ts_start date The start timestamp of the DNS request.

Example dhcp.json:

{
  "dhcp": {
    "assigned_ip": "0.0.0.0",
    "client_id": "00:0b:82:01:fc:42",
    "client_ip": "0.0.0.0",
    "client_mac": "00:0b:82:01:fc:42",
    "dhcp_type": "discover",
    "id": 15645,
    "lease_time": null,
    "next_server_ip": null,
    "params": [
      "dns_server",
      "ntp_server",
      "router",
      "subnet_mask"
    ],
    "rebinding_time": null,
    "relay_ip": null,
    "renewal_time": null,
    "requested_ip": "0.0.0.0",
    "subnet_mask": null,
    "type": "request"
  },
  "direction": "to_server",
  "dst_ip": "255.255.255.255",
  "dst_port": 67,
  "dst.homenet": true,
  "dst.host_tags": [],
  "network.community_id": "1:t9O1j0qj71O4wJM7gnaHtgmfev8=",
  "observer.ingress.interface.name": "ens192",
  "observer.name": "sensor1",
  "proto": "UDP",
  "source": "ABCDEFGHIJ0123456789:sensor1",
  "src_ip": "0.0.0.0",
  "src_port": 68,
  "src.homenet": true,
  "src.host_tags": [],
  "ts_start": 1420655017000
}

Kerberos record types

A KRB document represents a message in the Kerberos protocol.

Field name Type Description
client.geo.country_iso_code string If the IP address contained in the src_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
client.geo.location geo_point If the IP address contained in the src_ip field can be geo-located, the latitude and longitude is provided.
dst_hostname string The host name for dst_ip.
dst_ip ip The destination IP address.
dst_port number The destination port.
dst.homenet boolean Indicates if the destination IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
dst.host_tags string Contains the list of host tags that are assigned to the destination IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
geoip geoip The geoip information for dst_ip. This field is deprecated, no longer used, and will be removed in the future.
kerberos_cname string The client name.
kerberos_encryption string The encryption used.
kerberos_error_code string The error code.
kerberos_failed_request string The request that failed.
kerberos_msg_type string The message type.
kerberos_realm string The Kerberos realm.
kerberos_sname string The server name.
kerberos_weak_encryption boolean Whether weak encryption was used.
network.community_id string Stores a hash of the source and destination IPs and ports, as well as the protocol used in the communication. This is a tool-agnostic standard to identify flows. In this implementation, the (default) value of 0 is used as the seed in the hashing algorithm.
observer.ingress.interface.name string Stores the name of the sensor interface where the activity corresponding to the NTA record was observed.
observer.name string The custom name of the sensor that generated the record. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
server.geo.country_iso_code string If the IP address contained in the dst_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
server.geo.location geo_point If the IP address contained in the dst_ip field can be geo-located, the latitude and longitude is provided.
severity int A severity score associated to the KRB record. Currently unused.
source string The license of the sensor that uploaded the KRB record.
src_hostname string The hostname for src_ip.
src_ip ip The source IP address.
src_port number The source port.
src.homenet boolean Indicates if the source IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
src.host_tags string Contains the list of host tags that are assigned to the source IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
tags string List of tags associated to the KRB message. Currently unused.
ts_start date The start timestamp of the KRB request.

Example krb.json:

{
  "dst_ip": "192.168.1.254",
  "dst_hostname": null,
  "dst_port": 88,
  "dst.homenet": true,
  "dst.host_tags": [],
  "geoip": null,
  "kerberos_error_code": "KDC_ERR_PREAUTH_REQUIRED",
  "kerberos_cname": "",
  "kerberos_encryption": null,
  "kerberos_failed_request": "KRB_AS_REQ",
  "kerberos_msg_type": "KRB_ERROR",
  "kerberos_realm": "",
  "kerberos_sname": [
    "krbtgt/EXAMPLE.COM"
  ],
  "kerberos_weak_encryption": false,
  "network.community_id": "1:ftStasf+GJdjuwzOVMYqfIU1eN8=",
  "observer.ingress.interface.name": "ens192",
  "observer.name": "sensor1",
  "severity": null,
  "source": "ABCDEFGHIJ0123456789:sensor1",
  "src_hostname": null,
  "src_ip": "192.168.1.1",
  "src_port": 65044,
  "src.homenet": true,
  "src.host_tags": [],
  "tags": [],
  "ts_start": 1574841016000
}

Netflow record types

A netflow document represents a bidirectional flow between two endpoints.

Field name Type Description
app_layer_protocol string The application layer protocol corresponding to this flow, if detected.
bytes_in number The number of inbound bytes.
bytes_out number The number of outbound bytes.
client.geo.country_iso_code string If the IP address contained in the src_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
client.geo.location geo_point If the IP address contained in the src_ip field can be geo-located, the latitude and longitude is provided.
dns_name string The resolved DNS name.
dst_hostname string The host name for dst_ip.
dst_ip ip The destination IP address.
dst_port number The destination port.
fins_in number The number of inbound packets with the FIN flag set.
fins_out number The number of outbound packets with the FIN flag set.
geoip.country_code2 string The country code for the geolocation of dst_ip, if available. This field is deprecated, no longer used, and will be removed in the future.
geoip.location geo_point The coordinates for the geolocation of dst_ip, if available (comma separated). This field is deprecated, no longer used, and will be removed in the future.
network.community_id string Stores a hash of the source and destination IPs and ports, as well as the protocol used in the communication. This is a tool-agnostic standard to identify flows. In this implementation, the (default) value of 0 is used as the seed in the hashing algorithm.
observer.ingress.interface.name string Stores the name of the sensor interface where the activity corresponding to the NTA record was observed.
observer.name string The custom name of the sensor that generated the record. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
packets_in number The number of inbound packets.
packets_out number The number of outbound packets.
payload_bytes_signature number The sizes of the first 10 packets in the flow, if available.
proto string The transport protocol of the flow.
rsts_in number The number of inbound packets with the RST flag set.
rsts_out number The number of outbound packets with the RST flag set.
server.geo.country_iso_code string If the IP address contained in the dst_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
server.geo.location geo_point If the IP address contained in the dst_ip field can be geo-located, the latitude and longitude is provided.
severity number A severity score associated to the flow. Currently unused.
source string The license of the sensor that uploaded the flow.
src_hostname string The hostname for src_ip.
src_ip ip The source IP address.
src_port number The source port.
syns_in number The number of inbound packets with the SYN flag set.
syns_out number The number of outbound packets with the SYN flag set.
tags string List of tags associated to the flow. Currently unused.
ts_start date The start timestamp of the flow.

Example netflow.json:

{
  "app_layer_protocol": "HTTP",
  "bytes_in": 3307,
  "bytes_out": 5028,
  "dns_name": "example.com",
  "dst_ip": "93.184.216.34",
  "dst_port": 80,
  "fins_in": 0,
  "fins_out": 0,
  "geoip": null,
  "network.community_id": "1:v7RHer6LFhTBW/ilNzB+LwBXekE=",
  "observer.ingress.interface.name": "ens192",
  "observer.name": "sensor1",
  "packets_in": 10,
  "packets_out": 11,
  "payload_bytes_signature": [0],
  "proto": "TCP",
  "rsts_in": 0,
  "rsts_out": 0,
  "severity": null,
  "source": "ABCDEFGHIJ0123456789:sensor1",
  "src_ip": "192.168.1.1",
  "src_port": 39812,
  "syns_in": 0,
  "syns_out": 0,
  "tags": [],
  "ts_start": 1562230242000
}

Passive DNS record types

A passive DNS document represents a DNS request/response.

Field name Type Description
client.geo.country_iso_code string If the IP address contained in the src_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
client.geo.location geo_point If the IP address contained in the src_ip field can be geo-located, the latitude and longitude is provided.
dns.question.class string The class of records being queried.
dns.question.type string The type of record being queried. This is the mnemonic of the value stored in the rrtype field.
dns.response_code string The DNS response code. This is the name of the value stored in the error field.
dst_ip ip The destination IP address.
dst_port number The destination port.
dst.homenet boolean Indicates if the destination IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
dst.host_tags string Contains the list of host tags that are assigned to the destination IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
error number The response error code, if any.
geoip.country_code2 string The country code for the geolocation of dst_ip, if available. This field is deprecated, no longer used, and will be removed in the future.
geoip.lat number The latitude coordinates for the geolocation of dst_ip, if available. This field is deprecated, no longer used, and will be removed in the future.
geoip.location geo_point The coordinates for the geolocation of dst_ip, if available (comma separated). This field is deprecated, no longer used, and will be removed in the future.
geoip.lon number The longitude coordinates for the geolocation of dst_ip, if available. This field is deprecated, no longer used, and will be removed in the future.
n number The number of occurrences of this record.
network.community_id string Stores a hash of the source and destination IPs and ports, as well as the protocol used in the communication. This is a tool-agnostic standard to identify flows. In this implementation, the (default) value of 0 is used as the seed in the hashing algorithm.
observer.ingress.interface.name string Stores the name of the sensor interface where the activity corresponding to the NTA record was observed.
observer.name string The custom name of the sensor that generated the record. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
qclass number The query class.
rdata string The response data.
rrclass number The resource record class.
rrname string The resource record name.
rrtype conflict The resource record type.
server.geo.country_iso_code string If the IP address contained in the dst_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
server.geo.location geo_point If the IP address contained in the dst_ip field can be geo-located, the latitude and longitude is provided.
severity number A severity score associated to the DNS request/response. Currently unused.
source string The license of the sensor that uploaded the passive DNS record.
src_ip ip The source IP address.
src_port number The source port.
src.homenet boolean Indicates if the source IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
src.host_tags string Contains the list of host tags that are assigned to the source IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
tags string List of tags associated to the DNS request/response. For example, cdn-network.
ts_start date The start timestamp of the DNS request.
ttl number The TTL of the response.

Example pdns.json:

{
  "dns.question.class": "IN",
  "dns.question.type": "A",
  "dns.response_code": "NOERROR",
  "dst_ip": "4.2.2.2",
  "dst_port": 53,
  "dst.homenet": false,
  "dst.host_tags": [],
  "error": null,
  "geoip": null,
  "n": 1,
  "network.community_id": "1:aG/c2ro/zSr0+ThoXlw3kqx9/48=",
  "observer.ingress.interface.name": "ens192",
  "observer.name": "sensor1",
  "proto": "UDP",
  "rdata": [
    "172.232.11.155"
    "172.232.11.171",
  ],
  "rrname": "a1089.dscd.akamai.net",
  "rrtype": 1,
  "server": {
    "geo": {
      "country_iso_code": "US",
      "location": {
        "lat": 37.751
        "lon": -97.822,
      }
    }
  },
  "severity": null,
  "source": "ABCDEFGHIJ0123456789:sensor1",
  "src_ip": "192.168.100.254",
  "src_port": 0,
  "src.homenet": true,
  "src.host_tags": [],
  "tags": [
    "cdn-network"
  ],
  "ts_start": 1589195253000,
  "ttl": [
    1,
    1
  ]
}

RDP record types

An RDP record represents a message in the Microsoft Remote Desktop Protocol (RDP).

Field name Type Description
direction string The communication direction (to_server or to_client).
dst_hostname string The host name for dst_ip.
dst_ip ip The destination IP address.
dst_port number The destination port.
dst.homenet boolean Indicates if the destination IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
dst.host_tags string Contains the list of host tags that are assigned to the destination IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
network.community_id string Stores a hash of the source and destination IPs and ports, as well as the protocol used in the communication. This is a tool-agnostic standard to identify flows. In this implementation, the (default) value of 0 is used as the seed in the hashing algorithm.
observer.ingress.interface.name string Stores the name of the sensor interface where the activity corresponding to the NTA record was observed.
observer.name string The custom name of the sensor that generated the record. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
proto string The protocol used for the record: TCP.
rdp.channels string The requested data channel names.
rdp.client.build string The client OS and SP level.
rdp.client.capabilities string The client capabilities.
rdp.client.client_name string The client name.
rdp.client.color_depth string The numeric color depth supported.
rdp.client.connection_hint string The client connection hint.
rdp.client.desktop_height string The client desktop height value in pixels.
rdp.client.desktop_orientation string The client desktop orientation.
rdp.client.desktop_width string The client desktop width value in pixels.
rdp.client.device_scale_factor string The scale factor of the client desktop.
rdp.client.function_keys string The number of function keys.
rdp.client.id string The client product ID.
rdp.client.ime string The client input method editor (IME) file name.
rdp.client.keyboard_layout string The client keyboard layout.
rdp.client.keyboard_subtype string The client keyboard subtype.
rdp.client.keyboard_type string The client keyboard type.
rdp.client.physical_height string The physical height of the client display.
rdp.client.physical_width string The physical width of the client display.
rdp.client.product_id string The product ID of the client.
rdp.client.scale_factor string The scale factor of the client display.
rdp.client.serial_number string The client serial number.
rdp.client.version string The client version.
rdp.cookie string The RDP cookie.
rdp.error_code string The error code (in the event of an error).
rdp.event_type string The event type. For example, initial_request, initial_response, tls_handshake, connect_response, or connect_request.
rdp.flags string The RDP flags.
rdp.protocol string The RDP protocol. For example, hybrid, hybrid_ex, ssl, or rdp.
rdp.reason string The error explanation (in the event of an error).
rdp.server_supports list A list of features supported by the server.
rdp.tx_id string The flow ID.
rdp.x509_serials list A list of observed certificate serial numbers.
server.geo.country_iso_code string If the IP address contained in the dst_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
server.geo.location geo_point If the IP address contained in the dst_ip field can be geo-located, the latitude and longitude is provided.
source string The license of the sensor that observed the RDP traffic.
src_hostname string The hostname for src_ip.
src_ip ip The source IP address.
src_port number The source port.
src.homenet boolean Indicates if the source IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
src.host_tags string Contains the list of host tags that are assigned to the source IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
ts_start date The start timestamp of the RDP record.

Example rdp.json:

{
  "direction": "to_server",
  "dst_hostname": null,
  "dst_ip": "172.18.21.26",
  "dst_port": 3389,
  "dst.homenet": true,
  "dst.host_tags": [
    "legacy_asset"
  ],
  "network.community_id": "1:2KNqH2pOIWtoJLGW15pe24UfzeE=",
  "observer.ingress.interface.name": "ens192",
  "observer.name": "sensor1",
  "proto": "TCP",
  "rdp": {
    "channel": null,
    "client": {
      "build": "Windows 10",
      "capabilities": [
        "support_errinfo_pdf"
      ],
      "client_name": "x1810",
      "color_depth": 24,
      "connection_hint": null,
      "desktop_height": 600,
      "desktop_orientation": null,
      "desktop_width": 800,
      "device_scale_factor": null,
      "function_keys": 12,
      "id": null,
      "ime": null,
      "keyboard_layout": "en-US",
      "keyboard_subtype": null,
      "keyboard_type": "enhanced",
      "physical_height": null,
      "physical_width": null,
      "product_id": 1,
      "scale_factor": null,
      "serial_number": null,
      "version": "v5"
    },
    "cookie": null,
    "error_code": null,
    "event_type": "connect_request",
    "flags": null,
    "protocol": null,
    "reason": null,
    "server_supports": null,
    "tx_id": 2,
    "x509_serials": null
  },
  "source": "ABCDEFGHIJ0123456789:sensor1",
  "src_hostname": null,
  "src_ip": "172.18.21.19",
  "src_port": 22034,
  "src.homenet": true,
  "src.host_tags": [
    "user_domain"
  ],
  "ts_start": 1589790310000
}

SMB record types

An SMB document represents a message in the Server Message Block (SMB) protocol.

Field name Type Description
client.geo.country_iso_code string If the IP address contained in the src_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
client.geo.location geo_point If the IP address contained in the src_ip field can be geo-located, the latitude and longitude is provided.
command string The operation code that is requested by the client or responded to by the server.
create_create_ts date In a CREATE message, the creation timestamp.
create_delete_on_close boolean In a CREATE message, whether the file should be automatically deleted when the last open request on this file is closed.
create_disposition string In a CREATE message, defines the action the server MUST take if the file that is specified in the name field already exists. Valid values include:
  • FILE_SUPERSEDE
  • FILE_OPEN
  • FILE_CREATE
  • FILE_OPEN_IF
  • FILE_OVERWRITE
  • FILE_OVERWRITE_IF
create_fuid string In a CREATE message, the file GUID (SMB2+) or FID (SMB1).
create_is_directory boolean In a CREATE message, whether the file is a directory.
create_last_access_ts date In a CREATE message, the last access time.
create_last_changed_ts date In a CREATE message, the last changed time.
create_last_modified_ts date In a CREATE message, the last modified time.
create_path string In a CREATE message, the create path.
create_size number In a CREATE message, the file size.
dcerpc_bind_request_interfaces_ack_reason number In a DCERPC BIND request, the ack reason.
dcerpc_bind_request_interfaces_ack_result number In a DCE/RPC BIND request, the ack result.
dcerpc_bind_request_interfaces_uuid string In a DCE/RPC BIND request, the bind interfaces UUIDs.
dcerpc_bind_request_interfaces_version_major number In a DCE/RPC BIND request, the bind interfaces major version.
dcerpc_bind_request_interfaces_version_minor number In a DCE/RPC BIND request, the bind interfaces minor version.
dcerpc_call_id number The call ID.
dcerpc_generic_request_frag_cnt number In a DCE/RPC generic request, the number of fragments for the stub data.
dcerpc_generic_request_opnum number In a DCE/RPC generic request, the opnum.
dcerpc_generic_request_stub_data_size number In a DCE/RPC generic request, the total stub data size.
dcerpc_generic_response_frag_cnt number In a DCERCP generic response, the number of fragments for the stub data.
dcerpc_generic_response_stub_data_size number In a DCERCP generic response, the total stub data size.
dcerpc_request_cmd string In a DCE/RPC request, the command.
dcerpc_response_cmd string In a DCE/RPC response, the command.
dialect string The SMB dialect.
dst_hostname string The host name for dst_ip.
dst_ip ip The destination IP address.
dst_port number The destination port.
dst.homenet boolean Indicates if the destination IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
dst.host_tags string Contains the list of host tags that are assigned to the destination IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
file_event_filename string The filename.
file_event_fuid string The file UID.
file_event_share string The share.
geoip geoip The geoip information for dst_ip. This field is deprecated, no longer used, and will be removed in the future.
negotiate_client_dialects string In a NEGOTIATE message, the client dialects.
negotiate_client_guid string In a NEGOTIATE message, the client GUID.
negotiate_server_guid string In a NEGOTIATE message, the client GUID.
network.community_id string Stores a hash of the source and destination IPs and ports, as well as the protocol used in the communication. This is a tool-agnostic standard to identify flows. In this implementation, the (default) value of 0 is used as the seed in the hashing algorithm.
observer.ingress.interface.name string Stores the name of the sensor interface where the activity corresponding to the NTA record was observed.
observer.name string The custom name of the sensor that generated the record. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
session_id number The session ID (SMB2+) or user ID (SMB1).
session_setup_kerberos_cname string In a SESSION_SETUP message using Kerberos, the client name.
session_setup_kerberos_encryption string In a SESSION_SETUP message using Kerberos, the encryption used.
session_setup_kerberos_error_code int In a SESSION_SETUP message using Kerberos, the error code.
session_setup_kerberos_failed_request string In a SESSION_SETUP message using Kerberos, request that failed.
session_setup_kerberos_msg_type string In a SESSION_SETUP message using Kerberos, the message type.
session_setup_kerberos_realm string In a SESSION_SETUP message using Kerberos, the encryption used.
session_setup_kerberos_sname string In a SESSION_SETUP message using Kerberos, the server name.
session_setup_kerberos_weak_encryption boolean In a SESSION_SETUP message using Kerberos, whether weak encryption was used.
session_setup_ntlmssp_domain string In a SESSION_SETUP message using NTLMSSP, the domain.
session_setup_ntlmssp_host string In a SESSION_SETUP message using NLTMSSP, the host.
session_setup_ntlmssp_user string In a SESSION_SETUP message using NLTMSSP, the user.
session_setup_ntlmssp_version string In a SESSION_SETUP message using NLTMSSP, the version.
session_setup_request_host_native_lm string The SMB1 native LAN Manager string.
session_setup_request_host_native_os string The SMB1 native OS string.
session_setup_response_host_native_lm string The SMB1 native LAN Manager string.
session_setup_response_host_native_os string The SMB1 native OS string.
server.geo.country_iso_code string If the IP address contained in the dst_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
server.geo.location geo_point If the IP address contained in the dst_ip field can be geo-located, the latitude and longitude is provided.
severity int A severity score associated to the SMB message. Currently unused.
source string The license of the sensor that uploaded the SMB record.
src_hostname string The hostname for src_ip.
src_ip ip The source IP address.
src_port number The source port.
src.homenet boolean Indicates if the source IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
src.host_tags string Contains the list of host tags that are assigned to the source IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
status string The status value.
tags string List of tags associated to the SMB message. Currently unused.
tree_connect_service_request string In a TREE_CONNECT message, the service request.
tree_connect_service_response string In a TREE_CONNECT message, the service response.
tree_connect_share string In a TREE_CONNECT message, the share.
tree_connect_share_type string In a TREE_CONNECT message, the share type (FILE or PIPE).
tree_id number The tree ID.
ts_start date The start timestamp of the SMB message.
tx_id number The transaction ID.

Example smb.json:

{
  "command": "SMB2_COMMAND_IOCTL",
  "dcerpc_call_id": 3,
  "dcerpc_generic_request_frag_cnt": 1,
  "dcerpc_generic_request_opnum": 29,
  "dcerpc_generic_request_stub_data_size": 20,
  "dcerpc_generic_response_frag_cnt": 1,
  "dcerpc_generic_response_stub_data_size": 24,
  "dcerpc_request_cmd": "REQUEST",
  "dcerpc_response_cmd": "RESPONSE",
  "dialect": "3.11",
  "dst_hostname": null,
  "dst_ip": "192.168.1.254",
  "dst_port": 445,
  "dst.homenet": true,
  "dst.host_tags": [],
  "geoip": null,
  "network.community_id": "1:nfDCYHYfhLLbMcIGQu3z3r54hcw=",
  "observer.ingress.interface.name": "ens192",
  "observer.name": "sensor1",
  "session_id": 61595132621289,
  "severity": null,
  "source": "ABCDEFGHIJ0123456789:sensor1",
  "src_hostname": null,
  "src_ip": "192.168.1.1",
  "src_port": 55352,
  "src.homenet": true,
  "src.host_tags": [],
  "status": "STATUS_SUCCESS",
  "tags": [],
  "tree_id": 1,
  "ts_start": 1574841380000,
  "tx_id": 14
}

TLS record types

A TLS document represents a TLS session.

Field name Type Description
certificates_chain unknown Currently unused.
certificates_sha1_chain string The list of SHA1 hashes for the certificates in the certificate chain.
certificates_sha256_chain string The list of SHA256 hashes for the certificates in the certificate chain.
client.geo.country_iso_code string If the IP address contained in the src_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
client.geo.location geo_point If the IP address contained in the src_ip field can be geo-located, the latitude and longitude is provided.
dst_hostname string The host name for dst_ip.
dst_ip ip The destination IP address.
dst_port number The destination port.
dst.homenet boolean Indicates if the destination IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
dst.host_tags string Contains the list of host tags that are assigned to the destination IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
geoip.country_code2 string The country code for the geolocation of dst_ip, if available. This field is deprecated, no longer used, and will be removed in the future.
geoip.lat number The latitude coordinates for the geolocation of dst_ip, if available. This field is deprecated, no longer used, and will be removed in the future.
geoip.location geo_point The coordinates for the geolocation of dst_ip, if available (comma separated). This field is deprecated, no longer used, and will be removed in the future.
geoip.lon number The longitude coordinates for the geolocation of dst_ip, if available. This field is deprecated, no longer used, and will be removed in the future.
ja3_hash string The JA3 fingerprint of this TLS session.
leaf_certificate_issuer_info string Issuer information extracted from the leaf certificate.
leaf_certificate_subject_info string Subject information extracted from the leaf certificate.
n number The number of occurrences of this record.
network.community_id string Stores a hash of the source and destination IPs and ports, as well as the protocol used in the communication. This is a tool-agnostic standard to identify flows. In this implementation, the (default) value of 0 is used as the seed in the hashing algorithm.
observer.ingress.interface.name string Stores the name of the sensor interface where the activity corresponding to the NTA record was observed.
observer.name string The custom name of the sensor that generated the record. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
server.geo.country_iso_code string If the IP address contained in the dst_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
server.geo.location geo_point If the IP address contained in the dst_ip field can be geo-located, the latitude and longitude is provided.
severity number A severity score associated to the flow. Currently unused.
sni string The SNI of the session.
source string The license of the sensor that uploaded the TLS record.
src_hostname string The hostname for src_ip.
src_ip ip The source IP address.
src_port number The source port.
src.homenet boolean Indicates if the source IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
src.host_tags string Contains the list of host tags that are assigned to the source IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
tags string List of tags associated to the TLS request/response. For example, cloud-service-provider-saas.
tls.cipher string A string indicating the cipher used during the current connection. For example, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384.
tls.version string The numeric portion of the TLS protocol version. For example, 1.2.
tls.version_protocol string The protocol name portion of the TLS protocol version, normalized to lowercase. For example, tls.
ts_start date The start timestamp of the TLS request.

Example tls.json:

{
  "certificates_sha256_chain": [
    "3a2fbe92891e57fe05d57087f487e30f17e5a5f53ef403d618e5b74d7a7e6ecb",
    "773ef34f28c1ebcb8e2908e85178acf8d1fdd7f239ff241c69885bd2692caa67",
    "973a41276ffd01e027a2aad49e34c37486d3e976ff6a620b6712e33832041aa6",
    "c3846bf24b9e93ca64274c0ec67c1ecc50e24ffcacd2d74019350e81fe546ae4"
  ],
  "dst_hostname": null,
  "dst_ip": "18.205.93.255",
  "dst_port": 443,
  "dst.homenet": false,
  "dst.host_tags": [],
  "geoip": null,
  "ja3_hash": "535aca3d99fc247509dc50933cd71d37",
  "leaf_certificate_issuer_info": "C=US,ST=Arizona,L=Scottsdale,O=GoDaddy.com\\, Inc.,OU=http://certs.godaddy.com/repository/,CN=Go Daddy Secure Certificate Authority - G2",
  "leaf_certificate_subject_info": "OU=Domain Control Validated,CN=*.zoom.us",
  "n": 1,
  "network.community_id": "1:j55+gYCahlTwDVMPkoMQybL9RdQ=",
  "observer.ingress.interface.name": "ens192",
  "observer.name": "sensor1",
  "proto": "TCP",
  "server": {
    "geo": {
      "country_iso_code": "US",
      "location": {
        "lat": 39.0481
        "lon": -77.4728,
      }
    }
  },
  "severity": null,
  "sni": "zoom.us",
  "source": "ABCDEFGHIJ0123456789:sensor1",
  "src_hostname": null,
  "src_ip": "192.168.49.127",
  "src_port": 58811,
  "src.homenet": false,
  "src.host_tags": [],
  "tags": [
    "cloud-service-provider-saas"
  ],
  "tls.cipher": "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
  "tls.version": "1.2",
  "tls.version_protocol": "tls",
  "ts_start": 1589197473000
}

Webrequest record types

A webrequest document represents an HTTP request/response.

Field name Type Description
client.geo.country_iso_code string If the IP address contained in the src_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
client.geo.location geo_point If the IP address contained in the src_ip field can be geo-located, the latitude and longitude is provided.
dst_hostname string The host name for dst_ip.
dst_ip ip The destination IP address.
dst_port number The destination port.
dst.homenet boolean Indicates if the destination IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
dst.host_tags string Contains the list of host tags that are assigned to the destination IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
geoip.country_code2 string The country code for the geolocation of dst_ip, if available. This field is deprecated, no longer used, and will be removed in the future.
geoip.location geo_point The coordinates for the geolocation of dst_ip, if available (comma separated). This field is deprecated, no longer used, and will be removed in the future.
hostname string The hostname used in the request.
http_version string Whether this document correspond to a HTTP or HTTPS request.
method string The HTTP request verb.
n number The number of occurrences of this record.
network.community_id string Stores a hash of the source and destination IPs and ports, as well as the protocol used in the communication. This is a tool-agnostic standard to identify flows. In this implementation, the (default) value of 0 is used as the seed in the hashing algorithm.
observer.ingress.interface.name string Stores the name of the sensor interface where the activity corresponding to the NTA record was observed.
observer.name string The custom name of the sensor that generated the record. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
path string The request path.
referer string The value of the referer header.
request_headers string A list of request headers.
resource_path string The request resource path.
response_body_earlyhash string Unused.
response_body_type string Unused.
response_code number The HTTP response status code.
response_content_type string The response content-type.
response_headers string A list of response headers.
response_redirect string The redirection target, in case of a redirect response.
server.geo.country_iso_code string If the IP address contained in the dst_ip field can be geo-located, the 2-letter ISO 3166 country code is provided.
server.geo.location geo_point If the IP address contained in the dst_ip field can be geo-located, the latitude and longitude is provided.
severity number A severity score associated to the flow. Currently unused.
source string The license of the sensor that uploaded the passive webrequest record.
src_hostname string The hostname for src_ip.
src_ip ip The source IP address.
src_port number The source port.
src.homenet boolean Indicates if the source IP address is in the home network range. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
src.host_tags string Contains the list of host tags that are assigned to the source IP address. Existing records will not reflect current values after the configuration is modified. Updates may take a while to appear because information is cached.
tags string List of tags associated to the DNS request/response. For example, certificate-authority.
ts_start date The start timestamp of the HTTP request.
user_agent string The value of the user-agent header.

Example webrequest.json:

{
  "dst_hostname": null,
  "dst_ip": "205.185.216.42",
  "dst_port": 80,
  "dst.homenet": false,
  "dst.host_tags": [],
  "geoip": null,
  "hostname": "ctldl.windowsupdate.com",
  "http_version": "http",
  "method": "GET",
  "n": 0,
  "network.community_id": "1:+zZ7JOGS9LFUa8hi74jcZRNf+U0=",
  "observer.ingress.interface.name": "ens192",
  "observer.name": "sensor1",
  "path": "/msdownload/update/v3/static/trustedr/en/pinrulesstl.cab?f48b2fd8fbc22471",
  "proto": "TCP",
  "referer": "",
  "request_body": null,
  "request_headers": [
    "Accept-Encoding: peerdist",
    "Accept: */*",
    "Connection: Keep-Alive",
    "Host: ctldl.windowsupdate.com"
    "If-Modified-Since: Fri, 02 Jun 2017 17:39:05 GMT",
    "If-None-Match: \"80424021c7dbd21:0\"",
    "User-Agent: Microsoft-CryptoAPI/10.0",
    "X-P2P-PeerDist: Version=1.1",
    "X-P2P-PeerDistEx: MinContentInformation=1.0, MaxContentInformation=2.0",
  ],
  "resource_path": "/msdownload/update/v3/static/trustedr/en/pinrulesstl.cab",
  "response_body_earlyhash": null,
  "response_body_type": null,
  "response_code": 304,
  "response_content_type": null,
  "response_headers": [
    "Accept-Ranges: bytes",
    "Date: Mon, 11 May 2020 11:45:50 GMT",
    "ETag: \"80424021c7dbd21:0\"",
    "X-CCC: US",
    "X-CID: 9"
    "X-HW: 1589197550.dop027.la3.t,1589197550.cds040.la3.c",
  ],
  "response_redirect": null,
  "server": {
    "geo": {
      "country_iso_code": "US",
      "location": {
        "lon": -112.0717,
        "lat": 33.5083
      }
    }
  },
  "severity": null,
  "source": "ABCDEFGHIJ0123456789:sensor1",
  "src_hostname": "lldc.lldcqa.com",
  "src_ip": "10.2.93.254",
  "src_port": 52415,
  "src.homenet": true,
  "src.host_tags": [],
  "tags": [
    "certificate-authority"
  ],
  "ts_start": 1589197550000,
  "user_agent": "Microsoft-CryptoAPI/10.0"
}