ArcGIS SOC processes

Overview

Procedure

Installing & configuring Metricbeat on the ArcGIS Enterprise host to send the metrics regularly
Configure ArcGIS-SOC-Ingest-Pipeline for index & index templates

If Metricbeat is not yet active in the system, the installation process, configuration and start-up must first be carried out so that the index and index template are created in Elastic Index.

Installation & Configuration of Metricbeat

Download of Metricbeat via Download Metricbeat at Elastic

Configure Metricbeat Output (in metricbeat.yml)

Output configuration in Metricbeat

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["elasticsearch.test.conterra.de:443"]
  # Protocol - either `http` (default) or `https`.
  protocol: "https"
  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "<elastic_user>"
  password: "<password>"

System metric module configuration (in modules.d/system.yml)
Module configuration of "system" in Metricbeat
```
- module: system
  period: 10s
  metricsets:
    - cpu
    #- load
    - memory
    - network
    - process
    - process_summary
    - socket_summary
    #- entropy
    #- core
    #- diskio
    #- socket
    #- service
    #- users
  process.include_top_n:
    by_cpu: 50      # include top 50 processes by CPU
    by_memory: 50   # include top 50 processes by memory
```
Compared to the standard configuration, the following adjustments have been made here:
1. Increase the number of process information to be sent from 5 to 50. The reason for this is that all ArcSOC processes information should always be sent within one interval in order to obtain a complete picture.

Configure ArcGIS-SOC-Ingest-Pipeline for index & index templates

Configuration for the current index

To configure the pipeline in Elasticsearch for all indexes, the pipeline must be set to the affected indexes.

PUT /metricbeat-*/_settings
{
"index.final_pipeline" : "ct-monitor-metricbeat-arcsoc"
}

Configuration for the index template

To enable the pipeline for all future indexes that are created, the pipeline must be configured within the index template. This is done via the menu Stack Management > Index Management > Index Templates. There, the index template metricbeat-<version> must be edited in the section Index settings via the parameter final_pipeline.

Example of an updated index template

{
  "index": {
    "lifecycle": {
      "name": "metricbeat",
      "rollover_alias": "metricbeat-7.17.1"
    },
    "codec": "best_compression",
    "mapping": {
      "total_fields": {
        "limit": "10000"
      }
    },
    "refresh_interval": "5s",
    "number_of_shards": "1",
    "final_pipeline": "ct-monitor-metricbeat-arcsoc",
    "max_docvalue_fields_search": "200",
    "query": {
      "default_field": [
    // .... fields
      ]
    }
  }
}

As soon as a new version of Metricbeat is put into operation, the step to set the pipeline to Index and Index Template has to be repeated because of the new version pattern.