ArcGIS Enterprise log data

Procedure

  1. Executing the statements from dev-console/ct-arcgis-logfile.txt in the Kibana Dev Console

  2. Importing Kibana Dashboards, Queries and Index Patterns from kibana/ct-arcgis/export.ndjson file

  3. Configuring the logstash pipeline ct-arcgis-logfile

  4. Setting up the ingest pipeline ingest/ct-monitor-arcgis-parse-servicename.txt (see below)

  5. Configuring Filebeat on the ArcGIS Enterprise Host to poll the log files on a regular basis (see below)

  6. Verify ArcGIS Log Level is set correctly (see below)

Establishment of the Elastic ingest pipeline

The ingest pipeline extracts ArcGIS service names from the log message for those events where the ags.target field is not already populated.

Publish pipeline

The contents of

  • ingest/ct-monitor-temporal.txt

  • ingest/ct-monitor-arcgis-parse-servicename.txt

to be sent via the Kibana Dev Tools > Console.

Filebeat configuration - Notes

The Filebeat component must be installed for each ArcGIS host that is to be involved in collecting the log data. Currently Filebeat 7.x is supported, with Filebeat 8.x no problems could be found yet.

The Filebeat configuration is then done on the basis of the template filebeat/arcgis-logfile/filebeat.yml.

filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - c:\arcgisserver\logs\*\server\*.log
      - c:\arcgisserver\logs\*\services\*\*.log
      - c:\arcgisserver\logs\*\services\*\*\*.log
      - c:\arcgisserver\logs\*\services\System\*\*.log
    fields:
      type: server

    multiline.pattern: '^<Msg([^>]*?)>(.*)'
    multiline.negate: true
    multiline.match: after
output.logstash:
  hosts: ["logstash.host:5604"]
fields:
  env: PROD
Select the value under fields/type from server, portal, datastore to get better filtering possibilities in Kibana. The same applies to fields/env to distinguish between different stages.

Verify ArcGIS Log Level is set correctly

The evaluations and analyses of the service.monitor dashboards require a log level of 'Fine' at the ArcGIS Server. This value can be set via the ArcGIS Server Manager under Logs > Settings.