Release Notes 4.6

What’s New

ArcGIS SOC Process Analysis

Analyses based on system metrics are now possible for ArcSOC processes. This includes the development of the number of processes, their memory usage and CPU requirements. The diagrams thus enable the assessment of the current and historical system utilisation of ArcGIS Server hosts. The operation can estimate how strongly the system load by ArcGIS Server requests varies over the course of the day or develops in the long term. Through this information, existing resource bottlenecks can be recognised or planning needs for system scaling can be identified.

System-wide notifications in exceptional cases

service.monitor data collected in Elasticsearch can now serve as a trigger for messages from service.monitor notification system. Possible use cases include:

  • CPU load on ArcGIS Server for 5 minutes over 95%.

  • More than 10 ERROR log messages from con terra applications in the last 15 minutes

  • Failed FME server jobs

Update Notes

If you skip several versions during the update, please also follow all update notes of the intervening versions.
When upgrading from an old version, all Elasticsearch related objects like pipelines, templates, Kibana diagrams and ingest pipelines need to be updated.

Properties

The properties of the two web applications monitor and monitor-analytics have been adjusted and harmonised with the other con terra products. This concerns the properties of the mailing:

  • mail.host becomes mailing.host.
    This change must be made in external application.properties files.

Updating Elastic Index Lifecycle Management Policies & Templates

If service.monitor 4.5 is already in use, an update of the Index Lifecycle Policies and Index Templates for the used data sources is necessary (resources/analytics/elasticsearch/dev-console). In the templates, a bug with the alias configuration is fixed and the use of Elastic Ingest Pipelines is anchored.

Since version 4.5, if the data source to map.apps user interactions is already active, then the following steps must be performed (SRVMON-718):

  • Update ILM and Index Template as described above.

  • Execute each statement below, replacing -000001 with the value of the most recent index (this may be different for each index)

Update analytics usagelog indices config
POST ct-analytics-app*/_alias/analytics-usagelog-app
POST ct-analytics-app-000001/_alias/analytics-usagelog-app
{"is_write_index":true}


POST ct-analytics-map*/_alias/analytics-usagelog-map
POST ct-analytics-map-000001/_alias/analytics-usagelog-map
{"is_write_index":true}

POST ct-analytics-log*/_alias/analytics-usagelog-log
POST ct-analytics-log-000001/_alias/analytics-usagelog-log
{"is_write_index":true}

POST ct-analytics-server*/_alias/analytics-usagelog-server
POST ct-analytics-server-000001/_alias/analytics-usagelog-server
{"is_write_index":true}

POST ct-analytics-tool*/_alias/analytics-usagelog-tool
POST ct-analytics-tool-000001/_alias/analytics-usagelog-tool
{"is_write_index":true}

DELETE ct-analytics-app*/_alias/ct-analytics-app
DELETE ct-analytics-tool*/_alias/ct-analytics-tool
DELETE ct-analytics-map*/_alias/ct-analytics-map
DELETE ct-analytics-log*/_alias/ct-analytics-log
DELETE ct-analytics-server*/_alias/ct-analytics-server

In the case of using service.monitor with FME Server Jobs, manual adjustments are also necessary if the functionality is already in use (SRVMON-733). Due to an error, an incorrect assignment of indexes, aliases and patterns occurred here (ct-fme-jobs vs. ct-fme-job).

The stock data must be copied to the index pattern ct-fme-jobs-* under ILM administration. This is done by re-indexing into the alias ct-fme-jobs. Afterwards, the old index ct-fme-job can be removed.

Correct FME Jobs Index data location
# re-index pre-existing entries to index alias
POST _reindex?wait_for_completion=false
{
  "source": {
    "index": "ct-fme-job"
  },
  "dest": {
    "index": "ct-fme-jobs"
  }
}
# once re-indexing has finished and after verifying all fme jobs have been transfered, the old index is safe to delete
DELETE /ct-fme-job

# same procedure for fme jobroutes
POST _reindex?wait_for_completion=false
{
  "source": {
    "index": "ct-fme-jobroute"
  },
  "dest": {
    "index": "ct-fme-jobroutes"
  }
}
DELETE /ct-fme-jobroute

Notifications: Updating the monitoring database schema

Please run the SQL script for your database under resources/sql/upgrade/4.6.

Monitoring: Oracle DB data type check

If you use Oracle DB: Please check if resources/sql/upgrade/4.5/oracle-schema-changes.sql was already executed with the last upgrade.

Analytics Bundles for map.apps

The bundles for collecting user interactions now follow their own release cycle and have been renamed from e.g. usagelog_restservice to analytics_restservice. This must be taken into account when referencing the bundles in the app.json or the application.properties of map.apps.

Discontinued Features

map.apps based dashboards

Analytics dashboards anchored in the map.apps Manager are no longer part of this release. All dashboard development will take place in Elastic Kibana. Should you wish to continue using the map.apps based dashboards, you can use the corresponding bundles from version 4.5. In addition, the ability to run Line 3 App must be restored in map.apps. In this case, it is recommended to use the dashboards as map.apps App and not within map.apps Manager.

Monitoring charts

Removed the ability to graph monitoring data in the montoring web app. Alternatively, enable sending monitoring events to Elastic and use the Monitoring Kibana dashboards.

Monitoring UI designs

Only the everlasting named design is now supported for the monitoring component UI layout.

More

  • Apache Tomcat 8 (Monitoring web application)

  • Java 8 (Monitoring web application)

Deprecated Features

  • INSPIRE capabilities of Monitoring (Rules)

Known Limitations

The following list contains known issues of the version 4.6 at the time of release:

Issue Component Description

SRVMON-88

Monitoring

Provisioning of means to monitor INSPIRE Quality of Service requirements

Changelog

4.6.1

Fixed Issues

SRVMON-671

[Usagelog API] Attempt to access the attribute resolution of an undefined object

SRVMON-734

info.txt - property "info" is not filled correctly for monitor and monitor-analytics webapp

SRVMON-737

Web applications do not log due to wrong log4j2 dependency

SRVMON-738

Accessing monitor alerting page does not check user’s authentication status

SRVMON-745

Docs: Link to support site in wrong language

SRVMON-735

FME Jobs Dashboard - Engine Workload hover info does not contain correct date values & multiple engines are not reflected

SRVMON-747

FME Jobs Dashboard - Jobs only shown when start and end time differs in minutes or hours

4.6.0

New Features

SRVMON-723

Extension of the Analytics/map.apps dashboards

SRVMON-702

Bind notification template to a "harvested" Kibana Rule

SRVMON-701

ArcSOC Metric Analysis

SRVMON-699

Enable user notification of Elastic Alert Rules via monitoring

SRVMON-698

Kibana Dashboard for Monitoring data

SRVMON-697

Create monitoring event data Elastic Ingest Pipeline for extracting folder, service, host

SRVMON-694

[Elastic Alerting] Send user notification for Elastic Index Alerts through monitoring

SRVMON-690

Replace classical Monitoring diagrams w/ new Kibana Monitoring Dashboards

SRVMON-684

Provide Docker Image for service.monitor Monitoring

SRVMON-678

Provide map.apps Analytics integration for new search UI

Fixed Issues

SRVMON-733

Index-Pattern, Alias and Index Template Pattern are wrongly configured for fme-job harvesting

SRVMON-722

ArcGIS timeout analysis is faulty

SRVMON-718

[ILM] Inconsistent alias definition stops index rollover from working

SRVMON-673

Implementation in map.apps fails due to missing 'X-Frame-Options' 'sameorigin'

SRVMON-729

"Hilfe"-Link points to wrong location (Monitoring Dashboard)

SRVMON-663

Allow ArcGIS elapsed time values to be read using both floating point and floating comma format