Release Notes 4.5

Analytics

What’s New

FME Server support

FME Server is now offered as a new data source for central information collection, aggregation and analysis. At regular intervals, the jobs executed and logged in the FME Server are queried and stored in the index. On this basis, the utilisation of the FME Engines, the success rate of the jobs and any waiting times can be analysed. In addition, the graphical visualisation of the job utilisation allows a quick impression and good assistance in creating new job schedules to achieve an optimal utilisation of the engines.

In addition to the job information, the associated job logs are also available in service.monitor. This is another building block for gaining a holistic overview of what is happening in a GeoIT solution.

Use of Elastic Index Lifecycle Management

service.monitor now supports Elastic Index Lifecycle Management to achieve a lower-maintenance operation of the Elasticsearch cluster. If ILM is used, Elastic is responsible for when indices are created or deleted (e.g. for log files). This also simplifies the configuration of Logstash.

Optimisation for ArcGIS Server Analysis

ArcGIS Enterprise dashboards now show more information and are better tailored to the needs of users. This also includes the option to enrich events in terms of time so that analyses are possible on an hourly or daily basis. For example, it can be quickly recognised whether longer response times are to be expected at certain times of the day.

Update notes

If you skip several versions during the update, please also follow all update notes of the intervening versions.

No further update notes for service.monitor analytics 4.5 available.

Deprecated Features

The following are deprecated and will be removed in a future release:

  • Support of map.apps Manager based dashboard visualization (map.apps line 3)

Known Limitations

The following list contains issues and notes known at the date of release of version 4.5 of service.monitor Analytics.

Issue Component Description

[SRVMON-391]

User Interface — UI behaves slowly when using Internet Explorer 9 / 10

Due to slow javascript processing the user interface behaves slowly when using the product with Internet Explorer 9 or Internet Explorer 10.

For an optimal user experience we recommend to use the most current version of Google Chrome / Chromium.

[SRVMON-527]

Dashboard Printing using Firefox browser omits some widgets

When printing dashboards using Firefox browser certain widgets are not being printed in the effective output, although they are visible in print preview.

We recommend to use Chrome browser for printing as it provides the best output. Alternatively - when being fixed to Firefox browser - we recommended to create additional dashboards with fewer widgets for printing purposes. May be linked to Mozzila Bug 1309046 — SVG with external content via <use> does not print .

[SRVMON-531]

Histograms do not show all values of a time period if the number of aggregated hits at the margins is 0.

In certain cases, the histograms display fewer bars than the selected time period would indicate if the aggregation values at the (time) edges of the query are 0.

[SRVMON-542]

Upload of Analytics-Bundles fails for map.apps version <= 3.7

Workaround: Unpack the bundle JAR file and rezip it with a normal zip tool. Then, upload the newly created file.

Changelog

4.5.2

Bug & Improvement

[SRVMON-686]

Wrong XML Security API version stops monitor analytics webapp from starting

[SRVMON-688]

Provide Integration files for security.manager 4.18.x and map.apps 4.12

[SRVMON-689]

Add empty failover values for elastic search parameters in logstash pipelines

4.5.1

Fixed Security Issues

SRVMON-680

(Analytics) Update Log4J to version 2.16.0 (Critical log4j CVE (CVE-2021-44228))

4.5.0

Story

[SRVMON-635]

Provide Dashboard with real time FME job execution information

[SRVMON-636]

Provide visual information to understand FME engine occupation by job execution

[SRVMON-638]

Harvest FME Server job execution log lines

[SRVMON-642]

Initial FME Integration Dashboard

[SRVMON-659]

Provide Elastic Index Lifecycle Management for "ct-log" data

[SRVMON-662]

Provide Elastic Index Lifecycle Management for all logstash pipelines

[SRVMON-661]

Provide means to analyze histogram data aggregated by "hour of day" or "day of week"

Bug & Improvement

[SRVMON-665]

Verify arcgis logstash pipeline is capable of loading and transforming arcgis data store

[SRVMON-494]

Collect app title additionally to app id

[SRVMON-648]

Pseudonomize ArcGIS user name by default in logstash pipeline

[SRVMON-649]

Pseudonomize service.monitor Monitoring user name by default in logstash pipeline

[SRVMON-652]

Track registered bundles on application framework events

[SRVMON-653]

Improve FME Server job & log line integration

Monitoring

What’s New

Monitoring of FME Server

Detailed mechanisms now exist to use service.monitor to check the reliability, stability and capacity of FME Server. This includes simple queries about the available engines, checks to see if jobs are lingering in the queue for a long time and whether scheduled jobs have been successfully executed according to plan.

Further navigation aids

The user interface is now easier to use because there are direct jump options from a job to its expectations. It is now also possible to navigate directly between the individual jobs in the properties page of a service.

The next execution time of a job can now be seen directly in the dashboard.

Removed installer

Monitoring does no longer get installed by executing an installer program. Rather it gets deployed by using a web archive. In future this will simplify automation processes.

Notifications: Http Web-Hook

Another notification channel is available with the http-json web hook. In the notification case, a json message can be sent to any endpoint.

Update Notes

If you skip several versions during the update, please also follow all update notes of the intervening versions.

Migration of Oracle DB data type from LONG RAW to BLOB

In order to replace the data type LONG RAW, which is no longer recommended for Oracle databases, with the data type BLOB, the SQL script located under resources\sql\upgrade\4.5 can be executed. The migration forces the application.property value db.type to be set to the value oracle.

Deprecated Features

The following are deprecated and will be removed in a future release:

  • Apache Tomcat 8.5 support — Running the upcoming minor version with Tomcat 8.5 will not be possible. If not yet done, we recommend upgrading to Tomcat 9.0.

  • Java 8 — Running the upcoming minor version of service.monitor with a JRE 8 will not be possible. If not yet done, we recommend upgrading to Java 11.

  • Charting functionalities and widget capabilities of webapplication

  • Support for Standard sdi and inspire styles. This style will be fully replaced by everlasting style.

Known Limitations

Provisioning of means to monitor INSPIRE Quality of Service requirements (SRVMON-88)

INSPIRE’s Quality of Service requirements have evolved during the past few years. It now make some more flexible assumptions on how request shall be sent to the instance under monitoring and how the response of those instances shall be interpreted correctly. For INSPIRE View Services the GetMap request shall make use of varying BoundingBoxes and a blank response image cannot be considered as valid if features are present in the requested area.

However, service.monitor currently is not able to perform those variable and aggregated checks mentioned above. We recommend you to make use of the job configuration options to mimic the desired behavior in a similar way. I.e. when you are going to test the Performance of an INSPIRE View Service create a service endpoint with two jobs, one caring for GetCapabilities operation and the other for GetMap operation. Via the job execution interval you can try to weigh the number of requests performing a GetCapabilities and the number of requests performing a GetMap. For enabling this please see Flexible configuring INSPIRE services.

Changelog

4.5.3

Fixed Security Issues

SRVMON-691

(Monitoring) Update Log4J to version 2.17.1 and resolve other current CVEs as well (CVE-2021-40690, CVE-2021-22060, CVE-2017-16853, CVE-2021-37136)

4.5.2

Fixed Security Issues

SRVMON-681

(Monitoring) Update Log4J to version 2.16.0 (Critical log4j CVE (CVE-2021-44228))

4.5.1

Bug & Improvement

[SRVMON-624]

Clarify ssl smtp server usage

[SRVMON-650]

Bean reference not found in ONLY_AUTHN mode

[SRVMON-651]

data.directoy.location gets replaced during new rollout creation

[SRVMON-656]

Verify incoming monitoring template string to be valid JSON

[SRVMON-622]

Support Credential-Templates-Usage when applying DynamicRequests on job creation page

[SRVMON-655]

Provide reason information if an expectation fails

[SRVMON-657]

Allow FME Job Schedule monitoring on FME schedules not running with a CRON schedule

[SRVMON-658]

Show next execution time in dashboard for active services

[SRVMON-660]

Simplify navigation to notification & monitoring templates and expectations when editing jobs

4.5.0

Story

[SRVMON-627]

FME extended availability checks

[SRVMON-620]

Add support for plain http json notification endpoint

Bug & Improvement

[SRVMON-619]

Hibernate access error when using Oracle 11 database

[SRVMON-625]

Limit storing unexpected error reason text to lenght of database field

[SRVMON-626]

Do start verifying a cron string if empty