Installation

This page describes the new installation of smart.finder. Make sure that the specified system requirements are met, see System Requirements.

The following describes the steps required to integrate smart.finder with Apache Solr. The installation of Apache Solr is not documented here. Please follow the official Apache Solr Documentation .

If you want to update an existing installation, continue with Update an existing installation.

Installation of smart.finder

smart.finder consists of the following components, which are described in the following section:

component location

Apache Solr Index Configuration

[RELEASE-DIR]/resources/solr

Server Webapp

[RELEASE-DIR]/smartfinder-search.war

map.apps Bundles

[RELEASE-DIR]/ct-smartfinder-js-3.0.0.jar

map.apps apps

[RELEASE-DIR]/resources/apps

server CLI

[RELEASE-DIR]/cli

The following sections describe the installation of the individual components.

Apache Solr configuration

The configuration of Apache Solr is done in two steps:

  1. definition of required environment variables

  2. configuration of the smart.finder index

Definition of required environment variables

Before starting Apache Solr, you must define the following environment variables:

SOLR_ENABLE_STREAM_BODY=true
SOLR_ENABLE_REMOTE_STREAMING=true
SOLR_SECURITY_MANAGER_ENABLED=false
SOLR_MODULES=extraction,analysis-extras,scripting

The environment variables can be defined in different ways:

  • in the file /etc/default/solr.in.sh (Linux) or [SOLR_INSTALL_DIR]/bin/solr.in.cmd (Windows)

    • for Linux, insert the variables as mentioned above

    • for Windows, expand the values as follows: set SOLR_ENABLE_STREAM_BODY=true

  • as system environment variables

Configuration of the smart.finder index

The rollout of smart.finder contains preconfigured Apache Solr indexes. These must be copied to the SOLR_HOME directory.

The SOLR_HOME directory is located under [SOLR_INSTALL_DIR]/server/solr by default.

You can also start Apache Solr with a different directory. In this case, you must adapt this accordingly. The following options are available here:

  • in the file /etc/default/solr.in.sh (Linux) or [SOLR_INSTALL_DIR]/bin/solr.in.cmd (Windows)

  • as system environment variables

  • when starting Apache Solr with the parameter --solr-home <dir>

  1. Copy the contents of the directory [RELEASE-DIR]/resources/solr/cores into the SOLR_HOME directory. This creates the following directory structure:

    /SOLR_HOME
 /core0
 /featuresearch
 /indexingmanagement
 /lib
 /...

  2. Move the file SOLR_HOME/lib/jts-core-[VERSION].jar into the directory [SOLR_INSTALL_DIR]/server/solr-webapp/webapp/WEB-INF/lib. This library is required by Apache Solr for advanced spatial search functions.

  3. Start Apache Solr and call up the admin interface to ensure that the configuration was successful.

After copying, ensure that the correct access rights to the files and directories are set for Apache Solr. This applies in particular to Linux environments. Example for the SOLR_HOME directory if Solr was installed with the default user solr and the group solr:

chown -R solr:solr core0/ featuresearch/ indexingmanagement/ lib/

The same applies to the file jts-core-[VERSION].jar.

Optional: Securing direct access to Apache Solr

Direct access to the Apache Solr service can optionally be secured using Basic Authentication. To do this, configure the Basic Authentication Plugin in Apache Solr. Other authentication methods are not supported by smart.finder.

If security is provided by Basic Authentication, the user must be configured accordingly in smart.finder to access Apache Solr. To do this, use the properties solr.auth.username and solr.auth.password (or secret.solr.auth.password).

Install the server application

Start with installing the search service webapp smartfinder-search.war.

  1. Optionally, you can change the name of the WAR files. The name will appear in the URL to the application.

  2. Make sure the Tomcat server is started.

  3. Copy the WAR file into the folder %TOMCAT%\webapps. The file will then be unpacked automatically.
    Alternatively, you can use Tomcat Manager to install the WAR file. When the Tomcat service is started, you can access Tomcat Manager at http://<yourserver>:8080/manager/html.

  4. Now configure the base URL to the Apache Solr service. This is done in the application.properties file, which is used for the smart.finder server webapp. Further information on the configuration can be found in the configuration reference.

    The following property must be set:

    solr.server.url

    URL to the Apache Solr instance used.

    Default value: http://localhost:8983/solr

  5. Restart the Tomcat or the web application. The server application is now available at the following URL:

    https://<yourserver>/smartfinder-search

Installation of the client web app

The client application is installed in an existing map.apps instance. The installation and configuration takes place in three steps:

  1. installation of the bundles

  2. installation of the apps

  3. installation of the smart.finder job manager

Installation of the bundles

  1. Log in to map.apps Manager as admin user.

  2. Open the Bundles tab.

  3. Click on the + button above the list of installed bundles.

  4. Upload the file [RELEASE-DIR]/ct-smartfinder-js-3.0.0.jar. This contains all bundles for smart.finder.

As soon as the bundles have been uploaded, they are displayed in the list of installed bundles. To make it easier to distinguish the smart.finder bundles from other bundles, their names always start with the abbreviation sf_.

Installation of the apps

  1. Log in to map.apps Manager as admin user.

  2. Open the Apps tab.

  3. Click on the + button above the list of installed apps. The app wizard opens.

  4. Enter a name and a title for the app and proceed through the wizard to the step for uploading an app.

  5. The apps are stored as ZIP files in the [RELEASE-DIR]/resources/apps folder. Upload the desired app.

  6. Continue with the next steps in the wizard until the wizard is complete.

  7. Start the app by clicking on the Start button in the app list on the right-hand side.

Installation of the smart.finder Job Manager

To be able to use the smart.finder Job Manager within the map.apps Manager, proceed as follows:

  1. Open the map.apps configuration file application.properties. This is usually located in the [USER_HOME]/.mapapps folder.

  2. Search for the entry manager.config.viewbundles. If it already exists, add the value sf_jobadmin at the end. If it does not yet exist, add the entry with the following value:

    Example for manager.config.viewbundles
    manager.config.viewbundles=appmanagement,reportmanagement,bundlemanagement,mapapps-github-manager,bundleupdatechecker,sf_jobadmin

  3. The URL of the smart.finder server must now also be entered in the same file.

    finder.service.url=https://<yourserver>/smartfinder-search

    The finder.service.url property is used both by the Job Manager to manage the indexing jobs and by all apps for search queries.

  4. To define spatial extents for the CSW filter, you can add the property csw.filters.boundingboxes to this file as well.

    Example for csw.filters.boundingboxes:
    csw.filters.boundingboxes=[\
{\
'title': {\
  'de':'Deutschland',\
  'en':'Germany'\
},\
'minX': 5.8663153,\
'maxX': 15.0419319,\
'minY': 47.2701114,\
'maxY': 55.058347\
},\
{\
'title': {\
'de':'Frankreich',\
'en':'France'\
},\
'minX': -5.142222,\
'maxX': 9.662499,\
'minY': 41.333740,\
'maxY': 51.124199\
}\
]

    You can find a detailed description for this in the documentation of the "sf_jobadmin" bundle .

Restart Tomcat and reload map.apps Manager in the web browser. The new Indexing Jobs tab will then appear in the map.apps Manager.

‘smart.finder Server and map.apps with different domains’

If the smart.finder server component is accessible under a domain other than map.apps, make sure that the map.apps proxy servlet is enabled for access to the smart.finder server. In this case, you should add the domain name of the client to the CORS configuration property cors.allowed.origins in the application.properties file of the smart.finder server webapp.

If you have difficulties with CORS, you can also use the map.apps proxy servlet. For more information, see CORS settings.

Configuration smart.finder map.apps App

Everything is now configured to run the supplied apps. These apps provide you with an initial state for your own customisations.

If you want to use smart.finder in one of your own map.apps apps, you can add the bundle sf_full-screen-map to this app via the map.apps Manager. The bundle loads all other bundles required to enable the search via Omnisearch (the standard bundle in map.apps for keyword searches) and the display of results in a popup.

Further information on configuration can be found in the section Apps and bundles.

Installation of the CLI

The smart.finder CLI is a tool for managing indexing jobs via the command line.

finderctl is delivered as an executable file for Windows and Linux operating systems (x86-64 architecture). It is located under:

  • [RELEASE-DIR]\cli\windows_x64\finderctl.exe for Windows

  • [RELEASE-DIR]/cli/linux_x64/finderctl for Linux

‘Move directory’
You can also copy the finderctl directory including the auxiliary files to any other location. Also copy the auxiliary files next to the executable file. These may be required for the runtime of finderctl.

Add the storage location of finderctl to the search path of your system (e.g. $PATH or %PATH%). This allows you to run finderctl without having to specify the full path. Depending on your operating system, it may be necessary to authorise finderctl for execution.

Verify the installation by executing the following command:

~$ finderctl --version

Security aspects

You manage access to safety-critical functions completely in the server component of smart.finder. This includes creating indexing jobs, filling or deleting the index, and reading specific administrative interfaces. Each of these cases requires user authentication, which is requested in the browser.

Web Application Security

smart.finder has endpoints for querying and write access to the index or other core functions.

The configuration of the security settings is described in Authentication and Authorisation.

TLS/SSL Server Trust

When establishing connections to HTTPS endpoints through server-side Java components, make sure that the TLS/SSL server certificate of the requested host is accepted and trusted by the Java Virtual Machine (JVM). Otherwise, no HTTPS connection to the server is established, and the application responds incorrectly. This affects the smart.finder when you use job configuration in the manager to index sources that are accessed via an HTTPS endpoint (e.g. OGC CSW catalog).

The configuration option security.ssl.trustAny controls under which conditions a TLS server certificate is accepted:

security.ssl.trustAny=true

All certificates are accepted.
The only condition is that the certificate is valid for a certain period of time. The certificate does not have to be issued for the requested host or by a trusted authority. This setting should only be active in development environments.

security.ssl.trustAny=false

(delivery status)
Only certificates that are considered trustworthy by the JVM are accepted.
It is strongly recommended to keep this setting. This is especially true for productive environments. You must ensure that server certificates of HTTPS endpoints to which connections are made are trusted. Otherwise, an error will occur when a connection is established.

Trustworthiness of server certificates

For a server certificate to be considered trustworthy by the JVM, you must install either the concrete server certificate or its root certificate in the Java Runtime Environment (JRE). Trusted certificates or root certificates are stored by default in the JRE in a Java keystore under [JRE_HOME]/lib/security/cacerts. However, depending on the operating system or configuration of the JVM, it is possible that other sources for trusted certificates are used.

The following section describes which types of server certificates are usually used by HTTPS endpoints and what needs to be considered with regard to trustworthiness.

Certificates with root certificate installed in the JRE

Productive Websites that provide services over HTTPS typically have server certificates whose root certificate is issued by a trusted Certificate Authority (CA). These root certificates are pre-installed in the JRE and are therefore automatically classified as trustworthy.

No further steps are required to allow connection to an endpoint whose server certificate has a root certificate already installed in the JRE.

Certificates with a root certificate that is not installed in the JRE

Occasionally, organizations use server certificates issued by their own internal CA. The corresponding root certificate is therefore not part of the certificates pre-installed in the JRE. Connections to HTTPS endpoints with such server certificates will fail unless the root certificate is automatically added to all JRE installations by internal company processes.

To ensure that all server certificates issued by this internal CA are trusted, the CA’s root certificate must be added to the keystore [JRE_HOME]/lib/security/cacerts.

Self-signed certificates

Self-signed server certificates, i.e. certificates that reference themselves as issuers, are often used for testing purposes and should not be used in production systems. Since a self-signed server certificate is not part of the certificates already installed in the JRE, it is not trustworthy.

To allow HTTPS connections to a host that uses a self-signed server certificate, add this certificate to the [JRE_HOME]/lib/security/cacerts keystore.

Typical error messages

When establishing server-side HTTPS connections, various error situations can occur in connection with the validation of server certificates. The following sections list typical error messages that may appear in the log file of the server-side application or in the browser interfaces of the applications. In addition, possible causes are described and solutions are suggested. Depending on the specific infrastructure in which the affected application is operated, other causes and solutions may also need to be considered.

Typical error messages:

PKIX path building failed

Complete error message:

sun.security.validator.ValidatorException:
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

This error message is an indication that the certificate presented by the server for the HTTPS connection is not considered trustworthy by the JVM. It may be a self-signed certificate or the root certificate is not installed in the JRE.

Possible solutions:

  • Installing the root certificate in the JRE

  • Installing the server certificate in the JRE

  • If the requested server is under its own control: Provision of a new server certificate on the requested server, issued by a trusted CA.

No name matching [requested.host.name] found

Complete error message:

javax.net.ssl.SSLHandshakeException:
java.security.cert.CertificateException: No name matching maps.example.com found

This error message indicates that the host name used in the HTTPS request does not match the host name specified in the server certificate. The client (browser, Java application) compares the host name used in the request URL with the CN name of the Subject field in the server’s X509 certificate, for example

https://maps.example.com/

with

CN = www.example.com, O = Example LLC, L = Mountain Dew, ST = California, C = US.

If the two names do not match (upper/lower case is not relevant), the connection is terminated with the error message mentioned above. It is irrelevant whether the certificate would be classified as trustworthy. The trustworthiness check only takes place when it is ensured that the host names match.

This problem often occurs when clients can address the same server with different names because there are several DNS entries. Especially if the client is in the same domain as the server, the inconsistent use of non-qualified and fully qualified host names in request URLs or server certificates often leads to this problem. The use of the special host name localhost or IP addresses in URLs also causes this problem if the server certificate is issued to the qualified host name.

Possible solutions:

  • Use the host name in request URLs in the form specified in the server certificate.

  • Create and store a new server certificate for the host name requested by the client, for example, if it turns out that using a non fully qualified host name (or localhost or IP address) is not appropriate.

  • Create and store a new server certificate with alternative host names ("Subject Alternative DNS Names"), if it becomes clear, for example, that the server must inevitably be accessible under different host names.

No subject alternative DNS name [requested.host.name] found

Complete error message:

javax.net.ssl.SSLHandshakeException:
java.security.cert.CertificateException: No subject alternative DNS name matching maps.example.com found

This error message is a special case of the error message No name matching [requested.host.name] found. The only difference is that the server certificate has an additional field with alternative host names, but the host name used by the client does not match the Subject CN of the server certificate nor is it specified in its list of alternative host names (Subject Alternative DNS Names).

Possible solutions: The solutions suggested in the previous error message also apply here.

Indexable resources

smart.finder can index a number of different resources and thus make them searchable. In doing so, smart.finder automatically recognizes how a resource needs to be indexed. Therefore, no resource type or similar needs to be specified for indexing to be carried out correctly.

The following section provides an overview of the resource types that can be indexed. It also describes how the contents are indexed.

File based resources

The types listed in this table can be accessed either in the file system or via a URL. This means that http://example.com/file.ppt will produce the same index result as c:\file.ppt (assuming that file.ppt is identical in both cases).

Filename extension Name Indexed contents

doc

Microsoft Word 97-2003 Document

Complete Content

docx

Microsoft Word Document

Complete Content

geotiff

GeoTiff file

name + room section + specific metadata

html/xhtml

Hypertext Markup Language

Complete Text

jpg, png, gif

image files

path + file name

pdf

Adobe Acrobat Document

Complete content including metadata

ppt

Microsoft PowerPoint 97-2003 Presentation

Complete Content

pptx

Microsoft PowerPoint Presentation

Complete Content

rtf

Rich Text Format

Complete Content

shp

ESRI Shape file

name
room section, if .prj file exists
number of features
attribute names

txt

plain text

complete text

xls

Microsoft Excel 97-2003 Worksheet

Complete Content

xlsx

Microsoft Excel Worksheet

Complete Content

xml

XML

Complete content

Web services

Name Indexed contents Dedicated mapping file

OGC Catalog Service for the Web 2.0.2, ISO Application profile

Selected fields

jar:file:/ct-finder-api-3.0.0.jar!/de/conterra/finder/tika/xslt/iso19139.xslt

OGC Catalog Service for the Web 2.0.2, Capabilities

Selected Fields

jar:file:/ct-finder-api-3.0.0.jar!/de/conterra/finder/tika/xslt/csw.xslt

INSPIRE Discovery Service

Selected Fields

jar:file:/ct-finder-api-3.0.0.jar!/de/conterra/finder/tika/xslt/iso19139.xslt

INSPIRE Discovery Service, Capabilities

Selected Fields

jar:file:/ct-finder-api-3.0.0.jar!/de/conterra/finder/tika/xslt/csw.xslt

OGC Web Mapping Service 1.1.0 and 1.3.0 Capabilities

Selected Fields

jar:file:/ct-finder-api-3.0.0.jar!/de/conterra/finder/tika/xslt/wms.xslt

INSPIRE View Service Capabilities

Selected Fields

jar:file:/ct-finder-api-3.0.0.jar!/de/conterra/finder/tika/xslt/wms.xslt

ESRI ArcGIS Server REST Endpoint

Complete Content

internal

For indexing via an ESRI ArcGIS Server REST Endpoint, activate the Services Directory in the ArcGIS Server settings.