Indexing of ArcGIS Server layers

smart.finder allows you to index layers of an ArcGIS Feature Server and ArcGIS Map Server. To do this, you must adapt the indexing schema and create a mapping of the attributes of a feature type in the schema. The following example shows you the necessary steps.

smart.finder Server is delivered with a preconfigured ArcGIS Feature Server Setup. This setup consists of:

A Solr Core with preconfigured schema (smartsearch)
A sample app called full-screen-map-multicore

Both the indexing scheme and the layer configuration are already prepared for indexing a layer Points of Interest in Münster with a total of 18739 features. You can use this example as a blueprint for setting up additional layers. The following section explains in detail what has to be done for this.

Data basis

Let’s assume that you want to index a layer for Points of Interest in Münster. A feature of this layer has the following fields:

FID (type: esriFieldTypeOID, alias: FID, SQL Type: sqlTypeOther, length: 0, nullable: false, editable: false)
osm_id (type: esriFieldTypeString, alias: osm_id, SQL Type: sqlTypeOther, length: 10, nullable: true, editable: true)
code (type: esriFieldTypeSmallInteger, alias: code, SQL Type: sqlTypeOther, nullable: true, editable: true)
fclass (type: esriFieldTypeString, alias: fclass, SQL Type: sqlTypeOther, length: 28, nullable: true, editable: true)
name (type: esriFieldTypeString, alias: name, SQL Type: sqlTypeOther, length: 100, nullable: true, editable: true)
Shape__Area (type: esriFieldTypeDouble, alias: Shape__Area, SQL Type: sqlTypeDouble, nullable: true, editable: false)
Shape__Length (type: esriFieldTypeDouble, alias: Shape__Length, SQL Type: sqlTypeDouble, nullable: true, editable: false)

The fields osm_id, code and name should be taken into account when indexing the features.

Adaptation of the schema

smart.finder is delivered with a preconfigured core for this use case. The smartsearch core is located in the directory solr.home:

/smartfinder-search/WEB-INF/solr.home/smartsearch

smartsearch schema.xml

<fields>
    <field name="identifier" type="string" indexed="true" stored="true" required="false" multiValued="true"/>
    <field name="title" type="text" indexed="true" stored="true" required="false" multiValued="false"/>
    <field name="layer" type="text" indexed="true" stored="true" required="false" multiValued="false"/>
    [...]
    <field name="poi_name" type="text" indexed="true" stored="true" required="false" multiValued="false"/>
    <field name="poi_code" type="text" indexed="true" stored="true" required="false" multiValued="false"/>

    <!-- copy to search field -->
    <copyField source="identifier" dest="full_text"/>
    <copyField source="title" dest="full_text"/>
    <copyField source="layer" dest="full_text"/>
    <copyField source="poi_name" dest="full_text"/>
    <copyField source="poi_code" dest="full_text"/>
    [...]
</fields>

The following fields are mandatory in the Smart Search schema:

layer: Saves the name of the indexed layer
title: Explicit representation of search results is used

Further, you can add fields synonymously to the above mentioned procedure depending on the indexed ArcGIS Service Layer.

Link layer and schema

The link between the fields of the layer and the schema is established by the file /smartfinder-search/WEB-INF/classes/spring-feature-layer-config.xml. The existing (and executable) example shows how to map the names of the fields of the layer to the fields of the schema. To create a new layer configuration, copy the existing example, paste it into a new file, and then modify it to meet your needs.

The configuration is divided into two parts and consists of a layer configuration (LayerConfiguration) and an indexing configuration (IndexConfiguration). The layer configuration refers to a specific layer. The indexing configuration, on the other hand, contains configurations that can apply to several layers. To reduce redundancy, you can define one or more indexing configurations and reference them in the layer configurations. Within a layer configuration, it is possible to overwrite the settings of the indexing configuration. It is also possible to define all settings exclusively in the layer configuration without defining a separate indexing configuration.

Creating a new layer configuration

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                            http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.0.xsd">
    <!-- Default indexing configuration that can reused for all layer configurations -->
    <bean class="de.conterra.finder.api.arcgis.arcgisserver.IndexingConfiguration" name="defaultIndexingConfiguration">
        <property name="arcGISServerUrl" value="..."/>
        [...]
    </bean>
    <util:list id="layerConfigurations">
        <!-- Configuration for indexing feature layer with id 0 -->
        <bean class="de.conterra.finder.api.arcgis.arcgisserver.LayerConfiguration" name="layerConfigurationFederalStates">
            <!-- The indexing configuration that should be used -->
            <constructor-arg name="indexingConfiguration" ref="defaultIndexingConfiguration"/>
            <!-- Layer id of the layer that should be indexed -->
            <property name="id" value="0"/>
            <!-- Name of that will appear in the UI (faceted search)-->
            <property name="descriptiveName" value="POIs Münster"/>
            [...]
        </bean>
        <!-- Configuration of new custom layer -->
        <bean class="de.conterra.finder.api.arcgis.arcgisserver.LayerConfiguration" name="layerCustomFeatureServer">
            [...]
        </bean>
    </util:list>
</beans>

The following settings can be made for the indexing configuration (IndexingConfiguration):

name (required)

Unique name of the indexing configuration.

Example:

<bean class="de.[...].IndexingConfiguration" name="<CONFIGURATION_NAME>"> [...] </bean>

arcGISServerUrl (required)

URL to the layer of the ArcGIS Feature or Map Server.

Example:

<property name="arcGISServerUrl" value="http://example.de/FeatureServer"/>

indexedFieldsToSchemaMapping (required)

Mapping the fields of the schema to the fields of the layer.
In general: <entry key="<NAME_OF_FIELD_IN_LAYER>" value="<NAME_OF_FIELDS_IN_SCHEMA>"/>
Example:

<property name="indexedFieldsToSchemaMapping">
    <map>
        <entry key="osm_id" value="identifier"/>
        <entry key="code" value="poi_code"/>
        <entry key="name" value="poi_name"/>
    </map>
</property>

The fields osm_id, code and name are attributes of the feature layer, the corresponding fields identifier, poi_code and poi_name are the fields of the schema into which the values are indexed.

titleFields (required)

Appearance of the title field.
This is used to display the hits in the search box.

If you want to create a title for a feature with name="Roads" and code="0428" that is "- Roads (0428)", create the following configuration:

 <property name="titleFields">
    <list>
        <bean class="de.conterra.finder.api.arcgis.arcgisserver.TitleField">
            <property name="name" value="name"/>
            <property name="postDelimiter" value=" ("/>
            <property name="preDelimiter" value="- "/>
        </bean>
        <bean class="de.conterra.finder.api.arcgis.arcgisserver.TitleField">
            <property name="name" value="code"/>
            <property name="postDelimiter" value=")"/>
        </bean>
    </list>
</property>

Use the postDelimiter and preDelimiter properties to create the final title.

pageSize

Maximum number of requested features per request against the layer.

If this value is not set, the value globally configured for the indexing service arcgisServerContentIndexingService from the file /smartfinder-search/WEB-INF/classes/spring-indexing-config.xml is used.

Example:

<property name="pageSize" value="50"/>

indexGeometry

Defines whether the geometry of a feature should be indexed.
Default: true

Example:

<property name="indexGeometry" value="true"/>

translateDomainValues

Defines whether existing domain values of features should be considered as names in the index.
If false, only the code of one field is taken.
Default: false

Example:

<property name="translateDomainValues" value="false"/>

The following settings can be made for the layer configuration (LayerConfiguration):

name (required)

Unique name of the layer configuration.

Example:

<bean class="de.[...].LayerConfiguration" name="<CONFIGURATION_NAME>"> [...] </bean>

indexingConfiguration (required)

Reference to the indexing configuration

Example:

<constructor-arg name="indexingConfiguration" ref="defaultIndexingConfiguration"/>

id (required)

ID of the layer.

Example:

<property name="id" value="0"/>

descriptiveName

Descriptive name of the layer.
This is used to display the categories of search hits.

Example:

<property name="descriptiveName" value="POIs Münster"/>

Indexing of the layer

To index the layer, create an indexing job for a resource of type URL in the Job Manager .

The URL of these jobs points to the Feature Server layer that you previously defined in the layer configuration. Related to this is the URL: <arcGISServerUrl>/<id>.

For the prebuilt example, the URL is: https://services.arcgis.com/ObdAEOfl1Z5LP2D0/arcgis/rest/services/poi_muenster_WFL1/FeatureServer/0

After starting the job, the fields of the layer are indexed according to the configuration.

If the feature layer contains the error code Error: Error performing query operation during indexing it is possible that the heap size of the SOCs in the ArcGIS server is too small to handle the feature response to be provided completely.

You can solve this in two different ways:

Reduction of pageSize in the layer configuration of smart.finder
Adjusting the heap size in ArcGIS Server, see ESRI Support Error: Error performing query operation

Result of the indexing

To see the results open the example app: https://<yourserver>/smartfinder/?lang=en&app=full-screen-map-multicore.
Select the smart search core:
After entering the search term Schule, the search result is as follows:
A click on a search result in the list shows the feature on the map.