Indexing features

smart.finder indexes features from two kinds of sources:

For both sources, you define which attributes of a feature are written to which fields of the Solr schema. This section describes the configuration that is the same for both sources. The specifics of the sources can be found under ArcGIS Server layers and OGC API Features.

In this documentation, feature means an object of a source, that is, a feature of an ArcGIS Server layer or an item of an OGC API Features collection. Likewise, attribute means an attribute of an ArcGIS feature or a property of an OGC API item.

Maintenance and configuration of the search index

smart.finder is delivered with a preconfigured core featuresearch.

The fields required for indexing are automatically added at runtime in accordance with the configuration described below. Manual adjustments to the indexing scheme are not necessary.

If required, manual adjustments can still be carried out as described below.

Adjusting the indexing schema manually

The featuresearch core is located in the directory

[SOLR_HOME]/featuresearch

When managing the index, the managed-schema file is created from an existing schema.xml file (if available) when the application is started. The original file schema.xml is renamed schema.xml.bak and serves as a backup. Only the managed-schema file is actively used, which may not be changed manually. Changes to the file can only be made via the configuration described on this page or via the Solr Admin.

To enable manual adjustments, the smart.finder must first be stopped and the following steps carried out:

  1. Rename file schema.xml.bak to schema.xml

  2. Optionally: Copy the content from the file managed-schema to the file schema.xml if you have made modifications via the Solr Admin.

  3. Delete the managed-schema file

  4. Make adjustments to the schema.xml file

  5. Start smart.finder

After starting, the files are created or renamed again.

Structure of the configuration

You configure the link between the attributes of the features and the fields of the Solr schema in a Spring configuration file in the directory /smartfinder-search/WEB-INF/classes/. Each source has its own file:

Source Configuration file Class of the configuration List of configurations

spring-feature-layer-config.xml

LayerConfiguration

layerConfigurations

spring-ogc-features-config.xml

OgcCollectionConfiguration

ogcCollectionConfigurations

An indexing configuration describes how a layer or a collection is indexed. For example, you define which attributes are added to the Solr schema. Each indexing configuration is a bean element in the list of configurations of the respective file. Only the configurations in this list are considered when indexing.

The configurations of both sources use the same classes to map attributes, to build the title, and to define constant values:

Class Purpose

de.conterra.smartfinder.api.feature.FieldsIndexingConfig

Defines which attributes are indexed, see fieldsConfig (required).

de.conterra.smartfinder.api.feature.FieldToSchemaMapping

Maps an attribute to a field of the Solr schema, see [field-to-schema-mapping].

de.conterra.smartfinder.api.feature.TitleField

Defines how the title is assembled, see titleFields (required).

de.conterra.smartfinder.api.feature.ConstantFieldValue

Writes a constant value to a field of the Solr schema, see constantFieldValues.

Inheritance of settings

Often, several layers or collections of a service have the same attributes. This leads to repeated settings in the indexing configurations. To avoid this, an indexing configuration can inherit the settings of a parent configuration. A specific configuration can still override inherited values.

To do so, define the parent configuration as a bean element outside the list of configurations. Add the following element to each bean element in the list that should inherit the settings:

<constructor-arg name="parentConfiguration" ref="defaultIndexingConfiguration"/>

The value of the ref attribute refers to the name of the parent configuration. Therefore, the bean element of a parent configuration must have a unique name. You set the name with the name attribute of the bean element.

Example:

<bean class="..." name="defaultIndexingConfiguration">
    [...]
</bean>

Common parameters

You specify the parameters of an indexing configuration in single property elements within the bean element. Every property element has a name attribute that contains the name of the parameter. You define the value of the parameter either in a value attribute or as content of the property element.

The following list describes the parameters that are the same for ArcGIS Server layers and OGC API Features. The parameters that apply to only one source are described on the pages ArcGIS Server layers and OGC API Features.

Parameters annotated with required must be set either in the configuration itself or in a referenced parent configuration.

fieldsConfig (required)

Defines which attributes of the features are indexed and how their fields are named in the index.

The value of this parameter is a bean element of type FieldsIndexingConfig. This bean element can contain the properties fetchAllFieldsFromServerExcept and fieldToSchemaMappings.

Example:

<property name="fieldsConfig">
    <bean class="de.conterra.smartfinder.api.feature.FieldsIndexingConfig">
        <property name="fetchAllFieldsFromServerExcept">
            [...]
        </property>
        <property name="fieldToSchemaMappings">
            [...]
        </property>
    </bean>
</property>

The properties fetchAllFieldsFromServerExcept and fieldToSchemaMappings can be used individually or in combination, but at least one of the properties must be defined.

  • fetchAllFieldsFromServerExcept

    A list of attributes that are excluded from indexing.

    Example: Index all attributes except osm_id

    <property name="fetchAllFieldsFromServerExcept">
        <list>
            <value>osm_id</value>
        </list>
    </property>

    An empty list means that all attributes are indexed.

    Example: Index all attributes

    <property name="fetchAllFieldsFromServerExcept">
        <list/>
    </property>
  • fieldToSchemaMappings

    Explicit specification of the attributes to be indexed.

    The value of the property is a list element that contains a bean element of the class FieldToSchemaMapping for each attribute, where each bean has the following property elements:

    featureFieldName (mandatory) The name of the attribute in the source.

    indexFieldName (optional) The name of the field in the Solr schema. If specified, this name is used in the index. If no indexFieldName is specified, the field in the index gets its name from featureFieldName.

    indexFieldType (optional) The type of the field in the Solr schema, for example string or text. Default: text. With the type string, a search term must match the value exactly. This is suitable for facets and filters.

    smart.finder creates the fields in the index.

    Example:

    <property name="fieldToSchemaMappings">
        <list>
            <bean class="de.conterra.smartfinder.api.feature.FieldToSchemaMapping">
                <property name="featureFieldName" value="osm_id"/>
                <property name="indexFieldName" value="identifier"/>
            </bean>
            <bean class="de.conterra.smartfinder.api.feature.FieldToSchemaMapping">
                <property name="featureFieldName" value="code"/>
                <property name="indexFieldName" value="poi_code"/>
            </bean>
            <bean class="de.conterra.smartfinder.api.feature.FieldToSchemaMapping">
                <property name="featureFieldName" value="name"/>
                <property name="indexFieldName" value="poi_name"/>
            </bean>
        </list>
    </property>
titleFields (required)

Defines how the title field is assembled. The title field is used to display the hits in the search box.

To create the title "- Roads (0428)" from a feature with the values name="Roads" and code="0428", use the following configuration:

 <property name="titleFields">
    <list>
        <bean class="de.conterra.smartfinder.api.feature.TitleField">
            <property name="name" value="name"/>
            <property name="postDelimiter" value=" ("/>
            <property name="preDelimiter" value="- "/>
        </bean>
        <bean class="de.conterra.smartfinder.api.feature.TitleField">
            <property name="name" value="code"/>
            <property name="postDelimiter" value=")"/>
        </bean>
    </list>
</property>

The properties preDelimiter and postDelimiter denote the characters that are prepended or appended to the value of the attribute specified with the name property.

If a feature has no value for the specified attributes, smart.finder uses the default title no title.

descriptiveName

Name that appears in the user interface, for example in the faceted search. smart.finder writes it to the index field layer.

Example:

<property name="descriptiveName" value="POIs Münster"/>
indexGeometry

Defines whether the geometry of a feature is indexed.

Default: true

Example:

<property name="indexGeometry" value="true"/>
constantFieldValues

Constant values that smart.finder writes to a field of the index for every feature, regardless of the attributes of the feature. For example, you can use them to index values for facets and filters or the origin of the data.

The value of the parameter is a list element that contains a bean element of the class ConstantFieldValue for each constant value, where each bean has the following property elements:

indexFieldName (mandatory) The name of the field in the Solr schema.

value (conditional) The constant value that is written to the field. You must specify either value or values.

values (conditional) A list of constant values that are written to the field. You must specify either value or values.

indexFieldType (optional) The type of the field in the Solr schema. Default: string, that is, a search term must match the value exactly. This is suitable for facets and filters.

smart.finder creates the fields in the index.

If an attribute of the feature already fills the same index field, the field keeps the value of the feature. smart.finder does not write the constant value in this case.

As with all parameters, your own setting overrides the inherited value: If a configuration defines its own constantFieldValues, this list completely replaces the list of the parent configuration. The two lists are not merged. To keep an inherited value, repeat it in your own list.

If the indexFieldName or the value of a ConstantFieldValue is missing, the configuration is invalid and the indexing is aborted.

Example: All features get the categories POI and Leisure and the data source OSM.

<property name="constantFieldValues">
    <list>
        <bean class="de.conterra.smartfinder.api.feature.ConstantFieldValue">
            <property name="indexFieldName" value="category"/>
            <property name="values">
                 <list>
                     <value>POI</value>
                     <value>Leisure</value>
                 </list>
            </property>
        </bean>
        <bean class="de.conterra.smartfinder.api.feature.ConstantFieldValue">
            <property name="indexFieldName" value="dataSource"/>
            <property name="value" value="OSM"/>
        </bean>
    </list>
</property>

Tips for an extensive configuration

If you prepare many layers or collections for indexing, the configuration file can become very extensive and hard to read. Therefore, you can split the configuration into several files. This way, you configure services by topic, and the individual files are smaller and easier to maintain.

To do so, follow these steps:

  1. Create a separate file for each indexing configuration in the directory /smartfinder-search/WEB-INF/classes/. You can choose the names of these files freely.

  2. Include the files in the central configuration file with the import element.

  3. Add the beans of the imported files to the list of configurations. The value of the bean attribute corresponds to the name attribute of the respective configuration.

Note the following:

  • The name of the central configuration file must remain unchanged, that is, spring-feature-layer-config.xml or spring-ogc-features-config.xml.

  • The name of the list must remain unchanged, that is, layerConfigurations or ogcCollectionConfigurations.

The following example shows the central file spring-ogc-features-config.xml for two collections that are configured in the files spring-ogc-features-geotope.xml and spring-ogc-features-gewaesser.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.0.xsd">

    <import resource="classpath:spring-ogc-features-geotope.xml"/>
    <import resource="classpath:spring-ogc-features-gewaesser.xml"/>

    <util:list id="ogcCollectionConfigurations">
        <ref bean="collectionGeotope"/>
        <ref bean="collectionGewaesser"/>
    </util:list>
</beans>

For a complete example for ArcGIS Server layers, see Splitting the configuration into several files.