Spatial Search

Indexing GeoJSON and WKT

Using the bin/post tool:

bin/post -type "application/json" -url "http://localhost:8983/solr/mycollection/update?format=geojson" /path/to/geojson.file

The key parameter to pass in with your request is:

geofilt

The geofilt filter allows you to retrieve results based on the geospatial distance (AKA the "great circle distance") from a given point. Another way of looking at it is that it creates a circular shape filter. For example, to find all documents within five kilometers of a given lat/lon point, you could enter:

&q=*:*&fq={!geofilt sfield=store}&pt=45.15,-93.85&d=5

This filter returns all results within a circle of the given radius around the initial point:

bbox

The bbox filter is very similar to geofilt except it uses the bounding box of the calculated circle. See the blue box in the diagram below. It takes the same parameters as geofilt.

Here’s a sample query:

&q=*:*&fq={!bbox sfield=store}&pt=45.15,-93.85&d=5

The rectangular shape is faster to compute and so it’s sometimes used as an alternative to geofilt when it’s acceptable to return points outside of the radius. However, if the ideal goal is a circle but you want it to run faster, then instead consider using the RPT field and try a large distErrPct value like 0.1 (10% radius). This will return results outside the radius but it will do so somewhat uniformly around the shape.

Filtering by an Arbitrary Rectangle

Sometimes the spatial search requirement calls for finding everything in a rectangular area, such as the area covered by a map the user is looking at. For this case, geofilt and bbox won’t cut it. This is somewhat of a trick, but you can use Solr’s range query syntax for this by supplying the lower-left corner as the start of the range and the upper-right corner as the end of the range.

Here’s an example:

&q=*:*&fq=store:[45,-94 TO 46,-93]

LatLonType (deprecated) does not support rectangles that cross the dateline. For RPT and BBoxField, if you are non-geospatial coordinates (geo="false") then you must quote the points due to the space, e.g., "x y".

Optimizing: Cache or Not

It’s most common to put a spatial query into an "fq" parameter – a filter query. By default, Solr will cache the query in the filter cache.

If you know the filter query (be it spatial or not) is fairly unique and not likely to get a cache hit then specify cache="false" as a local-param as seen in the following example. The only spatial types which stand to benefit from this technique are LatLonPointSpatialField and LatLonType (deprecated). Enable docValues on the field (if it isn’t already). LatLonType (deprecated) additionally requires a cost="100" (or more) local-param.

&q=...mykeywords...&fq=...someotherfilters...&fq={!geofilt cache=false}&sfield=store&pt=45.15,-93.85&d=5

LLPSF does not support Solr’s "PostFilter".

geodist

geodist is a distance function that takes three optional parameters: (sfield,latitude,longitude). You can use the geodist function to sort results by distance or score return results.

For example, to sort your results by ascending distance, use a request like:

&q=*:*&fq={!geofilt}&sfield=store&pt=45.15,-93.85&d=50&sort=geodist() asc

To return the distance as the document score, use a request like:

&q={!func}geodist()&sfield=store&pt=45.15,-93.85&sort=score+asc&fl=*,score

Use as a Sub-Query to Expand Search Results

Here we will query for results in Jacksonville, Florida, or within 50 kilometers of 45.15,-93.85 (near Buffalo, Minnesota):

&q=*:*&fq=(state:"FL" AND city:"Jacksonville") OR {!geofilt}&sfield=store&pt=45.15,-93.85&d=50&sort=geodist()+asc

Facet by Distance

To facet by distance, you can use the frange query parser:

&q=*:*&sfield=store&pt=45.15,-93.85&facet.query={!frange l=0 u=5}geodist()&facet.query={!frange l=5.001 u=3000}geodist()

There are other ways to do it too, like using a {!geofilt} in each facet.query.

Boost Nearest Results

Using the DisMax or Extended DisMax, you can combine spatial search with the boost function to boost the nearest results:

&q.alt=*:*&fq={!geofilt}&sfield=store&pt=45.15,-93.85&d=50&bf=recip(geodist(),2,200,20)&sort=score desc

Schema Configuration for RPT

To use RPT, the field type must be registered and configured in schema.xml. There are many options for this field type.

And there are others: normWrapLongitude, datelineRule, validationRule, autoIndex, allowMultiOverlap, precisionModel. For further info, see notes below about spatialContextFactory implementations referenced above, especially the link to the JTS based one.

Standard Shapes

The RPT field types support a set of standard shapes: points, circles (aka buffered points), envelopes (aka rectangles or bounding boxes), line strings, polygons, and "multi" variants of these. The envelopes and line strings are Euclidean/cartesian (flat 2D) shapes. Underlying Solr is the Spatial4j library which implements them. To support other shapes, you can configure the spatialContextFactory attribute on the field type to reference other options. Two are available: JTS and Geo3D.

JTS and Polygons (flat)

The JTS Topology Suite is a popular computational geometry library with a Euclidean/cartesian (flat 2D) model. It supports a variety of shapes including polygons, buffering shapes, and some invalid polygon repair fall-backs. With the help of Spatial4j, included with Solr, the polygons support dateline (anti-meridian) crossing. You must download it (a JAR file) and put that in a special location internal to Solr: SOLR_INSTALL/server/solr-webapp/webapp/WEB-INF/lib/. You can readily download it here: https://mvnrepository.com/artifact/org.locationtech.jts/jts-core/1.15.0. It will not work if placed in other more typical Solr lib directories, unfortunately.

Set the spatialContextFactory attribute on the field type to JTS.

When activated, there are additional configuration attributes available; see org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory for the Javadocs, and remember to look at the superclass’s options as well. One option in particular you should most likely enable is autoIndex (i.e., use JTS’s PreparedGeometry) as it’s been shown to be a major performance boost for non-trivial polygons.

<fieldType name="location_rpt"   class="solr.SpatialRecursivePrefixTreeFieldType"
               spatialContextFactory="JTS"
               autoIndex="true"
               validationRule="repairBuffer0"
               distErrPct="0.025"
               maxDistErr="0.001"
               distanceUnits="kilometers" />

Once the field type has been defined, define a field that uses it.

Here’s an example polygon query for a field "geo" that can be either solr.SpatialRecursivePrefixTreeFieldType or RptWithGeometrySpatialField:

&q=*:*&fq={!field f=geo}Intersects(POLYGON((-10 30, -40 40, -10 -20, 40 20, 0 0, -10 30)))

Inside the parenthesis following the search predicate is the shape definition. The format of that shape is governed by the format attribute on the field type, defaulting to WKT. If you prefer GeoJSON, you can specify that instead.

Beyond this Reference Guide and Spatila4j’s docs, there are some details that remain at the Solr Wiki at https://cwiki.apache.org/confluence/display/solr/SolrAdaptersForLuceneSpatial4.

Geo3D and Polygons (on the ellipsoid)

Geo3D is the colloquial name of the Lucene spatial-3d module, included with Solr. It’s a computational geometry library implementing a variety of shapes (including polygons) on a sphere or WGS84 ellipsoid. Geo3D is particularly suited for spatial applications where the geometries cover large distances across the globe or are near the poles. Geo3D is named as-such due to its internal implementation that uses geocentric coordinates (X,Y,Z), not for 3-dimensional geometry, which it does not support. Despite these internal details, you still supply latitude and longitude as you would normally in Solr.

Set the spatialContextFactory attribute on the field type to Geo3D.

<fieldType name="geom"
  class="solr.SpatialRecursivePrefixTreeFieldType"
  spatialContextFactory="Geo3D"
  prefixTree="s2"
  planetModel="WGS84"/><!-- or "sphere" -->

Once the field type has been defined, define a field that uses it.

The prefixTree="s2" setting is optional and only possible with Geo3D. It was developed with Geo3D in mind to be more efficient than the other grids.

RptWithGeometrySpatialField

The RptWithGeometrySpatialField field type is a derivative of SpatialRecursivePrefixTreeFieldType that also stores the original geometry internally in Lucene DocValues, which it uses to achieve accurate search. It can also be used for indexed point fields. The Intersects predicate (the default) is particularly fast, since many search results can be returned as an accurate hit without requiring a geometry check. This field type is configured just like RPT except that the default distErrPct is 0.15 (higher than 0.025) because the grid squares are purely for performance and not to fundamentally represent the shape.

An optional in-memory cache can be defined in solrconfig.xml, which should be done when the data tends to have shapes with many vertices. Assuming you name your field "geom", you can configure an optional cache in solrconfig.xml by adding the following – notice the suffix of the cache name:

<cache name="perSegSpatialFieldCache_geom"
           class="solr.LRUCache"
           size="256"
           initialSize="0"
           autowarmCount="100%"
           regenerator="solr.NoOpRegenerator"/>

When using this field type, you will likely not want to mark the field as stored because it’s redundant with the DocValues data and surely larger because of the formatting (be it WKT or GeoJSON). To retrieve the spatial data in search results from DocValues, use the [geo] transformer — Transforming Result Documents.

Heatmap Faceting

The RPT field supports generating a 2D grid of facet counts for documents having spatial data in each grid cell. For high-detail grids, this can be used to plot points, and for lesser detail it can be used for heatmap generation. The grid cells are determined at index-time based on RPT’s configuration. At facet counting time, the indexed cells in the region of interest are traversed and a grid of counters corresponding to each cell are incremented. Solr can return the data in a straight-forward 2D array of integers or in a PNG which compresses better for larger data sets but must be decoded.

The heatmap feature is accessible both from Solr’s standard faceting feature, plus the newer more flexible JSON Facet API. We’ll proceed now with standard faceting. As a part of faceting, it supports the key local parameter as well as excluding tagged filter queries, just like other types of faceting do. This allows multiple heatmaps to be returned on the same field with different filters.

Here’s some sample output in JSON (with "…​" inserted for brevity):

{gridLevel=6,columns=64,rows=64,minX=-180.0,maxX=180.0,minY=-90.0,maxY=90.0,
counts_ints2D=[[0, 0, 2, 1, ....],[1, 1, 3, 2, ...],...]}

The output shows the gridLevel which is interesting since it’s often computed from other parameters. If an interface being developed allows an explicit resolution increase/decrease feature then subsequent requests can specify the gridLevel explicitly.

The minX, maxX, minY, maxY reports the region where the counts are. This is the minimally enclosing bounding rectangle of the input geom at the target grid level. This may wrap the dateline. The columns and rows values are how many columns and rows that the output rectangle is to be divided by evenly. Note: Don’t divide an on-screen projected map rectangle evenly to plot these rectangles/points since the cell data is in the coordinate space of decimal degrees if geo=true or whatever units were given if geo=false. This could be arranged to be the same as an on-screen map but won’t necessarily be.

The counts_ints2D key has a 2D array of integers. The initial outer level is in row order (top-down), then the inner arrays are the columns (left-right). If any array would be all zeros, a null is returned instead for efficiency reasons. The entire value is null if there is no matching spatial data.

If format=png then the output key is counts_png. It’s a base-64 encoded string of a 4-byte PNG. The PNG logically holds exactly the same data that the ints2D format does. Note that the alpha channel byte is flipped to make it easier to view the PNG for diagnostic purposes, since otherwise counts would have to exceed 2^24 before it becomes non-opague. Thus counts greater than this value will become opaque.