JSON Combined Query DSL

The Combined Query feature aims to execute multiple queries of multiple kinds across multiple shards of a collection and combine their result basis an algorithm (like Reciprocal Rank Fusion). It is extending JSON Query DSL ultimately enabling Hybrid Search.

This feature is currently unsupported for grouping and Cursors.

This feature works in both Standalone and SolrCloud modes and always performs distributed search execution. In Standalone (user-managed) mode, shard URLs must be explicitly allow-listed using the allowUrls parameter, otherwise Solr returns HTTP 403. For example:

"-Dsolr.security.allow.urls=http://localhost:8983/solr/"

Configuration Requirements

Combined Query Feature has a separate handler with class solr.CombinedQuerySearchHandler which can be configured as below:

<requestHandler name="/search" class="solr.CombinedQuerySearchHandler">
.....
</requestHandler>

In addition, the QueryComponent has been extended to create a new CombinedQueryComponent, which must be declared as a search component:

<searchComponent class="solr.CombinedQueryComponent" name="combined_query">
    <int name="maxCombinerQueries">2</int>
</searchComponent>

The Search Component also accepts parameters as below:

maxCombinerQueries: This parameter can be set to enforce an upper limit on the number of queries defined in combiner.query. It defaults to 5 if not set.

Query DSL Structure

The query structure is similar to JSON Query DSL except for how multiple queries are defined along with their parameters.

Multiple queries can be defined under the queries key by providing their name with the same syntax as a single query is defined with the key query.
In addition to the other supported parameters, there are several parameters which can be defined under params key as below:

combiner | Default: false

Enables the combined query mode when set to true.

combiner.query

The list of queries to be executed as defined in the queries key. Example: ["query1", "query2"]

combiner.algorithm | Default: rrf

The algorithm to be used for combining the results. Reciprocal Rank Fusion (RRF) is the in-built fusion algorithm. Any other algorithm can be configured using plugin.

combiner.rrf.k | Default: 60

The k parameter in the RRF algorithm.

Consider setting shards.rows to at least your deepest expected start + rows to enlarge the per-shard candidate pool used for fusion, giving more consistent ordering across pages.

Example

Below is an example with sample JSON query payload:

curl
SolrJ

curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
    "queries": {
        "lexical1": {
            "lucene": {
                "query": "name:apache"
            }
        },
        "vector": {
            "knn": {
                "f": "vector",
                "topK" :5,
                "query": "[0.1,-0.34,0.89,0.02]"
            }
        }
    },
    "limit": 5,
    "fields": ["id", "score", "name"],
    "params": {
        "combiner": true,
        "combiner.query": ["lexical1", "vector"],
        "combiner.algorithm": "rrf",
        "combiner.rrf.k": "15"
    }
}'

final Map<String, Object> queriesMap = new HashMap<>();
queriesMap.put("lexical1", Map.of("lucene", Map.of("query", "apache", "df", "name")));
queriesMap.put("vector", Map.of("knn", Map.of("query", [0.1,-0.34,0.89,0.02], "f", "vector", "topK", 5)));
final JsonQueryRequest query =
    new JsonQueryRequest()
        .setQueries(queriesMap)
        .withParam("id", "score", "name")
        .setLimit(5)
        .withParam("combiner", "true")
        .withParam("combiner.query", List.of("lexical1", "vector"))
        .withParam("combiner.algorithm", "rrf")
        .withParam("combiner.rrf.k", "15");
QueryResponse queryResponse = query.process(solrClient, COLLECTION_NAME);

Combiner Algorithm Plugin

As mentioned above, custom algorithms can be configured to combine the results across multiple queries using a Solr plugin.

The class to implement the custom logic has to extend org.apache.solr.handler.component.combine.QueryAndResponseCombiner, which is an abstract base class that provides a framework for implementing various algorithms used to merge ranked lists and shard documents.

The Combined Query Component definition takes the combiners parameter, where the custom class can be declared by specifying a name and the custom parameters required by the custom algorithm.

Example of the Search Component as below:

<searchComponent class="solr.CombinedQueryComponent" name="combined_query">
    <int name="maxCombinerQueries">2</int>
        <lst name="combiners">
           <lst name="customAlgorithm">
                <str name="class">org.apache.solr.handler.component.combine.CustomCombiner</str>
                <int name="customParam1">35</int>
                <str name="customParam2">customValue</str>
            </lst>
        </lst>
</searchComponent>

Then, when executing the combined query, the only thing that changes in the JSON query payload is the value specified in the combiner.algorithm parameter:

    ...
    "params": {
        "combiner": true,
        "combiner.query": ["lexical1", "vector"],
        "combiner.algorithm": "customAlgorithm"
    }
...

In this case, customAlgorithm is specified which is the name defined in the configuration; the RRF-specific parameters do not need to be provided.

Additional Resources

Blog posts: