JSON Query DSL
Queries and filters provided in JSON requests can be specified using a rich, powerful query DSL.
Query DSL Structure
The JSON Request API accepts query values in three different formats:
-
A valid query string that uses the default
deftype
(lucene
, in most cases), e.g.,title:solr
. -
A valid local params query string that specifies its
deftype
explicitly, e.g.,{!dismax qf=title}solr
. -
A valid JSON object with the name of the query parser and any relevant parameters, e.g.,
{ "lucene": {"df":"title", "query":"solr"}}
.-
The top level "query" JSON block generally only has a single property representing the name of the query parser to use. The value for the query parser property is a child block containing any relevant parameters as JSON properties. The whole structure is analogous to a "local-params" query string. The query itself (often represented in local params using the name
v
) is specified with the keyquery
instead.
-
All of these syntaxes can be used to specify queries for either the JSON Request API’s query
or filter
properties.
Query DSL Examples
The examples below show how to use each of the syntaxes discussed above to represent a query.
Each snippet represents the same basic search: the term iPod
in a field called name
:
Using the standard query API, with a simple query string:
curl
curl -X GET "http://localhost:8983/solr/techproducts/query?q=name:iPod"
SolrJ
final SolrQuery query = new SolrQuery("name:iPod");
final QueryResponse response = solrClient.query(COLLECTION_NAME, query);
Using the JSON Request API, with a simple query string:
curl
curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query" : "name:iPod"
}'
SolrJ
final JsonQueryRequest query = new JsonQueryRequest().setQuery("name:iPod");
final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
Using the JSON Request API, with a local-params string:
curl
curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query": "{!lucene df=name v=iPod}"
}'
SolrJ
final JsonQueryRequest query = new JsonQueryRequest().setQuery("{!lucene df=name}iPod");
final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
Using the JSON Request API, with a fully expanded JSON object:
curl
curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query": {
"lucene": {
"df": "name",
"query": "iPod"
}
}
}'
SolrJ
final Map<String, Object> queryTopLevel = new HashMap<>();
final Map<String, Object> luceneQueryProperties = new HashMap<>();
queryTopLevel.put("lucene", luceneQueryProperties);
luceneQueryProperties.put("df", "name");
luceneQueryProperties.put("query", "iPod");
final JsonQueryRequest query = new JsonQueryRequest().setQuery(queryTopLevel);
final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
Nested Queries
Many of Solr’s query parsers allow queries to be nested within one another. When these are used, requests using the standard query API quickly become hard to write, read, and understand. These sorts of queries are often much easier to work with in the JSON Request API.
Nested Boost Query Example
As an example, consider the three requests below, which wrap a simple query (the term iPod
in the field name
) within a boost query:
Using the standard query API:
curl
curl -X GET "http://localhost:8983/solr/techproducts/query?q={!boost b=log(popularity) v=\'{!lucene df=name}iPod\'}"
SolrJ
final SolrQuery query =
new SolrQuery("{!boost b=log(popularity) v=\'{!lucene df=name}iPod\'}");
final QueryResponse response = solrClient.query(COLLECTION_NAME, query);
Using the JSON Request API, with a mix of fully-expanded and local-params queries.
As you can see, the special key v
is replaced with the key query
:
curl
curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query" : {
"boost": {
"query": {!lucene df=name}iPod,
"b": "log(popularity)"
}
}
}'
SolrJ
final Map<String, Object> queryTopLevel = new HashMap<>();
final Map<String, Object> boostQuery = new HashMap<>();
queryTopLevel.put("boost", boostQuery);
boostQuery.put("b", "log(popularity)");
boostQuery.put("query", "{!lucene df=name}iPod");
final JsonQueryRequest query = new JsonQueryRequest().setQuery(queryTopLevel);
final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
Using the JSON Request API, with all queries fully expanded as JSON:
+
curl
curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query": {
"boost": {
"query": {
"lucene": {
"df": "name",
"query": "iPod"
}
},
"b": "log(popularity)"
}
}
}'
SolrJ
final Map<String, Object> queryTopLevel = new HashMap<>();
final Map<String, Object> boostProperties = new HashMap<>();
final Map<String, Object> luceneTopLevel = new HashMap<>();
final Map<String, Object> luceneProperties = new HashMap<>();
queryTopLevel.put("boost", boostProperties);
boostProperties.put("b", "log(popularity)");
boostProperties.put("query", luceneTopLevel);
luceneTopLevel.put("lucene", luceneProperties);
luceneProperties.put("df", "name");
luceneProperties.put("query", "iPod");
final JsonQueryRequest query = new JsonQueryRequest().setQuery(queryTopLevel);
final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
Nested Boolean Query Example
Query nesting is commonly seen when combining multiple query clauses together using pseudo-boolean logic with the BoolQParser.
The example below shows how the BoolQParser
can be used to create powerful nested queries.
In this example, a user searches for results with iPod
in the field name
which are not in the bottom half of the popularity
rankings.
curl
curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query": {
"bool": {
"must": [
{"lucene": {"df": "name", query: "iPod"}}
],
"must_not": [
{"frange": {"l": "0", "u": "5", "query": "popularity"}}
]
}
}
}'
SolrJ
final Map<String, Object> queryTopLevel = new HashMap<>();
final Map<String, Object> boolProperties = new HashMap<>();
final List<Object> mustClauses = new ArrayList<>();
final List<Object> mustNotClauses = new ArrayList<>();
final Map<String, Object> frangeTopLevel = new HashMap<>();
final Map<String, Object> frangeProperties = new HashMap<>();
queryTopLevel.put("bool", boolProperties);
boolProperties.put("must", mustClauses);
mustClauses.add("name:iPod");
boolProperties.put("must_not", mustNotClauses);
frangeTopLevel.put("frange", frangeProperties);
frangeProperties.put("l", 0);
frangeProperties.put("u", 5);
frangeProperties.put("query", "popularity");
mustNotClauses.add(frangeTopLevel);
final JsonQueryRequest query = new JsonQueryRequest().setQuery(queryTopLevel);
final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
If lucene
is the default query parser, the example above can be simplified to:
curl
curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query": {
"bool": {
"must": [
"name:iPod"
],
"must_not": "{!frange l=0 u=5}popularity"
}
}
}'
SolrJ
final Map<String, Object> queryTopLevel = new HashMap<>();
final Map<String, Object> boolProperties = new HashMap<>();
queryTopLevel.put("bool", boolProperties);
boolProperties.put("must", "name:iPod");
boolProperties.put("must_not", "{!frange l=0 u=5}popularity");
final JsonQueryRequest query = new JsonQueryRequest().setQuery(queryTopLevel);
final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
Example of referencing additional queries, tagging, and exclusions:
curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"queries": {
"query_filters":[ // 1.
{"#size_tag":{"field":{"f":"size","query":"XL"}}},
{"#color_tag":{"field":{"f":"color","query":"Red"}}} // 2.
]
},
"query": {
"bool": {
"must": {"param":"query_filters"}, // refer both of 1.
"excludeTags": "color_tag" // excluding 2.
}
}
}'
Thus, the query above will return only docs matching size:XL
.
Filter Queries
The syntaxes discussed above can also be used to specify query filters (under the filter
key) in addition to the main query itself.
For example, the above query can be rewritten using a filter clause as:
curl
curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query": {
"bool": {
"must_not": "{!frange l=0 u=5}popularity"
}
},
"filter: [
"name:iPod"
]
}'
SolrJ
final Map<String, Object> queryTopLevel = new HashMap<>();
final Map<String, Object> boolProperties = new HashMap<>();
queryTopLevel.put("bool", boolProperties);
boolProperties.put("must_not", "{!frange l=0 u=5}popularity");
final JsonQueryRequest query =
new JsonQueryRequest().setQuery(queryTopLevel).withFilter("name:iPod");
final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
Additional Queries
Multiple additional queries can be specified under queries
key with the same syntax alternatives described above.
Every entry could have multiple values in an array.
To reference these queries, use {"param":"query_name"}
, or old-style referencing "{!v=$query_name}"
.
Beware of arity for these references.
Depending on the context, a reference might be resolved into the first element in the array ignoring the later elements, e.g., if one changes the reference below from {"param":"electronic"}
to {"param":"manufacturers"}
, it’s equivalent to querying for manu:apple
, ignoring the later query.
These queries don’t impact the query result until explicit referencing.
curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"queries": {
"electronic": {"field": {"f":"cat", "query":"electronics"}},
"manufacturers": [
"manu:apple",
{"field": {"f":"manu", "query":"belkin"}}
]
},
"query":{"param":"electronic"}
}'
Overall this example doesn’t make much sense, but just demonstrates the syntax. This feature is useful in filtering domain in JSON Facet API domain changes. Note that these declarations add request parameters underneath, so using same names with other parameters might cause unexpected behavior.
Tagging in JSON Query DSL
Query and filter clauses can also be individually "tagged". Tags serve as handles for query clauses, allowing them to be referenced from elsewhere in the request. This is most commonly used by the filter-exclusion functionality offered by both traditional and JSON faceting.
Queries and filters are tagged by wrapping them in a surrounding JSON object.
The name of the tag is specified as a JSON key, with the query string (or object) becoming the value associated with that key.
Tag name properties are prefixed with a hash, and may include multiple tags, separated by commas.
For example: {"#title,tag2,tag3":"title:solr"}
.
Note that unlike the rest of the JSON request API which uses lax JSON parsing rules, tags must be surrounded by double-quotes because of the leading #
character.
The example below creates two tagged clauses: titleTag
and inStockTag
.
curl
curl -X POST http://localhost:8983/solr/techproducts/select -d '
{
"query": "*:*",
"filter": [
{
"#titleTag": "name:Solr"
},
{
"#inStockTag": "inStock:true"
}
]
}'
SolrJ
final Map<String, Object> titleTaggedQuery = new HashMap<>();
titleTaggedQuery.put("#titleTag", "name:Solr");
final Map<String, Object> inStockTaggedQuery = new HashMap<>();
inStockTaggedQuery.put("#inStockTag", "inStock:true");
final JsonQueryRequest query =
new JsonQueryRequest()
.setQuery("*:*")
.withFilter(titleTaggedQuery)
.withFilter(inStockTaggedQuery);
final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
Note that the tags created in the example above have no impact in how the search is executed. Tags will not affect a query unless they are referenced by some other part of the request that uses them.
Faceting Nested Documents
This paragraph binds many features together. Basically it’s an example of filter exclusions for nested documents when facets are counted over child document fields, but facet counts roll up in parent document counts. A typical scenario where you might need to do this is if you are displaying filters for products (parents) with SKUs with various colour/size options (children). Let’s go on item by item:
-
Filter exclusion is usually necessary when multiple filter values can be applied to each field. This is also also known as drill-sideways facets. See also tagging and filter exclusion.
-
Nested documents, or child documents, are described in Indexing Nested Documents. In the example below, they are referred as SKUs since this is a frequent use case for this feature.
-
Counting facets over children documents means that even though the result of the search are usually parent documents, the children documents and their fields are used for faceting.
-
Facet counts roll up means that child documents contribute facet hits because parent documents they are linked to are counted. For example, if a parent doc has two (2) red children, it’s counted as one (1) due to rollup.
{
"queries": {
"parents":"content_type:parentDocument", (1)
"sku_fqs": [{"#sku_attr1_tag":"sku_attr1:foo"}, (2)
{"#sku_attr2_tag":"sku_attr2:bar"}
]
},
"filter": {
"#sku_filters":{ (3)
"parent": {
"which": {"param":"parents"},
"filters": {"param":"sku_fqs"}
}}
},
"facet": {
"sku_attr1": { (4)
"type":"terms",
"field":"sku_attr1",
"limit":-1,
"domain": { (5)
"excludeTags":"sku_filters",
"blockChildren":"{!v=$parents}",
"filter":
"{!bool filter=$sku_fqs excludeTags=sku_attr1_tag}"
},
"facet": {
"by_parent":"uniqueBlock({!v=$parents})" (6)
}
}
}
}
1 | Defines a parents filter via Additional Queries for later usage. |
2 | Declares two SKU filters under different #tags, as described in Tagging in JSON Query DSL. |
3 | Defines tag block join query for later exclusion.
This joins SKUs to top level docs, and refers to both the parents query and the SKU filter queries defined earlier. |
4 | Calculates a drill-sideways terms facet over SKU documents.
The SKU field is defined as sku_attr1 , and we’ve set limit=-1 so we get all facet values. |
5 | Removes the top-level parent filter in the domain with excludeTags .
The blockChildren value references the all-parents query.
Then we define a filter to restrict to SKU docs but exclude one tag retaining only sku_attr2:bar . |
6 | Counts the subfacet in parent documents. See also uniqueBlock() and Block Join Counts. |