Blob Store API | Apache Solr Reference Guide 8.11

The Blob Store REST API provides REST methods to store, retrieve or list files in a Lucene index.

It can be used to upload a jar file which contains standard Solr components such as RequestHandlers, SearchComponents, or other custom code you have written for Solr. Schema components do not yet support the Blob Store.

When using the blob store, note that the API does not delete or overwrite a previous object if a new one is uploaded with the same name. It always adds a new version of the blob to the index. Because the .system collection is a standard Solr collection, deleting blobs is the same as deleting documents.

The blob store is only available when running in SolrCloud mode. Solr in standalone mode does not support use of a blob store.

The blob store API is implemented as a requestHandler. A special collection named ".system" is used to store the blobs. This collection can be created in advance, but if it does not exist it will be created automatically.

About the .system Collection

Before uploading blobs to the blob store, a special collection must be created and it must be named .system. Solr will automatically create this collection if it does not already exist, but you can also create it manually if you choose.

The BlobHandler is automatically registered in the .system collection. The solrconfig.xml, Schema, and other configuration files for the collection are automatically provided by the system and don’t need to be defined specifically.

If you do not use the -shards or -replicationFactor options, then defaults of numShards=1 and replicationFactor=3 (or maximum nodes in the cluster) will be used.

You can create the .system collection with the CREATE command of the Collections API, as in this example:

V1 API

curl http://localhost:8983/solr/admin/collections?action=CREATE&name=.system&replicationFactor=2&numShards=2

Note that this example will create the .system collection across 2 shards with a replication factor of 2; you may need to customize this for your Solr implementation.

V2 API

curl -X POST -H 'Content-type: application/json' -d '{"create": {"name": ".system", "numShards": "2", "replicationFactor": "2"}}' http://localhost:8983/api/collections

Note that this example will create the .system collection across 2 shards with a replication factor of 2; you may need to customize this for your Solr implementation.

The bin/solr script cannot be used to create the .system collection.

Upload Files to Blob Store

After the .system collection has been created, files can be uploaded to the blob store with a request similar to the following:

V1 API

curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @{filename} http://localhost:8983/solr/.system/blob/{blobname}

For example, to upload a file named "test1.jar" as a blob named "test", you would make a POST request like:

curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @test1.jar http://localhost:8983/solr/.system/blob/test

V2 API

curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @{filename} http://localhost:8983/api/collections/.system/blob/{blobname}

For example, to upload a file named "test1.jar" as a blob named "test", you would make a POST request like:

curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @test1.jar http://localhost:8983/api/collections/.system/blob/test

Note that by default, the blob store has a limit of 5Mb for any blob. This can be increased if necessary by changing the value for the maxSize setting in solrconfig.xml for the .system collection. See the section Configuring solrconfig.xml for information about how to modify solrconfig.xml for any collection.

A GET request will return the list of blobs and other details:

V1 API

For all blobs:

curl http://localhost:8983/solr/.system/blob?omitHeader=true

For a single blob:

curl http://localhost:8983/solr/.system/blob/test?omitHeader=true

Output:

{
  "response":{"numFound":1,"start":0,"docs":[
      {
        "id":"test/1",
        "md5":"20ff915fa3f5a5d66216081ae705c41b",
        "blobName":"test",
        "version":1,
        "timestamp":"2015-02-04T16:45:48.374Z",
        "size":13108}]
  }
}

V2 API

For all blobs:

curl http://localhost:8983/api/collections/.system/blob?omitHeader=true

For a single blob:

curl http://localhost:8983/api/collections/.system/blob/test?omitHeader=true

Output:

{
  "response":{"numFound":1,"start":0,"docs":[
      {
        "id":"test/1",
        "md5":"20ff915fa3f5a5d66216081ae705c41b",
        "blobName":"test",
        "version":1,
        "timestamp":"2015-02-04T16:45:48.374Z",
        "size":13108}]
  }
}

The filestream response writer can retrieve a blob for download, as in:

V1 API

For a specific version of a blob, include the version to the request:

curl http://localhost:8983/solr/.system/blob/{blobname}/{version}?wt=filestream > {outputfilename}

For the latest version of a blob, the {version} can be omitted:

curl http://localhost:8983/solr/.system/blob/{blobname}?wt=filestream > {outputfilename}

V2 API For a specific version of a blob, include the version to the request:

curl http://localhost:8983/api/collections/.system/blob/{blobname}/{version}?wt=filestream > {outputfilename}

For the latest version of a blob, the {version} can be omitted:

curl http://localhost:8983/api/collections/.system/blob/{blobname}?wt=filestream > {outputfilename}

Use a Blob in a Handler or Component

To use the blob as the class for a request handler or search component, you create a request handler in solrconfig.xml as usual. You will need to define the following parameters:

class: the fully qualified class name. For example, if you created a new request handler class called CRUDHandler, you would enter org.apache.solr.core.CRUDHandler.
runtimeLib: Set to true to require that this component should be loaded from the classloader that loads the runtime jars.

For example, to use a blob named test, you would configure solrconfig.xml like this:

<requestHandler name="/myhandler" class="org.apache.solr.core.myHandler" runtimeLib="true" version="1">
</requestHandler>

If there are parameters available in the custom handler, you can define them in the same way as any other request handler definition.

Blob store can only be used to dynamically load components configured in solrconfig.xml. Components specified in schema.xml cannot be loaded from blob store.

Deleting Blobs

Once loaded to the blob store, blobs are handled very similarly to usual indexed documents in Solr. To delete blobs, you can use the same approaches used to delete individual documents from the index, namely Delete By ID and Delete By Query.

For example, to delete a blob with the id test/1, you would issue a command like this:

curl -H 'Content-Type: application/json' -d '{"delete": {"id": "test/1"}}' http://localhost:8983/solr/.system/update?commit=true

Be sure to tell Solr to perform a commit as part of the request (commit=true in the above example) to see the change immediately. If you do not instruct Solr to perform a commit, Solr will use the .system collection autoCommit settings, which may not be the expected behavior.

You can also use the delete by query syntax, as so:

curl -H 'Content-Type: application/json' -d '{"delete": {"query": "id:test/1"}}' http://localhost:8983/solr/.system/update?commit=true

For more on deleting documents generally, see the section Sending JSON Update Commands.