Adding Custom Plugins in SolrCloud Mode

Uploading Jar Files

Use your own service to host the jars or you can use Solr itself to host the jars.

Use the Blob Store API to upload your jar files to Solr. This will to put your jars in the .system collection and distribute them across your SolrCloud nodes. These jars are added to a separate classloader and only accessible to components that are configured with the property runtimeLib=true. These components are loaded lazily because the .system collection may not be loaded when a particular core is loaded.

Config API Commands to use Jars as Runtime Libraries

The runtime library feature uses a special set of commands for the Config API to add, update, or remove jar files currently available in the blob store to the list of runtime libraries.

The following commands are used to manage runtime libs:

curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
   "add-runtimelib": { "name":"jarblobname", "version":2 },
   "update-runtimelib": { "name":"jarblobname", "version":3 },
   "delete-runtimelib": "jarblobname"
}'

curl http://localhost:8983/api/collections/techproducts/config -H 'Content-type:application/json' -d '{
   "add-runtimelib": { "name":"jarblobname", "version":2 },
   "update-runtimelib": { "name":"jarblobname", "version":3 },
   "delete-runtimelib": "jarblobname"
}'

The name to use is the name of the blob that you specified when you uploaded your jar to the blob store. You should also include the version of the jar found in the blob store that you want to use. These details are added to configoverlay.json.

The default SolrResourceLoader does not have visibility to the jars that have been defined as runtime libraries. There is a classloader that can access these jars which is made available only to those components which are specially annotated.

Every pluggable component can have an optional extra attribute called runtimeLib=true, which means that the components are not loaded at core load time. Instead, they will be loaded on demand. If all the dependent jars are not available when the component is loaded, an error is thrown.

This example shows creating a ValueSourceParser using a jar that has been loaded to the Blob store.

curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json' -d '{
  "create-valuesourceparser": {
    "name": "nvl",
    "runtimeLib": true,
    "class": "solr.org.apache.solr.search.function.NvlValueSourceParser,
    "nvlFloatValue": 0.0 }
}'

curl http://localhost:8983/api/collections/techproducts/config -H 'Content-type:application/json' -d '{
  "create-valuesourceparser": {
    "name": "nvl",
    "runtimeLib": true,
    "class": "solr.org.apache.solr.search.function.NvlValueSourceParser,
    "nvlFloatValue": 0.0 }
}'

Example: Using external service to host your jars

Hosting your jars externally is more convenient if you have a reliable server to host your jars. There is no need to create and manage a .system collection.

Step 1: Download a jar from github to the current directory

 curl -o runtimelibs.jar   -LO https://github.com/apache/lucene-solr/blob/master/solr/core/src/test-files/runtimecode/runtimelibs.jar.bin?raw=true

Step 2: Get the sha512 hash of the jar

 openssl dgst -sha512 runtimelibs.jar

Step 3: Start Solr with runtime lib enabled

 bin/solr start -e cloud -a "-Denable.runtime.lib=true" -noprompt

Step 4: Run a local server. Skip this step if you have another place to host your jars. Ensure that the url is set appropriately

 python -m SimpleHTTPServer 8000 &

Step 5: Add the jar to your collection gettingstarted

 curl http://localhost:8983/solr/gettingstarted/config -H 'Content-type:application/json' -d '{
    "add-runtimelib": { "name" : "testjar",
    "url":"http://localhost:8000/runtimelibs.jar" ,
    "sha512" : "d01b51de67ae1680a84a813983b1de3b592fc32f1a22b662fc9057da5953abd1b72476388ba342cad21671cd0b805503c78ab9075ff2f3951fdf75fa16981420"}
    }'

Step 6: Create a new request handler '/test' for the collection 'gettingstarted' from the jar we just added

curl http://localhost:8983/solr/gettingstarted/config -H 'Content-type:application/json' -d '{
    "create-requesthandler": { "name" : "/test",
    'class': 'org.apache.solr.core.RuntimeLibReqHandler', 'runtimeLib' : true}
    }'

Step 7: Test your request handler

curl  http://localhost:8983/solr/gettingstarted/test

output:

{
  "responseHeader":{
    "status":0,
    "QTime":0},
  "params":{},
  "context":{
    "webapp":"/solr",
    "path":"/test",
    "httpMethod":"GET"},
  "class":"org.apache.solr.core.RuntimeLibReqHandler",
  "loader":"org.apache.solr.core.MemClassLoader"}

 curl http://localhost:8983/solr/gettingstarted/config -H 'Content-type:application/json' -d '{
    "update-runtimelib": { "name" : "testjar",
    "url":"http://localhost:8000/runtimelibs_v2.jar" ,
    "sha512" : "<replace-the-new-sha512-digest-here>"}
    }'

Securing Runtime Libraries

A drawback of this feature is that it could be used to load malicious executable code into the system. However, it is possible to restrict the system to load only trusted jars using PKI to verify that the executables loaded into the system are trustworthy.

The following steps will allow you enable security for this feature. The instructions assume you have started all your Solr nodes with the -Denable.runtime.lib=true.

$ openssl genrsa -out priv_key.pem 512

$ openssl rsa -in priv_key.pem -pubout -outform DER -out pub_key.der

# Connect to ZooKeeper
# Replace the server location below with the correct ZooKeeper connect string for your installation.
$ .bin/zkCli.sh -server localhost:9983

# After connection, you will interact with the ZK prompt.
# Create the directories
[zk: localhost:9983(CONNECTED) 5] create /keys
[zk: localhost:9983(CONNECTED) 5] create /keys/exe

# Now create the public key file in ZooKeeper
# The second path is the path to the .der file on your local machine
[zk: localhost:9983(CONNECTED) 5] create /keys/exe/pub_key.der /myLocal/pathTo/pub_key.der

$ openssl dgst -sha1 -sign priv_key.pem myjar.jar | openssl enc -base64

curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @{filename}
http://localhost:8983/solr/.system/blob/{blobname}

curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json'  -d '{
  "add-runtimelib": {
    "name":"blobname",
    "version":2,
    "sig":"mW1Gwtz2QazjfVdrLFHfbGwcr8xzFYgUOLu68LHqWRDvLG0uLcy1McQ+AzVmeZFBf1yLPDEHBWJb5KXr8bdbHN/
           PYgUB1nsr9pk4EFyD9KfJ8TqeH/ijQ9waa/vjqyiKEI9U550EtSzruLVZ32wJ7smvV0fj2YYhrUaaPzOn9g0=" }
}'

curl http://localhost:8983/api/collections/techproducts/config -H 'Content-type:application/json'  -d '{
  "add-runtimelib": {
    "name":"blobname",
    "version":2,
    "sig":"mW1Gwtz2QazjfVdrLFHfbGwcr8xzFYgUOLu68LHqWRDvLG0uLcy1McQ+AzVmeZFBf1yLPDEHBWJb5KXr8bdbHN/
           PYgUB1nsr9pk4EFyD9KfJ8TqeH/ijQ9waa/vjqyiKEI9U550EtSzruLVZ32wJ7smvV0fj2YYhrUaaPzOn9g0=" }
}'