SolrCloud on AWS EC2

1. From the EC2 Dashboard, click Security Groups from the left-hand menu, under "Network & Security".
2. Click Create Security Group under the Security Groups section. Give your security group a descriptive name.
3. You can select one of the existing VPCs or create a new one.
4. We need two ports open for our cloud here:
   1. Solr port. In this example we will use Solr’s default port 8983.
   2. ZooKeeper Port: We’ll use Solr’s embedded ZooKeeper, so we’ll use the default port 9983 (see the Deploying with External ZooKeeper to configure external ZooKeeper).

5. Click Inbound to set inbound network rules, then select Add Rule. Select "Custom TCP" as the type. Enter 8983 for the "Port Range" and choose "My IP for the Source, then enter your public IP. Create a second rule with the same type and source, but enter 9983 for the port.

   This will limit access to your current machine. If you want wider access to the instance in order to collaborate with others, you can specify that, but make sure you only allow as much access as needed. A Solr instance should not be exposed to general Internet traffic.

6. Add another rule for SSH access. Choose "SSH" as the type, and again "My IP" for the source and again enter your public IP. You need SSH access on all instances to install and configure Solr.

7. Review the details, your group configuration should look like this:

   
8. Click Create when finished.
9. We need to modify the rules so that instances that are part of the group can talk to all other instances that are part of the same group. We could not do this while creating the group, so we need to edit the group after creating it to add this.
   1. Select the newly created group in the Security Group overview table. Under the "Inbound" tab, click Edit.
   2. Click Add rule. Choose All TCP from the pulldown list for the type, and enter 0-65535 for the port range. Specify the name of the current Security Group as the solr-sample.

10. Review the details, your group configuration should now look like this:

    
11. Click Save when finished.

Select an existing “private key file” or create a new one and download to your local machine so you will be able to login into the instances via SSH.

Download a stable version of ZooKeeper. In this example we’re using ZooKeeper v3.6.2. On the node you’re using to host ZooKeeper (zookeeper-node), download the package and untar it:

# download stable version of ZooKeeper, here 3.6.2
$ wget https://archive.apache.org/dist/zookeeper/zookeeper-3.6.2/zookeeper-3.6.2.tar.gz
# untar
$ tar -zxvf zookeeper-3.6.2.tar.gz

Add an environment variable for ZooKeeper’s home directory (ZOO_HOME) to the .bashrc for the user who will be running the process. The rest of the instructions assume you have set this variable. Correct the path to the ZooKeeper installation as appropriate if where you put it does not match the below.

$ export ZOO_HOME=$PWD/zookeeper-3.6.2
# put the env variable in .bashrc
# vim ~/.bashrc
export ZOO_HOME=/home/ec2-user/zookeeper-3.6.2

Change directories to ZOO_HOME, and create the ZooKeeper configuration by using the template provided by ZooKeeper.

$ cd $ZOO_HOME
# create ZooKeeper config by using zoo_sample.cfg
$ cp conf/zoo_sample.cfg conf/zoo.cfg

Create the ZooKeeper data directory in the filesystem, and edit the zoo.cfg file to uncomment the autopurge parameters and define the location of the data directory.

# create data dir for ZooKeeper, edit zoo.cfg, uncomment autopurge parameters
$ mkdir data
$ vim conf/zoo.cfg
# -- uncomment --
autopurge.snapRetainCount=3
autopurge.purgeInterval=1
# -- edit --
dataDir=data

Start ZooKeeper.

$ cd $ZOO_HOME
# start ZooKeeper, default port: 2181
$ bin/zkServer.sh start

On the the first node being used for Solr (solr-node-1), start Solr and tell it where to find ZooKeeper.

$ cd $SOLR_HOME
# start Solr node on 8983 and connect to ZooKeeper running on ZooKeeper node
$ bin/solr start -c -p 8983 -h solr-node-1 -z zookeeper-node:2181

On the second Solr node (solr-node-2), again start Solr and tell it where to find ZooKeeper.

$ cd $SOLR_HOME
# start Solr node on 8983 and connect to ZooKeeper running on ZooKeeper node
$ bin/solr start -c -p 8983 -h solr-node-1 -z zookeeper-node:2181