In this tutorial you will learn how to set up storm cluster in production mode.
Prerequisits
1. Java
2. Storm0.8.2
3. zookeeper
4.Atleast 2 system One for master node runs nimbus daemon , zookeeper node and other for
worker node(s) runs supervisor daemon
Here the host name of my master node is 'nimbus'.
you can check your hostname by typing 'hostname' command on your terminal.
Setup master node and zookeeper node
a) Install Java
Note: To install java on linux/ubuntu folllow the given link
b) Install storm-0.8.2
1. Download storm release (in my case storm-0.8.2) from the following link- 2. Extract the downloaded storm release using the following command $ tar -xvzf ~/Downloads/storm-0.8.2/ -C /usr/local 3. Set the path of bin directory of storm release to access the storm command from anywhere,
without typing the full path.
4. Create a local , storm configuration, where we will tell about our nimbus host.
To do it follow these steps- $ mkdir -p ~/.storm $ cat > storm.yaml $ nimbus.host: "192.168.1.99" // ip of your nimbus host C)Install the following native library- 1.To install ZeroMQ type the following commandmake sure that libtool, autoconf, automake, g++ is installed on your machine wget http://download.zeromq.org/zeromq-2.1.7.tar.gz tar -xvzf zeromq-2.1.7.tar.gz cd zeromq-2.1.7 ./configure NOTE: If you see this error message on your terminal configure: error: cannot link with -luuid, install uuid-dev, then install theuuid-dev by2) To install JZMQ,
using the command sudo apt -get install uuid-dev make sudo make installgit clone https://github.com/nathanmarz/jzmq.git cd jzmq ./autogen.sh ./configure make sudo make install3) set the storm .yaml file. #storm.yaml// name of the machine where you install zookeeper in my case this is nimbus storm.zookeeper.servers: - "nimbus" # - "server2" // name of the machine which you make your nimbus or master machine in my case this is also
// "nimbus" nimbus.host: "nimbus"
Install Zookeeper
Type the commond to install zookeeper-
$ sudo apt-get install zookeeper
Setup worker node
a) Install Java Note: To install java on linux/ubuntu folllow the given link
b) Install storm-0.8.2
1. Download storm release (in my case storm-0.8.2) from the following link-
https://github.com/nathanmarz/storm
2. Extract the downloaded storm release using the following command
$ tar -xvzf ~/Downloads/storm-0.8.2/ -C /usr/local
3. Set the path of bin directory of storm relase to access the storm command from anywhere,
without typing the full path.
4. Create a local , storm configuration, where we will tell about our nimbus host.
To do it follow these steps- $ mkdir -p ~/.storm $ cat > storm.yaml $ nimbus.host: "192.168.1.99" // ip of your nimbus host C)Install the following native library- 1. To install ZeroMQ type the following commandmake sure that libtool, autoconf, automake, g++ is installed on your machine wget http://download.zeromq.org/zeromq-2.1.7.tar.gz tar -xvzf zeromq-2.1.7.tar.gz cd zeromq-2.1.7 ./configure NOTE: If you see this error mesage on your terminal configure: error: cannot link with -luuid, install uuid-dev, then install theuuid-dev byusing the command
sudo apt -get install uuid-dev make sudo make install
2. To install JZMQ,git clone https://github.com/nathanmarz/jzmq.git cd jzmq ./autogen.sh ./configure make sudo make install
3. set the storm .yaml file. #storm.yaml// name of the machine where you install zookeeper in my case this is nimbus storm.zookeeper.servers: - "nimbus" # - "server2" // name of the machine which you make your nimbus or master machine in my case this is also
// "nimbus" nimbus.host: "nimbus"
No comments:
Post a Comment