Pages

Storm installation in production mode



 In this tutorial you will learn how to set up storm cluster in production mode.
  
 
storm cluster
 
 
Prerequisits
 1. Java 
 2. Storm0.8.2
 3. zookeeper 
 4.Atleast 2 system One for master node runs nimbus daemon , zookeeper node and other for 
    worker node(s) runs supervisor daemon
   Here the host name of my master node is 'nimbus'.
   you can check your hostname by typing 'hostname' command on your terminal.
  
Setup master node and zookeeper node
 
     a) Install Java 
         Note: To install java on linux/ubuntu folllow the given link


    b) Install storm-0.8.2
 
        1. Download storm release (in my case storm-0.8.2) from the following link-


       2. Extract the downloaded storm release using the following command

              $ tar -xvzf ~/Downloads/storm-0.8.2/ -C /usr/local

       3. Set the path of bin directory of storm release to access the storm command from anywhere,
        without typing the full path.

       4. Create  a local , storm configuration, where we will tell about our nimbus host. 
        To do it follow these steps-

            $ mkdir -p ~/.storm
            $ cat > storm.yaml
            $ nimbus.host: "192.168.1.99" // ip of your nimbus host

   C)Install the following native library-

      1.To install ZeroMQ type the following command
make sure that libtool, autoconf, automake, g++ is installed on your machine wget http://download.zeromq.org/zeromq-2.1.7.tar.gz tar -xvzf zeromq-2.1.7.tar.gz cd zeromq-2.1.7 ./configure NOTE: If you see this error message on your terminal configure: error: cannot link with -luuid, install uuid-dev, then install theuuid-dev by
using the command sudo apt -get install uuid-dev make sudo make install
2) To install JZMQ,
git clone https://github.com/nathanmarz/jzmq.git cd jzmq ./autogen.sh ./configure make sudo make install
3) set the storm .yaml file. #storm.yaml
// name of the machine where you install zookeeper in my case this is nimbus storm.zookeeper.servers: - "nimbus" # - "server2" // name of the machine which you make your nimbus or master machine in my case this is also
// "nimbus" nimbus.host: "nimbus" 
Install Zookeeper
 Type the commond to install zookeeper-

      $ sudo apt-get install zookeeper 
 
Setup worker node
 
      a) Install Java 

     Note: To install java on linux/ubuntu folllow the given link



     b) Install storm-0.8.2

         1. Download storm release (in my case storm-0.8.2) from the following link-

                 https://github.com/nathanmarz/storm

        2. Extract the downloaded storm release using the following command

              $ tar -xvzf ~/Downloads/storm-0.8.2/ -C /usr/local

        3. Set the path of bin directory of storm relase to access the storm command from anywhere,
            without typing the full path.

        4. Create  a local , storm configuration, where we will tell about our nimbus host. 
             To do it follow these steps-

                $ mkdir -p ~/.storm
                $ cat > storm.yaml
                $ nimbus.host: "192.168.1.99" // ip of your nimbus host

 
    C)Install the following native library-

        1. To install ZeroMQ type the following command
 
make sure that libtool, autoconf, automake, g++ is installed on your machine wget http://download.zeromq.org/zeromq-2.1.7.tar.gz tar -xvzf zeromq-2.1.7.tar.gz cd zeromq-2.1.7 ./configure NOTE: If you see this error mesage on your terminal configure: error: cannot link with -luuid, install uuid-dev, then install theuuid-dev byusing the command
       sudo apt -get install uuid-dev  
       make
       sudo make install 
      2. To install JZMQ,

git clone https://github.com/nathanmarz/jzmq.git cd jzmq ./autogen.sh ./configure make sudo make install
     3. set the storm .yaml file.

       #storm.yaml
// name of the machine where you install zookeeper in my case this is nimbus storm.zookeeper.servers: - "nimbus" # - "server2" // name of the machine which you make your nimbus or master machine in my case this is also
// "nimbus" nimbus.host: "nimbus" 

No comments:

Post a Comment