Installing Oozie and running Sample Job

                          Oozie Installation and Sample job walkthrough.


Apache Oozie™ is a workflow scheduler system to manage Apache Hadoop jobs.  Using Oozie, you can set up workflows that execute MapReduce jobs and coordinators that manage workflows.

This post assumes you have MapR 4.1 cluster installed and configured on the system with warden and ZK services stopped to install and configure Oozie .

1) Now setup correct Ecosystem Repo. Edit /etc/yum.repos.d/maprtech.repo and add ecosystem link as seen below.


[maprtech]
name=MapR Technologies
baseurl=http://package.mapr.com/releases/v4.1.0/redhat/
enabled=1
gpgcheck=0
protect=1

[maprecosystem]
name=MapR Technologies
baseurl=http://package.mapr.com/releases/ecosystem-4.x/redhat
enabled=1
gpgcheck=0
protect=1

2) Install Oozie 

 yum install mapr-oozie

3) For non-secure clusters, add the following two properties to the core-site.xml

( For MR1 cluster /opt/mapr/hadoop/hadoop-0.20.2/conf/core-site.xml )


<configuration>
<property>
  <name>hadoop.proxyuser.mapr.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.mapr.groups</name>
  <value>*</value>
</property>
</configuration>

4) Start the cluster

service mapr-zookeeper start      

Now,  service mapr-zookeeper qstatus   ( to check status of ZK )

service mapr-warden start            

5) Export the Oozie URL to your environment with the following command:

export OOZIE_URL='http://<Oozie_node>:11000/oozie'

6) Get into below path and check Oozie’s status with the following command .    ( You should see below relevant messages )

cd /opt/mapr/oozie/oozie-4.1.0/bin/


./oozie admin -status
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/mapr/oozie/oozie-4.1.0/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/mapr/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
System mode: NORMAL

Above indicates normal operation

Now Enabling the Oozie Web UI

The Oozie web UI can display your job status, logs, and other related information. The oozie.war file must include the extjs library to enable the web UI. After installing Oozie, perform the following steps to add the ExtJS library to your oozie.war file

1)  Download the extjs library under /root .

cd /root/
wget dev.sencha.com/deploy/ext-2.2.zip

2) If Oozie is running, shut it down:

maprcli node services -name oozie -action stop -nodes <space delimited list of nodes>

3)  Run the oozie-setup.sh script and specify the path to the extjs file.

cd /opt/mapr/oozie/oozie-4.1.0/
./bin/oozie-setup.sh prepare-war -extjs ~/ext-2.2.zip

4) Start Oozie.

maprcli node services -name oozie -action start -nodes <space delimited list of nodes>

Point your browser to http://<oozie_node>:11000/oozie (To review the web UI for Oozie)

Setup and Run Oozie Example .

After verifying the status of Oozie, set up and we can try the example to get familiar with Oozie.

1)  Extract the oozie examples archive oozie-examples.tar.gz

cd /opt/mapr/oozie/oozie-4.1.0/
gunzip oozie-examples.tar.gz ; tar xvf ./oozie-examples.tar -C /opt/mapr/oozie/oozie-4.1.0/

2) Copy the examples to MapR-FS. 

 hadoop fs -put examples /user/root/examples

3) Copy the input data to MapR-FS.

hadoop fs -put examples/input-data maprfs:///user/root/input-data

4) Change permissions on the examples to make them accessible to all users.

hadoop fs -chmod -R 777 /user/root/examples
hadoop fs -chmod -R 777 /user/root/input-data

5) Run an example with the oozie job command as below . 
First copy the Map-reduce example folder to MapR-FS and then run the Map-Reduce job as below.

hadoop fs -put /opt/mapr/oozie/oozie-4.1.0/examples/apps/map-reduce /user/root/examples/apps/

/opt/mapr/oozie/oozie-4.1.0/bin/oozie job -config /opt/mapr/oozie/oozie-4.1.0/examples/apps/map-reduce/job.properties -run


You can verify the JOB was successful in Oozie UI or via below command

/opt/mapr/oozie/oozie-4.1.0/bin/oozie job -info <job id>