Important Maprcli Commands

                               Important Maprcli Commands

Turn on Debug for maprcli command

maprcli service list -node `hostname` -cli.loglevel DEBUG

List all Alarms

/opt/mapr/bin/maprcli alarm list 

List all cldb's

/opt/mapr/bin/maprcli node listcldbs

List all Zk's

/opt/mapr/bin/maprcli node listzookeepers

Info for  certain details of the cluster nodes in column

/opt/mapr/bin/maprcli node list -columns  service,ip,disks,health,faileddisks,mtotal,cpus  

Total node info in cluster

/opt/mapr/bin/maprcli node list -json      

Find topology from command line 

/opt/mapr/bin/maprcli node topo -json

List's volume with detail's  

/opt/mapr/bin/maprcli volume list -json         

Lists all disk's from local node 

/opt/mapr/bin/maprcli disk list -host 127.0.0.1        

Displays all configuration cluster wide 

/opt/mapr/bin/maprcli config load -json             

Dumps SP’s usage and shows active container moves

/opt/mapr/bin/maprcli dump balancerinfo -json     

Dumps active Master balancing across the nodes

/opt/mapr/bin/maprcli dump rolebalancerinfo –json

Dumps Info about replication

/opt/mapr/bin/maprcli dump rereplicationinfo –json

Lists CLDB master

/opt/mapr/bin/maprcli node cldbmaster

Dump info stored in zookeeper

/opt/mapr/bin/maprcli dump zkinfo -zkconnect pt-mapr-001.shortcovers.local:5181,pt-mapr-002.shortcovers.local:5181,pt-mapr-003.shortcovers.local:5181 –json

Dump volume info with details for containers

/opt/mapr/bin/maprcli dump volumeinfo -volumename <VolumeName> –json

List's Virtual IP's

/usr/bin/maprcli virtualip list

Container Replication info

maprcli dump replicationmanagerinfo/rereplicationinfo/

replicationmanagerqueueinfo  -volumename  <volumename>

Switch Container Master from current master node

maprcli debugdb switchMaster -cid <CID> (except for CID 1)

Trace level information 

maprcli trace info

Container INFO

maprcli dump containerinfo -ids 2049

Allow node in cluster

maprcli node allow-into-cluster  -hostids hostids

Stop/Start/Restart Services

maprcli node services -tasktracker restart -nodes <node 1>

Removing Volume

maprcli volume remove -name  volumeName

Removing Volume link 

maprcli volume link  remove –path  <actual-path>

The Volume topology for given volume

maprcli volume list -columns mountdir,rackpath | grep <VolumeName>

URL for service

maprcli urls –name <jobtracker>

The node topology of all the nodes in the cluster

maprcli node list -columns hostname,id,racktopo

Setup ALARM 

maprcli alarm config save -values "<alarm name>,1,<email address>"

TT Map and Reduce Slots 

maprcli node list -columns h,ttmapSlots,ttReduceSlots

Max number of containers threshold set 

maprcli node list -columns mct,nmc

List offline containers

maprcli dump containers -type offline -json               

Change limit for container threshold

maprcli node modify -nodes <hostname> -maxContainers 60000  (Setting the maxContainers to 0 will not raise the alarm)

VIP Assignment

1. Move a VIP to another NIC.

   maprcli virtualip move -virtualip <vip> -tomac <mac-address>

2. At the time of adding the vips set the preferred macs for them.

   maprcli virtualip add -virtualip <vip> [-virtualipend <vip>]

      [-preferredmac <mapc-address]

3. Update the preferred macs for set of vips.

   maprcli virtualip edit -virtualip <vip> [-virtualipend <vip>]

      [-preferredmac <mapc-address]

4. List the vips with preferred mac setting (if any).

   maprcli virtualip list

  

Modify replication factor for volume

maprcli volume modify -name mapr.cldb.internal -minreplication 1

Install MapR Cluster

                                     Install MapR Cluster

Quick and dirty way to install MapR cluster within few mins with just few commands if you don't want to script this process or use GUI installer.

In this post its assumed you have a system with Red-Hat/CentOS 6.x already then below steps required to get MapR Hadoop Cluster up and running.


  1)  yum install wget -y
  2)  wget http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
  3)  rpm -Uvh epel-release-6*.rpm
  4)  rpm --import http://package.mapr.com/releases/pub/maprgpg.key
  5)  groupadd -g 5000 mapr ; useradd -g 5000 -u 5000 mapr
  6)  vi /etc/yum.repos.d/maprtech.repo           ----  Add below address

[maprtech]

name=MapR Technologies

baseurl=http://package.mapr.com/releases/v5.2.2/redhat/

enabled=1

gpgcheck=0

protect=1

[centos]

name=centos, Inc.

baseurl=http://mirror.centos.org/centos/6.9/os/x86_64/

enabled=1

gpgcheck=0

  7)  Now creating list of disk to be given to MFS, First disk is OS disk hence came up with below script to get list of disks.
( Below is example to get disk list on aws instance excluding OS disk )

fdisk -l  | grep dev | awk '{print $2}' | cut -d: -f1 | tail -n +2 >> /tmp/disks

   8)  Install all required services with Java.
yum install mapr-cldb mapr-core mapr-fileserver mapr-zookeeper mapr-zk-internal mapr-webserver java-devel -y
 
   9)  Add "/usr/lib/jvm/jre-1.6.0-openjdk.x86_64" to below env file as "JAVA_HOME"

 vi /opt/mapr/conf/env.sh

   10)  /opt/mapr/server/configure.sh -C Nodes -Z Nodes -N Cluster-Name
   11)  /opt/mapr/server/disksetup -F /tmp/disks
   12)  Start ZK service and check qstatus if it has leader.

 service mapr-zookeeper start
 service mapr-zookeeper qstatus

   13) Start warden

service mapr-warden start

14)  Check MCS to Apply License and monitor the cluster.

connect browser to https://{webserver host name}:8443

hadoop mfs commands

                    Useful hadoop mfs Command

1.     Display the usage statement for the hadoop mfs command

$ hadoop mfs

2.     Command to create symlinks in hadoop cluster

$ hadoop mfs -ln <target> <symlink>

3.     Determine which servers are storing chunks for those files

$ hadoop mfs –ls ~/mydir

4.     Determine if your subdirectory has compression turned on

$ hadoop mfs –lsd ~/mydir

5.     Lists files in the directory specified by <path>, with an additional column that displays the number of disk blocks per file. Disk blocks are 8192 bytes (8k)

$  hadoop mfs -lss /user

6.     Set the chunk size to 64K for your subdirectory

$ hadoop mfs -setchunksize 65536 ~/mydir

7.     Lists all paths recursively without crossing volume links

$ hadoop mfs -lsrv /user

8.     Lists the file name for the FID which is feeded to the command.

$ hadoop mfs -lsfid <cid.cinum.uniq>

9.     Turn off compression

$ hadoop mfs -setcompression off ~/mydir

Hadoop fs Commands

                                             Important hadoop fs commands

To  display the usage statement for the hadoop command run
$ hadoop -help

1.     Display the usage statement for the hadoop fs command
    $ hadoop fs

2.     List the contents of your home directory
    $ hadoop fs –ls ~

3.     Display a recursive listing of the /mapr directory
    $ hadoop fs –lsr /mapr

4.     Display the contents of the /tmp directory on hadoop cluster
    $ hadoop fs –ls /tmp

5.    Display the disk-usage of the /user/root directory
    $ hadoop fs -du /user/root

6.     Creates "test" file under "/user/root" directory on hadoop cluster
    $hadoop fs -touchz /user/root/test

7.     Make a sub-directory in your home directory.
    $ hadoop fs –mkdir ~/mydir

8.     Copy local /etc/hosts file to your hadoop cluster
    $ hadoop fs –copyFromLocal /etc/hosts ~/mydir

9.     Display the contents of the hosts file in directory u created.
  $ hadoop fs –ls ~/mydir

10. Change the permission of the hosts file .
   $ hadoop fs -chmod 777 ~/mydir/hosts

11. Move the hosts file from hadoop to local /tmp
    $ hadoop fs –moveToLocal ~/mydir/hosts /tmp/

12. Copy the hosts file from  hadoop to local /tmp
  $ hadoop fs –copyToLocal ~/mydir/hosts /tmp/

13. Display the contents of the hosts file in /tmp
     $ hadoop fs –cat /tmp/hosts

14. Populate your subdirectory with files and time the command    
    $ time hadoop fs -copyFromLocal /usr/bin/* ~/mydir

15. Remove all the files in your subdirectory    
    $ hadoop fs -rm ~/mydir/*