Deploying PACC container ( Pre-builting and Porting)

                                       Deploying PACC container ( Pre-builting and Porting)


This Blog can be treated as extension of earlier PACC blog for systems which cannot connect to the internet due to which assumption of "mapr-setup.sh" script fail about connecting to internet and downloading packages on the go  . In this one we will see how we can use "mapr-setup.sh" to container images and then copy the containers to nodes which have no internet access alternatively it can also be hosted on internal registry where host might have access.
http://abizeradenwala.blogspot.com/2017/05/pacc-docker.html

1) This Blog assumes you have a system where docker is running and All the steps till Step 5 have been followed properly to get below log line

Successfully built 7351f3e4dc0d

Edit '/tmp/docker_images/client/mapr-docker-client.sh' to set MAPR_CLUSTER and MAPR_CLDB_HOSTS and then execute it to start the container 

2) We can see Image 5.2.1_3.0_centos7_yarn_fuse this exists which we plan to port to another system.

[root@noderhel73 tmp]# docker images

REPOSITORY          TAG                           IMAGE ID            CREATED             SIZE

maprtech/pacc       5.2.1_3.0_centos7_yarn_fuse   7351f3e4dc0d        15 minutes ago      965MB

centos              centos7                       36540f359ca3        4 weeks ago         193MB

maprtech/pacc       5.2.1_3.0_centos7             71f7895f4618        3 months ago        583MB

maprtech/pacc       latest                        71f7895f4618        3 months ago        583MB

3) Save the docker Image into a file which i can copy to any host.

[root@noderhel73 tmp]# docker save maprtech/pacc:5.2.1_3.0_centos7_yarn_fuse -o /tmp/5.2.1_3.0_centos7_yarn_fuse

[root@noderhel73 tmp]# du -sh /tmp/5.2.1_3.0_centos7_yarn_fuse

932M /tmp/5.2.1_3.0_centos7_yarn_fuse

4) Now remove the image from the hosts, Since we already have downloaded Image file we need to port.

[root@noderhel73 tmp]# docker rmi maprtech/pacc:5.2.1_3.0_centos7_yarn_fuse

Untagged: maprtech/pacc:5.2.1_3.0_centos7_yarn_fuse

Deleted: sha256:7351f3e4dc0de79cb28594bcb45739e50a08da7fd39e5f402875b5bdc27ffa36

Deleted: sha256:8da89880e023bdf8a91311dfab13b846965c02f6822170ffadc5dc2b74ac9bc5

Deleted: sha256:cb8a082d2f4038bbccce84aeefa2ceb8a8eaed7e16cd9b843c5fa5e2d75389e3

Deleted: sha256:41fa2482071eca11206ddfdafb27bd1d225f724d59806a18908f578ad8ba775f

Deleted: sha256:7445f128a5ea1118702ffd4519fe3d9bd03ec34220da538cb79c7f78d7292e91

Deleted: sha256:9393a64c0356642749e42364485d42ca26590a76ac3fbf020e0234cc51d7dc77

Deleted: sha256:7d3938dcc12005912615189b2bc62644532789d615fd7e81bf449e6e630d2159

Deleted: sha256:3bf0c11add438bd161fb8038b12f250232ec6766a43378e3ede9a59b89480eb0

Deleted: sha256:bcc298a830c7a7ff2ac7b24fb628f95d22d220edb94f1868e854f104a35c92be

Deleted: sha256:5070d6a1c579860ae93a4818c09b8262926d0cbe4da7b4e355b2ece2d4c117e9

[root@noderhel73 tmp]# docker images

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE

centos              centos7             36540f359ca3        4 weeks ago         193MB

maprtech/pacc       5.2.1_3.0_centos7   71f7895f4618        3 months ago        583MB

maprtech/pacc       latest              71f7895f4618        3 months ago        583MB

5) You can copy "5.2.1_3.0_centos7_yarn_fuse" file which was created in Step 3 to Host which has no internet access under /tmp and run below command to load the image on that node.

[root@noderhel73 tmp]# docker load -i /tmp/5.2.1_3.0_centos7_yarn_fuse

c3166ac2e651: Loading layer [==================================================>]  236.8MB/236.8MB

8462671e0b31: Loading layer [==================================================>]  89.09kB/89.09kB

b3d3c7d1bf0b: Loading layer [==================================================>]  539.6MB/539.6MB

Loaded image: maprtech/pacc:5.2.1_3.0_centos7_yarn_fuse

Verify Image is loaded.

[root@noderhel73 tmp]# docker images

REPOSITORY          TAG                           IMAGE ID            CREATED             SIZE

maprtech/pacc       5.2.1_3.0_centos7_yarn_fuse   7351f3e4dc0d        21 minutes ago      965MB

centos              centos7                       36540f359ca3        4 weeks ago         193MB

maprtech/pacc       5.2.1_3.0_centos7             71f7895f4618        3 months ago        583MB

maprtech/pacc       latest                        71f7895f4618        3 months ago        583MB

[root@noderhel73 tmp]# 

6) Copy /tmp/docker_images/client/mapr-docker-client.sh script to the node where you would like to spin up PACC container .  This script is nothing but takes all the input for different variables and runs "docker run" command to spin up a container.

[root@noderhel73 tmp]# mkdir abizertest

[root@noderhel73 tmp]# cp /tmp/docker_images/client/mapr-docker-client.sh abizertest/

[root@noderhel73 tmp]# cd abizertest/

7) Modify mapr-docker-client.sh script to list "Cluster Name","CLDB hostname" and PosixMount path. 

[root@noderhel73 abizertest]# vi mapr-docker-client.sh 

#!/bin/sh

MAPR_CLUSTER=ClusterNFS4

MAPR_CLDB_HOSTS=10.10.70.117

# MapR POSIX client mount path to enable direct MapR-FS access

MAPR_MOUNT_PATH=/mapr

8) Run the script , It will connect you to docker container with access to cluster via Hadoop distributed commands or Posix mount as seen below .

[root@noderhel73 abizertest]# sh mapr-docker-client.sh 

Testing for cluster user account... 

Enter MapR cluster user name: root

 ...Success 

Configuring MapR client ( -c -C 10.10.70.117 -N ClusterNFS4)... 

create /opt/mapr/conf/conf.old

Configuring Hadoop-2.7.0 at /opt/mapr/hadoop/hadoop-2.7.0

Done configuring Hadoop

CLDB node list: 10.10.70.117:7222

Zookeeper node list: 

...Success 

Starting services (mapr-posix-client-container)... 

Started service mapr-posix-client-container 

...Success 

Posix Mounted :

[root@d388255453b8 /]# df -hP /mapr

Filesystem              Size  Used Avail Use% Mounted on

posix-client-container  119G  504M  118G   1% /mapr

[root@d388255453b8 /]# ls /mapr/ClusterNFS4/

a  apps  hbase  opt  test  tmp  user  var

Hadoop Command :

[root@d388255453b8 /]# hadoop fs -ls /

Found 8 items

-rw-r--r--   3 root       root                0 2017-04-27 01:20 /a

drwxr-xr-x   - 2147483632 2147483632          0 2017-04-20 16:08 /apps

drwxr-xr-x   - 2147483632 2147483632          0 2017-04-20 16:08 /hbase

drwxr-xr-x   - 2147483632 2147483632          0 2017-04-20 16:09 /opt

-rw-r--r--   3 root       root                0 2017-04-27 21:15 /test

drwxrwxrwx   - 2147483632 2147483632          0 2017-04-20 16:07 /tmp

drwxr-xr-x   - 2147483632 2147483632          0 2017-04-20 16:09 /user

drwxr-xr-x   - 2147483632 2147483632          1 2017-04-20 16:08 /var