NFS4 Setup

NFS4 Setup

MapR lets you mount the cluster via NFS so that your applications can read and write data directly. We use NFS Ganesha for supporting NFSv4 features. NFS Ganesha is an Open Source userspace implementation of the NFS server. The MapR NFSv4, running as a userspace process, registers callbacks with NFS Ganesha through the File System Abstraction Layer (FSAL), which is a shared library (libfsalmapr.so). NFS Ganesha loads and uses this library whenever MFS is exported/mounted. The FSAL, in turn uses FileClient (libMapRClient.so library) to connect to the cluster.

The following diagram illustrates how the MapR processes read and write operations to the MapR cluster using NFSv4. When the user enters a command (such as ls), the NFS client submits the request over TCP to the MapR NFSv4 server. The NFS server uses MapR FileClient to perform the requested operation on the cluster and returns the response to the NFS client over TCP.

Pre-req : 

This blog assumes you have a cluster installed . Run below command to secure the cluster .

[root@node2rhel73 initscripts]# /opt/mapr/server/configure.sh -secure -genkeys -Z 10.10.70.117 -C 10.10.70.117 -N dsemapr

Configuring Hadoop-2.7.0 at /opt/mapr/hadoop/hadoop-2.7.0

Done configuring Hadoop

CLDB node list: 10.10.70.117:7222

Zookeeper node list: 10.10.70.117:5181

Node setup configuration:  cldb fileserver webserver zookeeper

Log can be found at:  /opt/mapr/logs/configure.log

Creating 10 year self signed certificate with subjectDN='CN=node2rhel73'

Zookeeper found on this node, and it is not running. Starting Zookeeper

Warden is not running. Starting mapr-warden. Warden will then start all other configured services on this node

... Starting cldb

... Starting fileserver

... Starting webserver

To further manage the system, use "maprcli", or connect browser to https://node2rhel73:8443/

To stop and start this node, use "systemctl start/stop mapr-warden "

[root@node2rhel73 initscripts]# 

Verify : Zk is up .

[root@node2rhel73 initscripts]# /opt/mapr/initscripts/zookeeper qstatus

JMX enabled by default

Using config: /opt/mapr/zookeeper/zookeeper-3.4.5/conf/zoo.cfg

Mode: standalone

[root@node2rhel73 initscripts]#

Installation :

1) Install nfs-utils on host where NFSv4 is planned to be installed .

yum install nfs-utils -y

2) rpc.statd and rpcbind should be running .

(Formerly nfs-lock) 

[root@node2rhel73 initscripts]# /sbin/rpc.statd

[root@node2rhel73 initscripts]# ps -ef| grep rpc.st

rpcuser  29159     1  2 03:35 ?        00:00:00 /sbin/rpc.statd

root     29164  5553  0 03:35 pts/0    00:00:00 grep --color=auto rpc.st

[root@node2rhel73 initscripts]# 

• On Red Hat and CentOS v6.0 and higher, the rpcbind service (formerly portmapper)  must be running. You can use the command ps ax | grep rpcbind to check.

[root@node2rhel73 conf]# service rpcbind start

Redirecting to /bin/systemctl start  rpcbind.service

[root@node2rhel73 conf]# ps -ef| grep rpcbind

rpc       7980     1  0 20:17 ?        00:00:00 /sbin/rpcbind -w

root      8359  1168  0 20:19 pts/0    00:00:00 grep --color=auto rpcbind

[root@node2rhel73 conf]# 

3) Install "mapr-nfs4server" package , mapr-nfsganesha package will also be installed as a dependency package

yum install mapr-nfs4server -y

[root@node2rhel73 initscripts]# rpm -qa | grep nfs

mapr-nfsganesha-2.3.0.201704102214-1.noarch

nfs-utils-1.3.0-0.33.el7.x86_64

mapr-nfs4server-5.2.0.42783.GA-1.x86_64

libnfsidmap-0.25-15.el7.x86_64

[root@node2rhel73 initscripts]# 

4) Generate service ticket for NFSv4 service to use while starting .

[root@node2rhel73 initscripts]# maprlogin generateticket -type servicewithimpersonation -user mapr -out /tmp/mapr_impersonation_ticket -duration 30:0:0 -renewal 90:0:0

MapR credentials of user 'mapr' for cluster 'dsemapr' are written to '/tmp/mapr_impersonation_ticket'

[root@node2rhel73 initscripts]# cat /tmp/mapr_impersonation_ticket

dsemapr 94UewDZHYdAIewP1EGjlSmfVF0f9YPEUmY5AMnzVJGvncnlZs85cM8JP5GXGCVTh+FMv48MtmLFhxJihnIN20e7xE09/ZZAiZI8YfwJIfB56FSUWAY73iIvO+Uw8i/g+DpO4g7Hj4vVGcOxYW3LXJAOAYP5qJvXgrM/TJh2SqBBfIQQS+M+7fIz+r7RZZkEorpsnj/RPC2okXwSWk6Vwg+c1MAQkKBkBsT6wBTQ6d25yW55qECB6UILFMduJTvHgS0YQNUQIjbM4/ZZE8xuvjGEk3Sn6AvSAz5hfhOllRM7LimE=

[root@node2rhel73 initscripts]#

5)  Update the tkt location .

[root@node2rhel73 initscripts]# grep tkt /opt/mapr/conf/nfs4server.conf 

  tkt_location = /tmp/mapr_impersonation_ticket; 

[root@node2rhel73 initscripts]# 

Update the mount path and pseudo path .

[root@node2rhel73 initscripts]# grep -w "\/mapr" /opt/mapr/conf/nfs4server.conf 

  Path = /mapr;

  Pseudo = /mapr;

[root@node2rhel73 initscripts]# 

Comment out Krb5 since we are not using Kerberos while starting the Nfsv4 .

[root@node2rhel73 initscripts]# grep SecType /opt/mapr/conf/nfs4server.conf 

  # SecType = krb5;

[root@node2rhel73 initscripts]# 

6) Start Nfsv4 server .

[root@node2rhel73 initscripts]# service mapr-nfs4server start

Redirecting to /bin/systemctl start  mapr-nfs4server.service

[root@node2rhel73 initscripts]# service mapr-nfs4server status

Redirecting to /bin/systemctl status  mapr-nfs4server.service

● mapr-nfs4server.service - MapR Technologies, Inc. NFSv4 Server

   Loaded: loaded (/etc/systemd/system/mapr-nfs4server.service; enabled; vendor preset: disabled)

   Active: active (running) since Mon 2017-11-13 03:48:53 EST; 8s ago

  Process: 1622 ExecStart=/opt/mapr/initscripts/mapr-nfs4server start (code=exited, status=0/SUCCESS)

 Main PID: 1682 (nfs4server)

   CGroup: /system.slice/mapr-nfs4server.service

           ├─1682 /opt/mapr/bin/nfs4server -L /opt/mapr/logs/nfs4/nfs4server.log -f /opt/mapr/conf/nfs4server.conf -d

           └─1683 /opt/mapr/bin/ganesha.nfsd -F -L /opt/mapr/logs/nfs4/nfs4server.log -f /opt/mapr/conf/nfs4server.conf

Nov 13 03:48:50 node2rhel73 systemd[1]: Starting MapR Technologies, Inc. NFSv4 Server...

Nov 13 03:48:53 node2rhel73 systemd[1]: Started MapR Technologies, Inc. NFSv4 Server.

[root@node2rhel73 initscripts]# 

7) Show whats exported and mount Nfsv4 to access the cluster.

[root@node2rhel73 initscripts]# showmount -e

Export list for node2rhel73:

/mapr (everyone)

[root@node2rhel73 conf]# /opt/mapr/server/nfs4mgr list-exports

ExportId      Path

30            /mapr

0             /

[root@node2rhel73 conf]# 

[root@node2rhel73 initscripts]# mount -t nfs4 node2rhel73:/mapr /mapr

[root@node2rhel73 initscripts]# df -hP /mapr

Filesystem         Size  Used Avail Use% Mounted on

node2rhel73:/mapr  119G  493M  118G   1% /mapr

[root@node2rhel73 initscripts]# mount | grep mapr

node2rhel73:/mapr on /mapr type nfs4 (rw,relatime,vers=4.0,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,port=0,timeo=600,retrains=2,sec=sys,clientaddr=10.10.70.117,local_lock=none,addr=10.10.70.117)

[root@node2rhel73 initscripts]#

[root@node2rhel73 initscripts]# cd /mapr/dsemapr/

[root@node2rhel73 dsemapr]# ls

a  abizer  apps  hbase  mysql  opt  softlink  test  tmp  user  var

[root@node2rhel73 dsemapr]# hadoop fs -ls /

Found 11 items

-rwxrwxrwx   3 mapr mapr          0 2017-04-27 01:20 /a

drwxrwxrwx   - 26   26            1 2017-08-03 18:30 /abizer

drwxrwxrwx   - 26   26            0 2017-04-20 16:08 /apps

drwxrwxrwx   - 26   26            0 2017-04-20 16:08 /hbase

drwxrwxrwx   - 26   26            0 2017-08-25 23:21 /mysql

drwxrwxrwx   - 26   26            0 2017-04-20 16:09 /opt

-rwxrwxrwx   - 26   26            6 2017-08-07 18:11 /softlink

-rwxrwxrwx   3 26   26            0 2017-04-27 21:15 /test

drwxrwxrwx   - 26   26            2 2017-09-22 01:39 /tmp

drwxrwxrwx   - 26   26            1 2017-08-03 18:43 /user

drwxrwxrwx   - 26   26            1 2017-04-20 16:08 /var

[root@node2rhel73 dsemapr]#