MapR-DB Table Replication on Secure cluster( Locally )

                                MapR-DB Table Replication on Secure cluster( Locally )


This is another flavor of my prior blog where i need to have table replication locally.
http://abizeradenwala.blogspot.com/2016/03/mapr-db-table-replication.html

1) Make sure cluster knows about the gateway.

maprcli cluster gateway set -dstcluster <des_cluster_name> -gateways <dest_gateway_hostname> 

--Confirm cluster can resolve the G/W

[root@node10 ~]# maprcli cluster gateway resolve -dstcluster Destination

Source              GatewayHosts       

mapr-clusters.conf  10.10.70.110:7660

2) Create and load the source table.

[root@node10 ~]# echo "create '/user/mapr/AbiLocalt1', {NAME => 'c1', VERSIONS => 1}" | hbase shell

HBase Shell; enter 'help<RETURN>' for list of supported commands.

Type "exit<RETURN>" to leave the HBase Shell

Version 0.98.12-mapr-1605, r1b36a7c15cda3f507e22826b9062ccb778af694c, Fri Jun  3 04:45:12 UTC 2016

Not all HBase shell commands are applicable to MapR tables.

Consult MapR documentation for the list of supported commands.

create '/user/mapr/AbiLocalt1', {NAME => 'c1', VERSIONS => 1}

0 row(s) in 0.9310 seconds

Hbase::Table - /user/mapr/AbiLocalt1

[root@node10 ~]# echo "put '/user/mapr/AbiLocalt1', 'r1', 'c1', '$(date)'" | hbase shell 

HBase Shell; enter 'help<RETURN>' for list of supported commands.

Type "exit<RETURN>" to leave the HBase Shell

Version 0.98.12-mapr-1605, r1b36a7c15cda3f507e22826b9062ccb778af694c, Fri Jun  3 04:45:12 UTC 2016

Not all HBase shell commands are applicable to MapR tables.

Consult MapR documentation for the list of supported commands.

put '/user/mapr/AbiLocalt1', 'r1', 'c1', 'Thu Mar 23 13:01:51 PDT 2017'

0 row(s) in 0.0850 seconds

3) Now setup replica for the source table.

maprcli table replica autosetup -path /user/mapr/AbiLocalt1 -replica /user/mapr/AbiLocalreplt1

4) Now scan the destination table to check if table data is replicated.

[root@node10 ~]# echo "scan '/user/mapr/AbiLocalreplt1'" | hbase shell 

HBase Shell; enter 'help<RETURN>' for list of supported commands.

Type "exit<RETURN>" to leave the HBase Shell

Version 0.98.12-mapr-1605, r1b36a7c15cda3f507e22826b9062ccb778af694c, Fri Jun  3 04:45:12 UTC 2016

Not all HBase shell commands are applicable to MapR tables.

Consult MapR documentation for the list of supported commands.

scan '/user/mapr/AbiLocalreplt1'

ROW                                           COLUMN+CELL                                                                                                                          

 r1                                           column=c1:, timestamp=1490299319331, value=Thu Mar 23 13:01:51 PDT 2017                                                              

1 row(s) in 0.1080 seconds

5) Finally to test the replication I will push 2 rows and confirm replica stream writes to destination table.

[root@node10 ~]# echo "put '/user/mapr/AbiLocalt1', 'r1', 'c1', 'abi'" | hbase shell 

HBase Shell; enter 'help<RETURN>' for list of supported commands.

Type "exit<RETURN>" to leave the HBase Shell

Version 0.98.12-mapr-1605, r1b36a7c15cda3f507e22826b9062ccb778af694c, Fri Jun  3 04:45:12 UTC 2016

Not all HBase shell commands are applicable to MapR tables.

Consult MapR documentation for the list of supported commands.

put '/user/mapr/AbiLocalt1', 'r1', 'c1', 'abi'

0 row(s) in 0.0860 seconds

[root@node10 ~]# echo "put '/user/mapr/AbiLocalt1', 'r2', 'c1', 'abizer'" | hbase shell 

HBase Shell; enter 'help<RETURN>' for list of supported commands.

Type "exit<RETURN>" to leave the HBase Shell

Version 0.98.12-mapr-1605, r1b36a7c15cda3f507e22826b9062ccb778af694c, Fri Jun  3 04:45:12 UTC 2016

Not all HBase shell commands are applicable to MapR tables.

Consult MapR documentation for the list of supported commands.

put '/user/mapr/AbiLocalt1', 'r2', 'c1', 'abizer'

0 row(s) in 0.1070 seconds

Source table :- 

[root@node10 ~]# echo "scan '/user/mapr/AbiLocalt1'" | hbase shell 

HBase Shell; enter 'help<RETURN>' for list of supported commands.

Type "exit<RETURN>" to leave the HBase Shell

Version 0.98.12-mapr-1605, r1b36a7c15cda3f507e22826b9062ccb778af694c, Fri Jun  3 04:45:12 UTC 2016

Not all HBase shell commands are applicable to MapR tables.

Consult MapR documentation for the list of supported commands.

scan '/user/mapr/AbiLocalt1'

ROW                                           COLUMN+CELL                                                                                                                          

 r1                                           column=c1:, timestamp=1490299674775, value=abi                                                                                       

 r2                                           column=c1:, timestamp=1490299746021, value=abizer                                                                                    

2 row(s) in 0.1110 seconds

Destination table :- 

[root@node10 ~]# echo "scan '/user/mapr/AbiLocalreplt1'" | hbase shell 

HBase Shell; enter 'help<RETURN>' for list of supported commands.

Type "exit<RETURN>" to leave the HBase Shell

Version 0.98.12-mapr-1605, r1b36a7c15cda3f507e22826b9062ccb778af694c, Fri Jun  3 04:45:12 UTC 2016

Not all HBase shell commands are applicable to MapR tables.

Consult MapR documentation for the list of supported commands.

scan '/user/mapr/AbiLocalreplt1'

ROW                                           COLUMN+CELL                                                                                                                          

 r1                                           column=c1:, timestamp=1490299674775, value=abi                                                                                       

 r2                                           column=c1:, timestamp=1490299746021, value=abizer                                                                                    

2 row(s) in 0.1080 seconds

Check Gateway logs to make sure replication stream is pushing the data :-

2017-03-23 13:09:06,0250 INFO Replicator fs/gateway/src/cc/replicator.cc:552 Thread: 0 FamilyMap Name c1 famId src: 1 dst: 1

2017-03-23 13:09:08,1536 INFO Replicator fs/gateway/src/cc/replicator.cc:1799 Thread: 0 >FlushStream id: 8636429065916677938 bucket 2063.39.131436 maxSrcBucketOff 0 flushOff 53 numRows 1 atime 1490299748153 KeyRange[r2, r2]

2017-03-23 13:09:08,1564 INFO Replicator fs/gateway/src/cc/replicator.cc:1813 Thread: 0 <FlushStream id: 8636429065916677938 bucket 2063.39.131436 startOff 0 maxOff 53 TimeRange[1490299746021, 1490299746021] table /user/mapr/AbiLocalreplt1