Labels

Friday, May 19, 2017

How to connect to Spark Thrift from Spark 2.1.0

How to connect to Spark Thrift from external BI tools(tableau)for Spark SQL on Spark 2.1.0


Step 1)  Start Spark Thrift Server on port other than 10000(Because hs2 
port is on 10000). In my scenario starting thrift on port 10001 
/opt/mapr/spark/spark-2.1.0/sbin/start-thriftserver.sh --hiveconf  hive.server2.thrift.port=10001 

Step 2) export LD_LIBRARY_PATH=/opt/mapr/hadoop/hadoop-2.7.0/lib/native/ 

Step 3) Using client beeline to connect to spark thrift url

[mapr@vn4 lib]$ /opt/mapr/spark/spark-2.1.0/bin/beeline 

Beeline version 1.6.1-mapr-1611 by Apache Hive 
beeline> !connect jdbc:hive2://localhost:10001 
Connecting to jdbc:hive2://localhost:10001 
Enter username for jdbc:hive2://localhost:10001: mapr 
Enter password for jdbc:hive2://localhost:10001: **** 
17/03/07 16:00:17 INFO Utils: Supplied authorities: localhost:10001 
17/03/07 16:00:17 INFO Utils: Resolved authority: localhost:10001 
17/03/07 16:00:17 INFO HiveConnection: Will try to open client transport 
with JDBC Uri: jdbc:hive2://localhost:10001 
Connected to: Spark SQL (version 1.6.1) 
Driver: Spark Project Core (version 1.6.1-mapr-1611) 
Transaction isolation: TRANSACTION_REPEATABLE_READ 
0: jdbc:hive2://localhost:10001> 

Step 4) Command to stop spark thrift. 
/opt/mapr/spark/spark-2.1.0/sbin/stop-thriftserver.sh

No comments:

Post a Comment