lingzhi007
- 浏览: 120115 次
- 性别:
- 来自: 杭州
-
文章分类
最新评论
正确方法 ：
0\
拷贝对应目录到 spark2 jars
-rw-r--r--. 1 root root 1396867 3月 2 23:19 hbase-client-1.1.2.2.6.1.0-129.jar
-rw-r--r--. 1 root root 575960 3月 2 23:20 hbase-common-1.1.2.2.6.1.0-129.jar
-rw-r--r--. 1 root root 4956260 3月 2 23:20 hbase-protocol-1.1.2.2.6.1.0-129.jar
-rw-r--r--. 1 root root 4580584 3月 2 23:20 hbase-server-1.1.2.2.6.1.0-129.jar
-rw-r--r--. 1 root root 115935 3月 2 22:05 hive-hbase-handler-1.2.1.jar
1、spark-defaults.conf
+
spark.executor.extraClassPath /usr/hdp/current/hbase-client/lib/hbase-client.jar:/usr/hdp/current/hbase-client/lib/hbase-server.jar:/usr/hdp/current/hbase-client/lib/hbase-protocol.jar:/usr/hdp/current/hbase-client/lib/guava-12.0.1.jar:/usr/hdp/current/hbase-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hbase-client/lib/zookeeper.jar:/usr/hdp/current/hbase-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hbase-client/lib/hbase-hadoop2-compat.jar:/usr/hdp/current/hbase-client/lib/hbase-hadoop-compat.jar:/usr/hdp/current/hbase-client/lib/metrics-core-2.2.0.jar:/usr/hdp/current/hbase-client/conf:/usr/hdp/current/spark2-client/jars/hive-hbase-handler-1.2.1.jar
spark.driver.extraClassPath /usr/hdp/current/hbase-client/lib/hbase-client.jar:/usr/hdp/current/hbase-client/lib/hbase-server.jar:/usr/hdp/current/hbase-client/lib/hbase-protocol.jar:/usr/hdp/current/hbase-client/lib/guava-12.0.1.jar:/usr/hdp/current/hbase-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hbase-client/lib/zookeeper.jar:/usr/hdp/current/hbase-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hbase-client/lib/hbase-hadoop2-compat.jar:/usr/hdp/current/hbase-client/lib/hbase-hadoop-compat.jar:/usr/hdp/current/hbase-client/lib/metrics-core-2.2.0.jar:/usr/hdp/current/hbase-client/conf:/usr/hdp/current/spark2-client/jars/hive-hbase-handler-1.2.1.jar
2\起服务
/usr/hdp/current/spark2-client/sbin/start-thriftserver.sh --hiveconf hive.server2.thrift.port=9994 --hiveconf hive.server2.thrift.bind.host=ark3 --master yarn --deploy-mode client --num-executors 2 --executor-cores 2 --executor-memory 12g
0\
拷贝对应目录到 spark2 jars
-rw-r--r--. 1 root root 1396867 3月 2 23:19 hbase-client-1.1.2.2.6.1.0-129.jar
-rw-r--r--. 1 root root 575960 3月 2 23:20 hbase-common-1.1.2.2.6.1.0-129.jar
-rw-r--r--. 1 root root 4956260 3月 2 23:20 hbase-protocol-1.1.2.2.6.1.0-129.jar
-rw-r--r--. 1 root root 4580584 3月 2 23:20 hbase-server-1.1.2.2.6.1.0-129.jar
-rw-r--r--. 1 root root 115935 3月 2 22:05 hive-hbase-handler-1.2.1.jar
1、spark-defaults.conf
+
spark.executor.extraClassPath /usr/hdp/current/hbase-client/lib/hbase-client.jar:/usr/hdp/current/hbase-client/lib/hbase-server.jar:/usr/hdp/current/hbase-client/lib/hbase-protocol.jar:/usr/hdp/current/hbase-client/lib/guava-12.0.1.jar:/usr/hdp/current/hbase-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hbase-client/lib/zookeeper.jar:/usr/hdp/current/hbase-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hbase-client/lib/hbase-hadoop2-compat.jar:/usr/hdp/current/hbase-client/lib/hbase-hadoop-compat.jar:/usr/hdp/current/hbase-client/lib/metrics-core-2.2.0.jar:/usr/hdp/current/hbase-client/conf:/usr/hdp/current/spark2-client/jars/hive-hbase-handler-1.2.1.jar
spark.driver.extraClassPath /usr/hdp/current/hbase-client/lib/hbase-client.jar:/usr/hdp/current/hbase-client/lib/hbase-server.jar:/usr/hdp/current/hbase-client/lib/hbase-protocol.jar:/usr/hdp/current/hbase-client/lib/guava-12.0.1.jar:/usr/hdp/current/hbase-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hbase-client/lib/zookeeper.jar:/usr/hdp/current/hbase-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hbase-client/lib/hbase-hadoop2-compat.jar:/usr/hdp/current/hbase-client/lib/hbase-hadoop-compat.jar:/usr/hdp/current/hbase-client/lib/metrics-core-2.2.0.jar:/usr/hdp/current/hbase-client/conf:/usr/hdp/current/spark2-client/jars/hive-hbase-handler-1.2.1.jar
2\起服务
/usr/hdp/current/spark2-client/sbin/start-thriftserver.sh --hiveconf hive.server2.thrift.port=9994 --hiveconf hive.server2.thrift.bind.host=ark3 --master yarn --deploy-mode client --num-executors 2 --executor-cores 2 --executor-memory 12g
- hive-hbase-handler-1.2.1.jar (113.2 KB)
- 下载次数: 0
分享到：
发表评论
-
Spark SQL运行 过程 抄的别人的，记录 学习
2018-05-13 23:07 1007抄的别人的，觉得写的特别好 val FILESOURCE ... -
thriftserver log4j.properties 生效
2018-04-09 11:46 420/home/isuhadoop/spark2/sbin/sta ... -
udaf 返回的 子属性
2018-03-20 13:22 414udaf 返回的 子属性 spark.sql(" ... -
spark datasource
2018-03-16 16:36 638DataFrameWriter format val c ... -
如何 map 端 Join。
2018-03-04 19:31 594Hive 中 修改表的 rawDataSize = 1 1 ... -
spark thrift server 修改
2018-03-04 12:58 556org.apache.spark.sql.hive.thrif ... -
scala package
2018-01-25 09:48 488#scala 打包 mvn clean scala:com ... -
SPARK SERVER
2018-01-23 22:15 497sbin/start-thriftserver.sh --dr ... -
driver class
2018-01-21 22:11 497sbin/start-thriftserver.sh -- ... -
spark thrift server 调试
2017-10-20 15:50 832spark-hive-thriftserver 本地调试 ... -
spark SQL conf
2017-10-18 14:36 586org.apache.spark.sql.internal.S ... -
java 死锁 ，内存问题 分析
2017-10-17 10:50 315jstack -l pid /opt/soft/jdk/ ... -
thriftServer proxy
2017-10-16 14:21 900sudo yum install haproxy 257 ... -
hive spark conf
2017-09-26 17:44 1274CREATE TABLE org_userbehavior_a ... -
get day
2017-09-19 08:41 250def timeDayNow() = { var ... -
thriftserver
2017-09-14 19:47 429export SPARK_CONF_DIR=/home/yun ... -
thriftserver dynamicallocation
2017-09-08 14:41 548./sbin/start-thriftserver.sh -- ... -
test code2
2017-09-03 13:45 463package org.test.udf import co ... -
test code
2017-08-24 17:52 260def taskcal(data:Array[(String, ... -
struct streaming SQL udf udaf
2017-08-22 11:50 647spark aggregator class H ...
相关推荐
spark-hive-thriftserver_2.11-2.1.spark-hive-thrift
spark-hive_2.11-2.3.0...spark-hive-thriftserver_2.11-2.3.0.jar log4j-2.15.0.jar slf4j-api-1.7.7.jar slf4j-log4j12-1.7.25.jar curator-client-2.4.0.jar curator-framework-2.4.0.jar curator-recipes-2.4.0.jar
spark和hive结合依赖，如何使用请看我博客https://blog.csdn.net/z1987865446/article/details/109372818
Hadoop Hive HBase Spark Storm概念解释
hbase_hive_thrift_php
小牛学堂-大数据24期-04-Hadoop Hive Hbase Flume Sqoop-12天适合初学者.txt
hadoop hive hbase安装过程
被编译的hive-hbase-handler-1.2.1.jar，用于在Hive中创建关联HBase表的jar，解决创建Hive关联HBase时报FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop....
Hadoop Hive与Hbase整合配置
该文档保护了目前比较流行的大数据平台的原理过程梳理。Hadoop，Hive，Hbase，Spark,MapReduce,Storm
hive和hbase整合的时候，如果出现不兼容的情况需要手动编译：hive-hbase-hander-1.2.2.jar把这个jar替换掉hive/lib里的那个jar包
大数据 HIVE HBASE 区别
Hadoop hbase hive sqoop集群环境安装配置及使用文档
配置，测试，导入数据详细操作，CREATE TABLE hive_hbase_table(key int, value string,name string) hadoop jar /usr/lib/hbase/hbase-0.90.4-cdh3u3.jar importtsv -Dimporttsv.columns=HBASE_ROW_KEY, catgyname...
包含了大量的hadoop hive hbase学习资料
spark学习 hadoop hive hbase flink教程 linux 从入门到精通 一个开源、成体系的大数据学习教程。spark学习 hadoop hive hbase flink教程 linux 从入门到精通 一个开源、成体系的大数据学习教程。spark学习 hadoop ...
此文档是本人在工作中用到的知识总结出来的整合过程，本人是菜鸟，希望得到大神们的建议。
hive0.8.1和hbase0.92.0集成的hive-hbase-handler.Jar包,里面包含：hbase-0.92.0.jar、hbase-0.92.0-tests.jar、hive-hbase-handler-0.9.0-SNAPSHOT.jar。经测试没有问题。