围炉网

一行代码,一篇日志,一个梦想,一个世界

cloudera manager free edition install notes

  1. update /etc/hosts to remove 127.0.1.1
  2. install may take long time
  3. $ sudo -u hdfs hadoop fs -mkdir /tmp
    $ sudo -u hdfs hadoop fs -chmod -R 1777 /tmp
  4. $ sudo -u hdfs hadoop fs -mkdir /tmp/mapred
    $ sudo -u hdfs hadoop fs -chmod -R 1777 /tmp/mapred
  5. $ sudo -u hdfs hadoop fs -chmod -R 1777 /
  6. data node cannot register to namenode
    <property><name>dfs.hosts</name><value></value><final>true</final></property>
  7. java.io.IOException: Cannot create an instance of InputSplit class = org.apache.hadoop.hive.hbase.HBaseSplit:org.apache.hadoop.hive.hbase.HBaseSplit<property>   <name>hive.aux.jars.path</name>   <value>file:///home/hadoop/cdh3/hive-0.7.0-cdh3u0/lib/hive-hbase-handler-0.7.0-cdh3u0.jar,file:///home/hadoop/cdh3/hive-0.7.0-cdh3u0/lib/hbase-0.90.1-cdh3u0.jar,file:///home/hadoop/cdh3/hive-0.7.0-cdh3u0/lib/zookeeper-3.3.1.jar</value> </property> 

    The storage handler is built as an independent module, hive-hbase-handler-x.y.z.jar, which must be available on the Hive client auxpath, along with HBase, Guava and ZooKeeper jars. It also requires the correct configuration property to be set in order to connect to the right HBase master. See the HBase documentation for how to set up an HBase cluster.
    Here's an example using CLI from a source build environment, targeting a single-node HBase server. (Note that the jar locations and names have changed in Hive 0.9.0, so for earlier releases, some changes are needed.)

    $HIVE_SRC/build/dist/bin/hive –auxpath $HIVE_SRC/build/dist/lib/hive-hbase-handler-0.9.0.jar,$HIVE_SRC/build/dist/lib/hbase-0.92.0.jar,$HIVE_SRC/build/dist/lib/zookeeper-3.3.4.jar,$HIVE_SRC/build/dist/lib/guava-r09.jar -hiveconf hbase.master=hbase.yoyodyne.com:60000

    Here's an example which instead targets a distributed HBase cluster where a quorum of 3 zookeepers is used to elect the HBase master:
    $HIVE_SRC/build/dist/bin/hive –auxpath $HIVE_SRC/build/dist/lib/hive-hbase-handler-0.9.0.jar,$HIVE_SRC/build/dist/lib/hbase-0.92.0.jar,$HIVE_SRC/build/dist/lib/zookeeper-3.3.4.jar,$HIVE_SRC/build/dist/lib/guava-r09.jar -hiveconf hbase.zookeeper.quorum=zk1.yoyodyne.com,zk2.yoyodyne.com,zk3.yoyodyne.com

    The handler requires Hadoop 0.20 or higher, and has only been tested with dependency versions hadoop-0.20.x, hbase-0.92.0 and zookeeper-3.3.4. If you are not using hbase-0.92.0, you will need to rebuild the handler with the HBase jar matching your version, and change the –auxpath above accordingly. 

发表评论

您的电子邮箱地址不会被公开。 必填项已用*标注

沪ICP备15009335号-2