hadoop生态系统学习之路（十）MR将结果输出到hbase-白红宇

hadoop生态系统学习之路（十）MR将结果输出到hbase

阅读量：5924 次

发布时间：2019-06-19

本文共 42450 字，大约阅读时间需要 141 分钟。

之前讲了MR将结果输出到hdfs、hive、db，今天再给大家分享一下，怎样将结果输出到hbase。

首先，提一句，笔者在hadoop集群运行此MR的时候报了一个错误。是一个jar包的缘故，这个错误是hbase版本号的bug，在以下笔者会为大家介绍以及怎样解决问题。

好了，笔者将分以下几个步骤进行介绍：

一、pom依赖


            
      
       0.96.1.1-cdh5.0.0
            
            
              
       
        org.apache.hbase
               
       
        hbase-client
               
       
        ${hbase.version}
               
                   
                        
         
          jdk.tools
                         
         
          jdk.tools
                     
                    
                      
         
          hbase-protocol
                       
         
          org.apache.hbase
                     
                
             
            
                
       
        org.apache.hbase
                 
       
        hbase-common
                 
       
        ${hbase.version}
             
            
                
       
        org.apache.hbase
                 
       
        hbase-server
                 
       
        ${hbase.version}
                 
                   
                      
         
          hbase-protocol
                       
         
          org.apache.hbase
                     
                  
             
            
                
       
        org.apache.hbase
                 
       
        hbase-thrift
                 
       
        ${hbase.version}
                 
                   
                      
         
          hbase-protocol
                       
         
          org.apache.hbase
                     
                  
             
            
                
       
        org.apache.hbase
                 
       
        hbase-testing-util
                 
       
        ${hbase.version}
                 
       
        test
                 
                   
                      
         
          hbase-protocol
                       
         
          org.apache.hbase
                     
                  
             
            
                
       
        org.apache.hbase
                 
       
        hbase-protocol
                 
       
        ${hbase.version}
                 
       
        compile

二、hbase表创建以及hdfs数据准备

首先，我们运行hbase shell进入命令行，然后运行：

create 'user_info', {NAME => 'info'}

创建user_info表，列族是info。

然后hdfs数据还是之前入hive以及db的文件，这里就不贴出来了。

三、MR代码实现

首先。我们来看主类：

/** * Project Name:mr-demo * File Name:LoadDataToHbaseMR.java * Package Name:org.qiyongkang.mr.hbasestore * Date:2016年4月10日下午4:24:11 * Copyright (c) 2016, CANNIKIN(http://http://code.taobao.org/p/cannikin/src/) All Rights Reserved. **/package org.qiyongkang.mr.hbasestore;import java.io.IOException;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.Path;import org.apache.hadoop.hbase.HBaseConfiguration;import org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil;import org.apache.hadoop.io.IntWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Job;import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;/** * ClassName:LoadDataToHbaseMR 
 * Function: TODO ADD FUNCTION. 
 * Reason:   TODO ADD REASON. 
 * Date:     2016年4月10日 下午4:24:11 
 * @author   qiyongkang * @version   * @since    JDK 1.6 * @see       */public class LoadDataToHbaseMR {    public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {        String tablename = "user_info";        Configuration conf = HBaseConfiguration.create();        conf.set("hbase.zookeeper.quorum", "172.31.25.8,172.31.25.2,172.31.25.3");        Job job = Job.getInstance(conf, "hbase store");        job.setJarByClass(LoadDataToHbaseMR.class);        job.setMapperClass(HbaseStoreMapper.class);        job.setReducerClass(HbaseStoreReducer.class);        // 设置输入文件路径        FileInputFormat.addInputPath(job, new Path("/qiyongkang/input"));        job.setNumReduceTasks(1);        TableMapReduceUtil.initTableReducerJob(tablename, HbaseStoreReducer.class, job);        job.setMapOutputKeyClass(Text.class);        job.setMapOutputValueClass(IntWritable.class);        System.exit(job.waitForCompletion(true) ?

0 : 1); } }

然后再来看Mapper类：

/** * Project Name:mr-demo * File Name:HbaseStoreMapper.java * Package Name:org.qiyongkang.mr.hbasestore * Date:2016年4月10日下午4:24:30 * Copyright (c) 2016, CANNIKIN(http://http://code.taobao.org/p/cannikin/src/) All Rights Reserved. **/package org.qiyongkang.mr.hbasestore;import java.io.IOException;import org.apache.hadoop.io.IntWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Mapper;/** * ClassName:HbaseStoreMapper 
 * Function: TODO ADD FUNCTION. 
 * Reason:   TODO ADD REASON. 
 * Date:     2016年4月10日 下午4:24:30 
 * @author   qiyongkang * @version   * @since    JDK 1.6 * @see       */public class HbaseStoreMapper extends Mapper
      
       {
          private IntWritable one = new IntWritable(1);    @Override    protected void map(Object key, Text value, Mapper
       
        .Context context)            throws IOException, InterruptedException {        context.write(value, one);    }}

这个Mapper基本上没干啥事。

再来看看Reducer：

/** * Project Name:mr-demo * File Name:HbaseStoreReducer.java * Package Name:org.qiyongkang.mr.hbasestore * Date:2016年4月10日下午5:01:56 * Copyright (c) 2016, CANNIKIN(http://http://code.taobao.org/p/cannikin/src/) All Rights Reserved. **/package org.qiyongkang.mr.hbasestore;import java.io.IOException;import org.apache.hadoop.hbase.client.Mutation;import org.apache.hadoop.hbase.client.Put;import org.apache.hadoop.hbase.mapreduce.TableReducer;import org.apache.hadoop.hbase.util.Bytes;import org.apache.hadoop.io.IntWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.io.WritableComparable;import org.apache.hadoop.mapreduce.Reducer;/** * ClassName:HbaseStoreReducer 
 * Function: TODO ADD FUNCTION. 
 * Reason:   TODO ADD REASON. 
 * Date:     2016年4月10日 下午5:01:56 
 * @author   qiyongkang * @version   * @since    JDK 1.6 * @see       */public class HbaseStoreReducer extends TableReducer
      
       >{    @Override    protected void reduce(Text key, Iterable
       
         values,            Reducer
        
         , Mutation>.Context context)                    throws IOException, InterruptedException {        // 每行以制表符分隔 id, account, name, age        String[] strs = key.toString().split("\t");        //id为rowKey        Put put = new Put(strs[0].getBytes());        //account        put.add(Bytes.toBytes("info"), Bytes.toBytes("account"), Bytes.toBytes(strs[1]));        //name        put.add(Bytes.toBytes("info"), Bytes.toBytes("name"), Bytes.toBytes(strs[2]));        //age        put.add(Bytes.toBytes("info"), Bytes.toBytes("age"), Bytes.toBytes(strs[3]));        context.write(null, put);    }}

我们在reducer再进行split。然后把结果写入到hbase。

四、运行以及版本号bug解决方式

首先，我们打好包。直接在hadoop集群上运行。会报错例如以下：

16/04/10 18:34:25 INFO zookeeper.ClientCnxn: Opening socket connection to server massdata8/172.31.25.8:2181. Will not attempt to authenticate using SASL (unknown error)java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString

我在网上查了半天，结果发现是hbase版本号的原因。

笔者公司hadoop集群所使用的hbase版本号是hbase-0.96.1.1-cdh5.0.0，使用这个刚好遇到这个bug，这个bug在后面的版本号已经解决。

那么，怎样解决呢？大家能够參考此博客：

以下，我们运行：

HADOOP_CLASSPATH=/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar:/etc/hbase/conf hadoop jar mr-demo-0.0.1-SNAPSHOT-jar-with-dependencies.jar

便可运行成功，主要是这个hbase版本号，会动态载入jar文件。后面的日志我们能够看到。例如以下：

HADOOP_CLASSPATH=/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar:/etc/hbase/conf hadoop jar mr-demo-0.0.1-SNAPSHOT-jar-with-dependencies.jar 16/04/11 12:54:17 INFO client.RMProxy: Connecting to ResourceManager at massdata8/172.31.25.8:803216/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-cdh5.0.0--1, built on 03/28/2014 04:21 GMT16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:host.name=massdata816/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_7516/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/jdk1.7.0_75/jre16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/etc/hadoop/conf:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jaxb-api-2.2.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-beanutils-1.7.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-el-1.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/mockito-all-1.8.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/activation-1.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/snappy-java-1.0.4.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-httpclient-3.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-configuration-1.6.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/hue-plugins-3.5.0-cdh5.0.0-SNAPSHOT.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jetty-6.1.26.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jersey-json-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-collections-3.2.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-math3-3.1.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/xz-1.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/slf4j-log4j12.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jackson-xc-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-digester-1.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/zookeeper-3.4.5-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/xmlenc-0.52.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/httpclient-4.2.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jackson-jaxrs-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jetty-util-6.1.26.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/avro.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jasper-compiler-5.5.23.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jackson-core-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/netty-3.6.2.Final.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jets3t-0.9.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/paranamer-2.3.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/slf4j-api-1.7.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/stax-api-1.0-2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/java-xmlbuilder-0.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jettison-1.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jsch-0.1.42.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/httpcore-4.2.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-beanutils-core-1.8.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-net-3.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/junit-4.8.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-column-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-pig-bundle-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-cascading.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-encoding-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-thrift-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-avro-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-avro.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-annotations-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-thrift.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-thrift-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-generator.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-hadoop-bundle-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-common-2.3.0-cdh5.0.0-tests.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-scrooge.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-encoding-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-common.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-pig-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-cascading-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-format-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-scrooge-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-encoding.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-common-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-generator-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-column.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-test-hadoop2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-nfs.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-nfs-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-pig.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-common-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-annotations.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-common-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-cascading-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-auth-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-common.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-hadoop-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-format-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-pig-bundle.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-format.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-avro-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-hadoop.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-column-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-hadoop-bundle.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-auth.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-generator-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-scrooge-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-hadoop-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-pig-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/./:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-el-1.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jetty-6.1.26.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/xmlenc-0.52.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jetty-util-6.1.26.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/netty-3.6.2.Final.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/.//hadoop-hdfs-2.3.0-cdh5.0.0-tests.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/.//hadoop-hdfs-nfs.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/.//hadoop-hdfs-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/.//hadoop-hdfs-nfs-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/.//hadoop-hdfs.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jaxb-api-2.2.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/activation-1.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/guice-servlet-3.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-httpclient-3.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jetty-6.1.26.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/javax.inject-1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jersey-json-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/aopalliance-1.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/xz-1.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jackson-xc-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jersey-guice-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/zookeeper-3.4.5-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jackson-jaxrs-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jetty-util-6.1.26.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jline-0.9.94.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jackson-core-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/stax-api-1.0-2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/guice-3.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jersey-client-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jettison-1.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-tests.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-api-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-nodemanager-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-common-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-client-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-common.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-client.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-web-proxy-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-common.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-tests-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-applications-distributedshell-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-api.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-resourcemanager-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-common-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/junit-4.10.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/xz-1.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/avro.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/jackson-core-asl-1.8.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/guice-3.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/hamcrest-core-1.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jaxb-api-2.2.2.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-beanutils-1.7.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-extras-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-el-1.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-distcp-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//junit-4.10.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-distcp.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//activation-1.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//snappy-java-1.0.4.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-sls-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-httpclient-3.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jsp-api-2.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-configuration-1.6.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-datajoin-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jetty-6.1.26.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//guava-11.0.2.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-rumen-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jersey-json-1.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-archives.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-streaming.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-collections-3.2.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-math3-3.1.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-archives-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//xz-1.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jackson-xc-1.8.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-gridmix-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//asm-3.2.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jersey-core-1.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-digester-1.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-gridmix.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//zookeeper-3.4.5-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//xmlenc-0.52.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//httpclient-4.2.5.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jackson-jaxrs-1.8.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jetty-util-6.1.26.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//metrics-core-3.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//avro.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jasper-compiler-5.5.23.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-io-2.4.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jackson-core-asl-1.8.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-extras.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jets3t-0.9.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//paranamer-2.3.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//stax-api-1.0-2.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//java-xmlbuilder-0.4.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-rumen.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-sls.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//log4j-1.2.17.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-codec-1.4.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jersey-server-1.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-lang-2.6.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jettison-1.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-auth-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//servlet-api-2.5.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jsch-0.1.42.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-streaming-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hamcrest-core-1.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-datajoin.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-auth.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//httpcore-4.2.5.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-beanutils-core-1.8.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-net-3.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.3.0-cdh5.0.0-tests.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-cli-1.2.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar:/etc/hbase/conf16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/lib/native16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.compiler=
      
       16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd6416/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-279.el6.x86_6416/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:user.name=hdfs16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/hdfs16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/qiyongkang16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=172.31.25.2:2181,172.31.25.8:2181,172.31.25.3:2181 sessionTimeout=60000 watcher=hconnection-0xfcc6c91, quorum=172.31.25.2:2181,172.31.25.8:2181,172.31.25.3:2181, baseZNode=/hbase16/04/11 12:54:17 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0xfcc6c91 connecting to ZooKeeper ensemble=172.31.25.2:2181,172.31.25.8:2181,172.31.25.3:218116/04/11 12:54:17 INFO zookeeper.ClientCnxn: Opening socket connection to server massdata8/172.31.25.8:2181. Will not attempt to authenticate using SASL (unknown error)16/04/11 12:54:17 INFO zookeeper.ClientCnxn: Socket connection established to massdata8/172.31.25.8:2181, initiating session16/04/11 12:54:17 INFO zookeeper.ClientCnxn: Session establishment complete on server massdata8/172.31.25.8:2181, sessionid = 0x353975519d0d25a, negotiated timeout = 6000016/04/11 12:54:18 INFO mapreduce.TableOutputFormat: Created table instance for user_info16/04/11 12:54:18 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.16/04/11 12:54:21 INFO input.FileInputFormat: Total input paths to process : 116/04/11 12:54:22 INFO mapreduce.JobSubmitter: number of splits:116/04/11 12:54:22 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1458262657013_098916/04/11 12:54:22 INFO impl.YarnClientImpl: Submitted application application_1458262657013_098916/04/11 12:54:22 INFO mapreduce.Job: The url to track the job: http://massdata8:8088/proxy/application_1458262657013_0989/16/04/11 12:54:22 INFO mapreduce.Job: Running job: job_1458262657013_098916/04/11 12:54:32 INFO mapreduce.Job: Job job_1458262657013_0989 running in uber mode : false16/04/11 12:54:32 INFO mapreduce.Job:  map 0% reduce 0%16/04/11 12:54:42 INFO mapreduce.Job:  map 100% reduce 0%16/04/11 12:54:51 INFO mapreduce.Job:  map 100% reduce 100%16/04/11 12:54:52 INFO mapreduce.Job: Job job_1458262657013_0989 completed successfully16/04/11 12:54:53 INFO mapreduce.Job: Counters: 49    File System Counters        FILE: Number of bytes read=182        FILE: Number of bytes written=230209        FILE: Number of read operations=0        FILE: Number of large read operations=0        FILE: Number of write operations=0        HDFS: Number of bytes read=259        HDFS: Number of bytes written=0        HDFS: Number of read operations=2        HDFS: Number of large read operations=0        HDFS: Number of write operations=0    Job Counters         Launched map tasks=1        Launched reduce tasks=1        Data-local map tasks=1        Total time spent by all maps in occupied slots (ms)=7763        Total time spent by all reduces in occupied slots (ms)=8240        Total time spent by all map tasks (ms)=7763        Total time spent by all reduce tasks (ms)=8240        Total vcore-seconds taken by all map tasks=7763        Total vcore-seconds taken by all reduce tasks=8240        Total megabyte-seconds taken by all map tasks=7949312        Total megabyte-seconds taken by all reduce tasks=8437760    Map-Reduce Framework        Map input records=8        Map output records=8        Map output bytes=168        Map output materialized bytes=178        Input split bytes=117        Combine input records=0        Combine output records=0        Reduce input groups=8        Reduce shuffle bytes=178        Reduce input records=8        Reduce output records=8        Spilled Records=16        Shuffled Maps =1        Failed Shuffles=0        Merged Map outputs=1        GC time elapsed (ms)=70        CPU time spent (ms)=2560        Physical memory (bytes) snapshot=960487424        Virtual memory (bytes) snapshot=3228942336        Total committed heap usage (bytes)=1584398336    Shuffle Errors        BAD_ID=0        CONNECTION=0        IO_ERROR=0        WRONG_LENGTH=0        WRONG_MAP=0        WRONG_REDUCE=0    File Input Format Counters         Bytes Read=142    File Output Format Counters         Bytes Written=0

这样，便运行成功啦。

五、查看结果

以下，我们在hbase运行scan ‘user_info’，能够看到：

说明结果入hbase成功！

ok，就说到这儿了。事实上也能够读取hbase的表数据，大家也能够去研究一下。

你可能感兴趣的文章