Hadoop MapReduce
By processing delimited file mapreduce hadoop , I am going to show how to process transaction data which is in csv file format. 1) Load the csv file to HDFS using hadoop fs -copyFromLocal [filepath] [destination HDFS path] 2) Create a Transaction.java which contains all Mapper, Reducer and Driver class. Using this any kinds of[…]
When we have large number of small files for example millions of small xml, how to process using hadoop mapreduce by using SequenceFileInputFormat is what I am going show you now the Sequencefile processing mapreduce hadoop 1) Create Driver class SeqDriver.java 2) Mapper class MySeqMapper.java using this code you can process sequence files.
Hadoop provides default input formats like TextInputFormat, NLineInputFormat, KeyValueInputFormat etc., when you get a different types of files for processing you have to create your own custom input format for processing using MapReduce jobs Here I am going to show you how to processing XML files using MapReduce Job by creating custom XMLInputFormat (xmlinputformat hadoop)[…]
Reading SeqeunceFile hadoop mapreduce , You have created a sequence using sequence writer, once done now you want to check whether sequence file in hadoop created successfully or not by reading the sequence file present in HDFS Input to the program is the location of the Sequence File in Hadoop HDFS. You can run this[…]