public class ChunkRecordReader extends java.lang.Object implements org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,ChunkWritable>
Constructor and Description |
---|
ChunkRecordReader(org.apache.hadoop.conf.Configuration job,
org.apache.hadoop.mapred.FileSplit split)
Constructs a ChunkRecordReader instance.
|
Modifier and Type | Method and Description |
---|---|
void |
close()
Closes the input stream.
|
org.apache.hadoop.io.LongWritable |
createKey()
Used by the client of this class to create the 'key' output parameter for
next() method.
|
ChunkWritable |
createValue()
Used by the client of this class to create the 'value' output parameter
for next() method.
|
long |
getPos()
Returns the position of the unread tail of the file
|
float |
getProgress()
Gets the progress within the split.
|
org.apache.hadoop.hdfs.DFSInputStream.ReadStatistics |
getReadStatistics()
Returns statistics of the input stream's read operation: total bytes
read, bytes read locally, bytes read in short-circuit (directly from file
descriptor).
|
boolean |
next(org.apache.hadoop.io.LongWritable key,
ChunkWritable value)
Fetches the next data chunk from the file split.
|
public ChunkRecordReader(org.apache.hadoop.conf.Configuration job, org.apache.hadoop.mapred.FileSplit split) throws java.io.IOException
job
- the job configurationsplit
- contains the file name, begin byte of the split and the
bytes lengthjava.io.IOException
- if an I/O error occurs when accessing the file or
creating input stream to read from itpublic org.apache.hadoop.hdfs.DFSInputStream.ReadStatistics getReadStatistics()
public org.apache.hadoop.io.LongWritable createKey()
createKey
in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,ChunkWritable>
public ChunkWritable createValue()
createValue
in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,ChunkWritable>
public boolean next(org.apache.hadoop.io.LongWritable key, ChunkWritable value) throws java.io.IOException
next
in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,ChunkWritable>
key
- - output parameter. When method returns will contain the key -
the number of the start byte of the chunkvalue
- - output parameter. When method returns will contain the
value - the chunk, a byte array inside the ChunkWritable
instancejava.io.IOException
- if an I/O error occurred while reading the next chunk
or linepublic float getProgress() throws java.io.IOException
getProgress
in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,ChunkWritable>
java.io.IOException
public long getPos() throws java.io.IOException
getPos
in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,ChunkWritable>
java.io.IOException
public void close() throws java.io.IOException
close
in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,ChunkWritable>
java.io.IOException