public abstract class HdfsSplittableDataAccessor extends Plugin implements ReadAccessor
Modifier and Type | Field and Description |
---|---|
protected org.apache.hadoop.conf.Configuration |
conf |
protected java.lang.Object |
data |
protected org.apache.hadoop.mapred.InputFormat<?,?> |
inputFormat |
protected java.util.ListIterator<org.apache.hadoop.mapred.InputSplit> |
iter |
protected org.apache.hadoop.mapred.JobConf |
jobConf |
protected java.lang.Object |
key |
protected org.apache.hadoop.mapred.RecordReader<java.lang.Object,java.lang.Object> |
reader |
Constructor and Description |
---|
HdfsSplittableDataAccessor(InputData input,
org.apache.hadoop.mapred.InputFormat<?,?> inFormat)
Constructs an HdfsSplittableDataAccessor
|
Modifier and Type | Method and Description |
---|---|
void |
closeForRead()
When user finished reading the file, it closes the RecordReader
|
protected boolean |
getNextSplit()
Sets the current split and initializes a RecordReader who feeds from the
split
|
protected abstract java.lang.Object |
getReader(org.apache.hadoop.mapred.JobConf jobConf,
org.apache.hadoop.mapred.InputSplit split)
Specialized accessors will override this method and implement their own
recordReader.
|
boolean |
isThreadSafe()
Checks if the plugin is thread safe or not, based on inputData.
|
boolean |
openForRead()
Fetches the requested fragment (file split) for the current client
request, and sets a record reader for the job.
|
OneRow |
readNextObject()
Fetches one record from the file.
|
protected org.apache.hadoop.conf.Configuration conf
protected org.apache.hadoop.mapred.RecordReader<java.lang.Object,java.lang.Object> reader
protected org.apache.hadoop.mapred.InputFormat<?,?> inputFormat
protected java.util.ListIterator<org.apache.hadoop.mapred.InputSplit> iter
protected org.apache.hadoop.mapred.JobConf jobConf
protected java.lang.Object key
protected java.lang.Object data
public HdfsSplittableDataAccessor(InputData input, org.apache.hadoop.mapred.InputFormat<?,?> inFormat)
input
- all input parameters coming from the client requestinFormat
- the HDFS InputFormat
the caller wants to usepublic boolean openForRead() throws java.lang.Exception
openForRead
in interface ReadAccessor
java.lang.Exception
- if opening the resource failedprotected abstract java.lang.Object getReader(org.apache.hadoop.mapred.JobConf jobConf, org.apache.hadoop.mapred.InputSplit split) throws java.io.IOException
jobConf
- the hadoop jobconf to use for the selected InputFormatsplit
- the input split to be read by the accessorjava.io.IOException
- if recordreader could not be createdprotected boolean getNextSplit() throws java.io.IOException
java.io.IOException
- if record reader could not be createdpublic OneRow readNextObject() throws java.io.IOException
readNextObject
in interface ReadAccessor
java.io.IOException
public void closeForRead() throws java.lang.Exception
closeForRead
in interface ReadAccessor
java.lang.Exception
- if closing the resource failedpublic boolean isThreadSafe()
Plugin
isThreadSafe
in class Plugin