org.apache.hawq.pxf.plugins.hdfs

Class HdfsAtomicDataAccessor

java.lang.Object

org.apache.hawq.pxf.api.utilities.Plugin

org.apache.hawq.pxf.plugins.hdfs.HdfsAtomicDataAccessor

All Implemented Interfaces:

ReadAccessor

Direct Known Subclasses:

QuotedLineBreakAccessor

public abstract class HdfsAtomicDataAccessor
extends Plugin
implements ReadAccessor

Base class for enforcing the complete access of a file in one accessor. Since we are not accessing the file using the splittable API, but instead are using the "simple" stream API, it means that we cannot fetch different parts (splits) of the file in different segments. Instead each file access brings the complete file. And, if several segments would access the same file, then each one will return the whole file and we will observe in the query result, each record appearing number_of_segments times. To avoid this we will only have one segment (segment 0) working for this case - enforced with isWorkingSegment() method. Naturally this is the less recommended working mode since we are not making use of segment parallelism. HDFS accessors for a specific file type should inherit from this class only if the file they are reading does not support splitting: a protocol-buffer file, regular file, ...

Field Summary

Fields

Modifier and Type	Field and Description
protected java.io.InputStream	inp

Fields inherited from class org.apache.hawq.pxf.api.utilities.Plugin

inputData

Constructor Summary

Constructors

Constructor and Description
HdfsAtomicDataAccessor(InputData input) Constructs a HdfsAtomicDataAccessor object.

Method Summary

Modifier and Type	Method and Description
void	closeForRead() Closes the access stream when finished reading the file
boolean	isThreadSafe() Checks if the plugin is thread safe or not, based on inputData.
boolean	openForRead() Opens the file using the non-splittable API for HADOOP HDFS file access This means that instead of using a FileInputFormat for access, we use a Java stream.
OneRow	readNextObject() Fetches one record from the file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

inp

protected java.io.InputStream inp

Constructor Detail

HdfsAtomicDataAccessor

public HdfsAtomicDataAccessor(InputData input)

Constructs a HdfsAtomicDataAccessor object.

Parameters:

input - all input parameters coming from the client

Method Detail

openForRead

public boolean openForRead()
                    throws java.lang.Exception

Opens the file using the non-splittable API for HADOOP HDFS file access This means that instead of using a FileInputFormat for access, we use a Java stream.

Specified by:

openForRead in interface ReadAccessor

Returns:

true for successful file open, false otherwise

Throws:

java.lang.Exception - if opening the resource failed

readNextObject

public OneRow readNextObject()
                      throws java.io.IOException

Fetches one record from the file.

Specified by:

readNextObject in interface ReadAccessor

Returns:

a OneRow record as a Java object. Returns null if none.

Throws:

java.io.IOException

closeForRead

public void closeForRead()
                  throws java.lang.Exception

Closes the access stream when finished reading the file

Specified by:

closeForRead in interface ReadAccessor

Throws:

java.lang.Exception - if closing the resource failed

isThreadSafe

public boolean isThreadSafe()

Description copied from class: Plugin

Checks if the plugin is thread safe or not, based on inputData.

Overrides:

isThreadSafe in class Plugin

Returns:

true if plugin is thread safe