org.apache.hawq.pxf.api

Class Fragmenter

java.lang.Object

org.apache.hawq.pxf.api.utilities.Plugin

org.apache.hawq.pxf.api.Fragmenter

Direct Known Subclasses:

HBaseDataFragmenter, HdfsDataFragmenter, HiveDataFragmenter

public abstract class Fragmenter
extends Plugin

Abstract class that defines the splitting of a data resource into fragments that can be processed in parallel.

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<Fragment>	fragments

Fields inherited from class org.apache.hawq.pxf.api.utilities.Plugin

inputData

Constructor Summary

Constructors

Constructor and Description
Fragmenter(InputData metaData) Constructs a Fragmenter.

Method Summary

Modifier and Type	Method and Description
abstract java.util.List<Fragment>	getFragments() Gets the fragments of a given path (source name and location of each fragment).
FragmentsStats	getFragmentsStats() Default implementation of statistics for fragments.

Methods inherited from class org.apache.hawq.pxf.api.utilities.Plugin

isThreadSafe

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

fragments

protected java.util.List<Fragment> fragments

Constructor Detail

Fragmenter

public Fragmenter(InputData metaData)

Constructs a Fragmenter.

Parameters:

metaData - the input data

Method Detail

getFragments

public abstract java.util.List<Fragment> getFragments()
                                               throws java.lang.Exception

Gets the fragments of a given path (source name and location of each fragment). Used to get fragments of data that could be read in parallel from the different segments.

Returns:

list of data fragments

Throws:

java.lang.Exception - if fragment list could not be retrieved

getFragmentsStats

public FragmentsStats getFragmentsStats()
                                 throws java.lang.Exception

Default implementation of statistics for fragments. The default is:

• number of fragments - as gathered by getFragments()
• first fragment size - 64MB
• total size - number of fragments times first fragment size

Each fragmenter implementation can override this method to better match its fragments stats.

Returns:

default statistics

Throws:

java.lang.Exception - if statistics cannot be gathered