Defining a Command-Based Writable External Web Table

You can define writable external web tables to send output rows to an application or script. The application must accept an input stream, reside in the same location on all of the HAWQ segment hosts, and be executable by the gpadmin user. All segments in the HAWQ system run the application or script, whether or not a segment has output rows to process.

Use CREATE WRITABLE EXTERNAL WEB TABLE to define the external table and specify the application or script to run on the segment hosts. Commands execute from within the database and cannot access environment variables (such as $PATH). Set environment variables in the EXECUTE clause of your writable external table definition. For example:

=# CREATE WRITABLE EXTERNAL WEB TABLE output (output text) 
    EXECUTE 'export PATH=$PATH:/home/gpadmin/programs; myprogram.sh' 
    ON 6
    FORMAT 'TEXT'
    DISTRIBUTED RANDOMLY;

The following HAWQ variables are available for use in OS commands executed by a web or writable external table. Set these variables as environment variables in the shell that executes the command(s). They can be used to identify a set of requests made by an external table statement across the HAWQ array of hosts and segment instances.

Table 1. External Table EXECUTE Variables

Variable Description
$GP_CID Command count of the transaction executing the external table statement.
$GP_DATABASE The database in which the external table definition resides.
$GP_DATE The date on which the external table command ran.
$GP_MASTER_HOST The host name of the HAWQ master host from which the external table statement was dispatched.
$GP_MASTER_PORT The port number of the HAWQ master instance from which the external table statement was dispatched.
$GP_SEG_DATADIR The location of the data directory of the segment instance executing the external table command.
$GP_SEG_PG_CONF The location of the hawq-site.xml file of the segment instance executing the external table command.
$GP_SEG_PORT The port number of the segment instance executing the external table command.
$GP_SEGMENT_COUNT The total number of segment instances in the HAWQ system.
$GP_SEGMENT_ID The ID number of the segment instance executing the external table command (same as dbid in gp_segment_configuration).
$GP_SESSION_ID The database session identifier number associated with the external table statement.
$GP_SN Serial number of the external table scan node in the query plan of the external table statement.
$GP_TIME The time the external table command was executed.
$GP_USER The database user executing the external table statement.
$GP_XID The transaction ID of the external table statement.