org.apache.hadoop.mapreduce.lib.db

Class DataDrivenDBRecordReader<T extends DBWritable>

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,T>

org.apache.hadoop.mapreduce.lib.db.DBRecordReader<T>

org.apache.hadoop.mapreduce.lib.db.DataDrivenDBRecordReader<T>

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

Direct Known Subclasses:

MySQLDataDrivenDBRecordReader, OracleDataDrivenDBRecordReader

public class DataDrivenDBRecordReader<T extends DBWritable>
extends DBRecordReader<T>

A RecordReader that reads records from a SQL table, using data-driven WHERE clause splits. Emits LongWritables containing the record number as key and DBWritables as value.

Constructor Summary

Constructors

Constructor and Description
DataDrivenDBRecordReader(DBInputFormat.DBInputSplit split, java.lang.Class<T> inputClass, org.apache.hadoop.conf.Configuration conf, java.sql.Connection conn, DBConfiguration dbConfig, java.lang.String cond, java.lang.String[] fields, java.lang.String table, java.lang.String dbProduct)

Method Summary

Methods

Modifier and Type	Method and Description
protected java.lang.String	getSelectQuery() Returns the query for selecting the records, subclasses can override this for custom behaviour.

Methods inherited from class org.apache.hadoop.mapreduce.lib.db.DBRecordReader

close, createValue, executeQuery, getConditions, getConnection, getCurrentKey, getCurrentValue, getDBConf, getFieldNames, getPos, getProgress, getSplit, getStatement, getTableName, initialize, next, nextKeyValue, setStatement

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataDrivenDBRecordReader

public DataDrivenDBRecordReader(DBInputFormat.DBInputSplit split,
                        java.lang.Class<T> inputClass,
                        org.apache.hadoop.conf.Configuration conf,
                        java.sql.Connection conn,
                        DBConfiguration dbConfig,
                        java.lang.String cond,
                        java.lang.String[] fields,
                        java.lang.String table,
                        java.lang.String dbProduct)
                         throws java.sql.SQLException

Parameters:

split - The InputSplit to read data for

Throws:

java.sql.SQLException

Method Detail

getSelectQuery

protected java.lang.String getSelectQuery()

Returns the query for selecting the records, subclasses can override this for custom behaviour.

Overrides:

getSelectQuery in class DBRecordReader<T extends DBWritable>