org.apache.hadoop.contrib.index.example

Class LineDocRecordReader

java.lang.Object

org.apache.hadoop.contrib.index.example.LineDocRecordReader

All Implemented Interfaces:

RecordReader<DocumentID,LineDocTextAndOp>

public class LineDocRecordReader
extends java.lang.Object
implements RecordReader<DocumentID,LineDocTextAndOp>

A simple RecordReader for LineDoc for plain text files where each line is a doc. Each line is as follows: documentIDopcontent, where op can be "i", "ins" or "insert" for insert, "d", "del" or "delete" for delete, or "u", "upd" or "update" for update.

Constructor Summary

Constructors

Constructor and Description
LineDocRecordReader(org.apache.hadoop.conf.Configuration job, FileSplit split) Constructor

Method Summary

Methods

Modifier and Type	Method and Description
void	close() Close this InputSplit to future operations.
DocumentID	createKey() Create an object of the appropriate type to be used as a key.
LineDocTextAndOp	createValue() Create an object of the appropriate type to be used as a value.
long	getPos() Returns the current position in the input.
float	getProgress() How much of the input has the RecordReader consumed i.e.
boolean	next(DocumentID key, LineDocTextAndOp value) Reads the next key/value pair from the input for processing.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LineDocRecordReader

public LineDocRecordReader(org.apache.hadoop.conf.Configuration job,
                   FileSplit split)
                    throws java.io.IOException

Constructor

Parameters:

job -

split -

Throws:

java.io.IOException

Method Detail

close

public void close()
           throws java.io.IOException

Description copied from interface: RecordReader

Close this InputSplit to future operations.

Specified by:

close in interface RecordReader<DocumentID,LineDocTextAndOp>

Throws:

java.io.IOException

createKey

public DocumentID createKey()

Description copied from interface: RecordReader

Create an object of the appropriate type to be used as a key.

Specified by:

createKey in interface RecordReader<DocumentID,LineDocTextAndOp>

Returns:

a new key object.

createValue

public LineDocTextAndOp createValue()

Description copied from interface: RecordReader

Create an object of the appropriate type to be used as a value.

Specified by:

createValue in interface RecordReader<DocumentID,LineDocTextAndOp>

Returns:

a new value object.

getPos

public long getPos()
            throws java.io.IOException

Description copied from interface: RecordReader

Returns the current position in the input.

Specified by:

getPos in interface RecordReader<DocumentID,LineDocTextAndOp>

Returns:

the current position in the input.

Throws:

java.io.IOException

getProgress

public float getProgress()
                  throws java.io.IOException

Description copied from interface: RecordReader

How much of the input has the RecordReader consumed i.e. has been processed by?

Specified by:

getProgress in interface RecordReader<DocumentID,LineDocTextAndOp>

Returns:

progress from 0.0 to 1.0.

Throws:

java.io.IOException

next

public boolean next(DocumentID key,
           LineDocTextAndOp value)
             throws java.io.IOException

Description copied from interface: RecordReader

Reads the next key/value pair from the input for processing.

Specified by:

next in interface RecordReader<DocumentID,LineDocTextAndOp>

Parameters:

key - the key to read data into

value - the value to read data into

Returns:

true iff a key/value was read, false if at EOF

Throws:

java.io.IOException