org.apache.hadoop.mapreduce.lib.output

Class SequenceFileAsBinaryOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<K,V>

org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<K,V>

org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

org.apache.hadoop.mapreduce.lib.output.SequenceFileAsBinaryOutputFormat

public class SequenceFileAsBinaryOutputFormat
extends SequenceFileOutputFormat<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

An OutputFormat that writes keys, values to SequenceFiles in binary(raw) format

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SequenceFileAsBinaryOutputFormat.WritableValueBytes Inner class used for appendRaw

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	KEY_CLASS
static java.lang.String	VALUE_CLASS

Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

BASE_OUTPUT_NAME, PART

Constructor Summary

Constructors

Constructor and Description
SequenceFileAsBinaryOutputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
void	checkOutputSpecs(JobContext job) Check for validity of the output-specification for the job.
RecordWriter<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>	getRecordWriter(TaskAttemptContext context) Get the RecordWriter for the given task.
static java.lang.Class<? extends org.apache.hadoop.io.WritableComparable>	getSequenceFileOutputKeyClass(JobContext job) Get the key class for the SequenceFile
static java.lang.Class<? extends org.apache.hadoop.io.Writable>	getSequenceFileOutputValueClass(JobContext job) Get the value class for the SequenceFile
protected org.apache.hadoop.io.SequenceFile.Writer	getSequenceWriter(TaskAttemptContext context, java.lang.Class<?> keyClass, java.lang.Class<?> valueClass)
static void	setSequenceFileOutputKeyClass(Job job, java.lang.Class<?> theClass) Set the key class for the SequenceFile
static void	setSequenceFileOutputValueClass(Job job, java.lang.Class<?> theClass) Set the value class for the SequenceFile

Methods inherited from class org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat

getOutputCompressionType, setOutputCompressionType

Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

getCompressOutput, getDefaultWorkFile, getOutputCommitter, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

KEY_CLASS

public static java.lang.String KEY_CLASS

VALUE_CLASS

public static java.lang.String VALUE_CLASS

Constructor Detail

SequenceFileAsBinaryOutputFormat

public SequenceFileAsBinaryOutputFormat()

Method Detail

setSequenceFileOutputKeyClass

public static void setSequenceFileOutputKeyClass(Job job,
                                 java.lang.Class<?> theClass)

Set the key class for the SequenceFile

This allows the user to specify the key class to be different from the actual class (BytesWritable) used for writing

Parameters:

job - the Job to modify

theClass - the SequenceFile output key class.

setSequenceFileOutputValueClass

public static void setSequenceFileOutputValueClass(Job job,
                                   java.lang.Class<?> theClass)

Set the value class for the SequenceFile

This allows the user to specify the value class to be different from the actual class (BytesWritable) used for writing

Parameters:

job - the Job to modify

theClass - the SequenceFile output key class.

getSequenceFileOutputKeyClass

public static java.lang.Class<? extends org.apache.hadoop.io.WritableComparable> getSequenceFileOutputKeyClass(JobContext job)

Get the key class for the SequenceFile

Returns:

the key class of the SequenceFile

getSequenceFileOutputValueClass

public static java.lang.Class<? extends org.apache.hadoop.io.Writable> getSequenceFileOutputValueClass(JobContext job)

Get the value class for the SequenceFile

Returns:

the value class of the SequenceFile

getRecordWriter

public RecordWriter<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable> getRecordWriter(TaskAttemptContext context)
                                                                                                    throws java.io.IOException

Description copied from class: OutputFormat

Get the RecordWriter for the given task.

Overrides:

getRecordWriter in class SequenceFileOutputFormat<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Parameters:

context - the information about the current task.

Returns:

a RecordWriter to write the output for the job.

Throws:

java.io.IOException

getSequenceWriter

protected org.apache.hadoop.io.SequenceFile.Writer getSequenceWriter(TaskAttemptContext context,
                                                         java.lang.Class<?> keyClass,
                                                         java.lang.Class<?> valueClass)
                                                              throws java.io.IOException

Throws:

java.io.IOException

checkOutputSpecs

public void checkOutputSpecs(JobContext job)
                      throws java.io.IOException

Description copied from class: OutputFormat

Check for validity of the output-specification for the job.

This is to validate the output specification for the job when it is a job is submitted. Typically checks that it does not already exist, throwing an exception when it already exists, so that output is not overwritten.

Overrides:

checkOutputSpecs in class FileOutputFormat<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Parameters:

job - information about the job

Throws:

java.io.IOException - when output should not be attempted