org.apache.hadoop.mapred

Class FileOutputCommitter

java.lang.Object

org.apache.hadoop.mapreduce.OutputCommitter

org.apache.hadoop.mapred.OutputCommitter

org.apache.hadoop.mapred.FileOutputCommitter

public class FileOutputCommitter
extends OutputCommitter

An OutputCommitter that commits files specified in job output directory i.e. ${mapred.output.dir}.

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.commons.logging.Log	LOG
static java.lang.String	SUCCEEDED_FILE_NAME
static java.lang.String	TEMP_DIR_NAME Temporary directory name

Constructor Summary

Constructors

Constructor and Description
FileOutputCommitter()

Method Summary

Methods

Modifier and Type	Method and Description
void	abortJob(JobContext context, int runState) Delete the temporary directory, including all of the work directories.
void	abortTask(TaskAttemptContext context) Discard the task output
void	cleanupJob(JobContext context) Deprecated.
void	commitJob(JobContext context) For committing job's output after successful job completion.
void	commitTask(TaskAttemptContext context) To promote the task's temporary output to final output location The task's output is moved to the job's output directory.
boolean	needsTaskCommit(TaskAttemptContext context) Check whether task needs a commit
void	setupJob(JobContext context) For the framework to setup the job output during initialization
void	setupTask(TaskAttemptContext context) Sets up output for the task.

Methods inherited from class org.apache.hadoop.mapred.OutputCommitter

abortJob, abortTask, cleanupJob, commitJob, commitTask, needsTaskCommit, setupJob, setupTask

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

TEMP_DIR_NAME

public static final java.lang.String TEMP_DIR_NAME

Temporary directory name

See Also:

Constant Field Values

SUCCEEDED_FILE_NAME

public static final java.lang.String SUCCEEDED_FILE_NAME

See Also:

Constant Field Values

Constructor Detail

FileOutputCommitter

public FileOutputCommitter()

Method Detail

setupJob

public void setupJob(JobContext context)
              throws java.io.IOException

Description copied from class: OutputCommitter

For the framework to setup the job output during initialization

Specified by:

setupJob in class OutputCommitter

Parameters:

context - Context of the job whose output is being written.

Throws:

java.io.IOException - if temporary output could not be created

commitJob

public void commitJob(JobContext context)
               throws java.io.IOException

Description copied from class: OutputCommitter

For committing job's output after successful job completion. Note that this is invoked for jobs with final run state as JobStatus.SUCCEEDED.

Overrides:

commitJob in class OutputCommitter

Parameters:

context - Context of the job whose output is being written.

Throws:

java.io.IOException

cleanupJob

@Deprecated
public void cleanupJob(JobContext context)
                throws java.io.IOException

Deprecated.

Description copied from class: OutputCommitter

For cleaning up the job's output after job completion

Overrides:

cleanupJob in class OutputCommitter

Throws:

java.io.IOException

abortJob

public void abortJob(JobContext context,
            int runState)
              throws java.io.IOException

Delete the temporary directory, including all of the work directories.

Overrides:

abortJob in class OutputCommitter

Parameters:

context - the job's context

runState - final run state of the job, should be JobStatus.KILLED or JobStatus.FAILED

Throws:

java.io.IOException

setupTask

public void setupTask(TaskAttemptContext context)
               throws java.io.IOException

Description copied from class: OutputCommitter

Sets up output for the task.

Specified by:

setupTask in class OutputCommitter

Parameters:

context - Context of the task whose output is being written.

Throws:

java.io.IOException

commitTask

public void commitTask(TaskAttemptContext context)
                throws java.io.IOException

Description copied from class: OutputCommitter

To promote the task's temporary output to final output location The task's output is moved to the job's output directory.

Specified by:

commitTask in class OutputCommitter

Parameters:

context - Context of the task whose output is being written.

Throws:

java.io.IOException - if commit is not

abortTask

public void abortTask(TaskAttemptContext context)
               throws java.io.IOException

Description copied from class: OutputCommitter

Discard the task output

Specified by:

abortTask in class OutputCommitter

Throws:

java.io.IOException

needsTaskCommit

public boolean needsTaskCommit(TaskAttemptContext context)
                        throws java.io.IOException

Description copied from class: OutputCommitter

Check whether task needs a commit

Specified by:

needsTaskCommit in class OutputCommitter

Returns:

true/false

Throws:

java.io.IOException