org.apache.hadoop.mapred

Class MapOutputFile

java.lang.Object

org.apache.hadoop.mapred.MapOutputFile

@InterfaceAudience.LimitedPrivate(value="MapReduce")
@InterfaceStability.Unstable
public class MapOutputFile
extends java.lang.Object

Manipulate the working area for the transient store for maps and reduces. This class is used by map and reduce tasks to identify the directories that they need to write to/read from for intermediate files. The callers of these methods are from child space and see mapreduce.cluster.local.dir as taskTracker/jobCache/jobId/attemptId This class should not be used from TaskTracker space.

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.fs.Path	getInputFile(int mapId) Return a local reduce input file created earlier
org.apache.hadoop.fs.Path	getInputFileForWrite(TaskID mapId, long size) Create a local reduce input file name.
org.apache.hadoop.fs.Path	getOutputFile() Return the path to local map output file created earlier
org.apache.hadoop.fs.Path	getOutputFileForWrite(long size) Create a local map output file name.
org.apache.hadoop.fs.Path	getOutputIndexFile() Return the path to a local map output index file created earlier
org.apache.hadoop.fs.Path	getOutputIndexFileForWrite(long size) Create a local map output index file name.
org.apache.hadoop.fs.Path	getSpillFile(int spillNumber) Return a local map spill file created earlier.
org.apache.hadoop.fs.Path	getSpillFileForWrite(int spillNumber, long size) Create a local map spill file name.
org.apache.hadoop.fs.Path	getSpillIndexFile(int spillNumber) Return a local map spill index file created earlier
org.apache.hadoop.fs.Path	getSpillIndexFileForWrite(int spillNumber, long size) Create a local map spill index file name.
void	removeAll() Removes all of the files related to a task.
void	setConf(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getOutputFile

public org.apache.hadoop.fs.Path getOutputFile()
                                        throws java.io.IOException

Return the path to local map output file created earlier

Returns:

path

Throws:

java.io.IOException

getOutputFileForWrite

public org.apache.hadoop.fs.Path getOutputFileForWrite(long size)
                                                throws java.io.IOException

Create a local map output file name.

Parameters:

size - the size of the file

Returns:

path

Throws:

java.io.IOException

getOutputIndexFile

public org.apache.hadoop.fs.Path getOutputIndexFile()
                                             throws java.io.IOException

Return the path to a local map output index file created earlier

Returns:

path

Throws:

java.io.IOException

getOutputIndexFileForWrite

public org.apache.hadoop.fs.Path getOutputIndexFileForWrite(long size)
                                                     throws java.io.IOException

Create a local map output index file name.

Parameters:

size - the size of the file

Returns:

path

Throws:

java.io.IOException

getSpillFile

public org.apache.hadoop.fs.Path getSpillFile(int spillNumber)
                                       throws java.io.IOException

Return a local map spill file created earlier.

Parameters:

spillNumber - the number

Returns:

path

Throws:

java.io.IOException

getSpillFileForWrite

public org.apache.hadoop.fs.Path getSpillFileForWrite(int spillNumber,
                                             long size)
                                               throws java.io.IOException

Create a local map spill file name.

Parameters:

spillNumber - the number

size - the size of the file

Returns:

path

Throws:

java.io.IOException

getSpillIndexFile

public org.apache.hadoop.fs.Path getSpillIndexFile(int spillNumber)
                                            throws java.io.IOException

Return a local map spill index file created earlier

Parameters:

spillNumber - the number

Returns:

path

Throws:

java.io.IOException

getSpillIndexFileForWrite

public org.apache.hadoop.fs.Path getSpillIndexFileForWrite(int spillNumber,
                                                  long size)
                                                    throws java.io.IOException

Create a local map spill index file name.

Parameters:

spillNumber - the number

size - the size of the file

Returns:

path

Throws:

java.io.IOException

getInputFile

public org.apache.hadoop.fs.Path getInputFile(int mapId)
                                       throws java.io.IOException

Return a local reduce input file created earlier

Parameters:

mapId - a map task id

Returns:

path

Throws:

java.io.IOException

getInputFileForWrite

public org.apache.hadoop.fs.Path getInputFileForWrite(TaskID mapId,
                                             long size)
                                               throws java.io.IOException

Create a local reduce input file name.

Parameters:

mapId - a map task id

size - the size of the file

Returns:

path

Throws:

java.io.IOException

removeAll

public void removeAll()
               throws java.io.IOException

Removes all of the files related to a task.

Throws:

java.io.IOException

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)