org.apache.hadoop.mapreduce.server.tasktracker

Class Localizer

java.lang.Object

org.apache.hadoop.mapreduce.server.tasktracker.Localizer

public class Localizer
extends java.lang.Object

NOTE: This class is internal only and not intended for users!!

Constructor Summary

Constructors

Constructor and Description
Localizer(org.apache.hadoop.fs.FileSystem fileSys, java.lang.String[] lDirs) Create a Localizer instance

Method Summary

Methods

Modifier and Type	Method and Description
void	initializeAttemptDirs(java.lang.String user, java.lang.String jobId, java.lang.String attemptId) Create taskDirs on all the disks.
void	initializeJobDirs(java.lang.String user, JobID jobId) Prepare the job directories for a given job.
void	initializeJobLogDir(JobID jobId) Create job log directory and set appropriate permissions for the directory.
void	initializeUserDirs(java.lang.String user) Initialize the local directories for a particular user on this TT.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Localizer

public Localizer(org.apache.hadoop.fs.FileSystem fileSys,
         java.lang.String[] lDirs)

Create a Localizer instance

Parameters:

fileSys -

lDirs -

Method Detail

initializeUserDirs

public void initializeUserDirs(java.lang.String user)
                        throws java.io.IOException

Initialize the local directories for a particular user on this TT. This involves creation and setting permissions of the following directories

• $mapred.local.dir/taskTracker/$user
• $mapred.local.dir/taskTracker/$user/jobcache
• $mapred.local.dir/taskTracker/$user/distcache

Parameters:

user -

Throws:

java.io.IOException

initializeJobDirs

public void initializeJobDirs(java.lang.String user,
                     JobID jobId)
                       throws java.io.IOException

Prepare the job directories for a given job. To be called by the job localization code, only if the job is not already localized.
Here, we set 700 permissions on the job directories created on all disks. This we do so as to avoid any misuse by other users till the time TaskController.initializeJob(java.lang.String, java.lang.String, org.apache.hadoop.fs.Path, org.apache.hadoop.fs.Path, org.apache.hadoop.mapred.TaskUmbilicalProtocol, java.net.InetSocketAddress) is run at a later time to set proper private permissions on the job directories.

Parameters:

user -

jobId -

Throws:

java.io.IOException

initializeAttemptDirs

public void initializeAttemptDirs(java.lang.String user,
                         java.lang.String jobId,
                         java.lang.String attemptId)
                           throws java.io.IOException

Create taskDirs on all the disks. Otherwise, in some cases, like when LinuxTaskController is in use, child might wish to balance load across disks but cannot itself create attempt directory because of the fact that job directory is writable only by the TT.

Parameters:

user -

jobId -

attemptId -

Throws:

java.io.IOException

initializeJobLogDir

public void initializeJobLogDir(JobID jobId)
                         throws java.io.IOException

Create job log directory and set appropriate permissions for the directory.

Parameters:

jobId -

Throws:

java.io.IOException