public class FileOutputCommitter extends OutputCommitter
OutputCommitter
that commits files specified
in job output directory i.e. ${mapred.output.dir}.Modifier and Type | Field and Description |
---|---|
protected static String |
TEMP_DIR_NAME
Temporary directory name
|
Constructor and Description |
---|
FileOutputCommitter(Path outputPath,
TaskAttemptContext context)
Create a file output committer
|
Modifier and Type | Method and Description |
---|---|
void |
abortTask(TaskAttemptContext context)
Delete the work directory
|
void |
cleanupJob(JobContext context)
Delete the temporary directory, including all of the work directories.
|
void |
commitTask(TaskAttemptContext context)
Move the files from the work directory to the job output directory
|
Path |
getWorkPath()
Get the directory that the task should write results into
|
boolean |
needsTaskCommit(TaskAttemptContext context)
Did this task write any files in the work directory?
|
void |
setupJob(JobContext context)
Create the temporary directory that is the root of all of the task
work directories.
|
void |
setupTask(TaskAttemptContext context)
No task setup required.
|
protected static final String TEMP_DIR_NAME
public FileOutputCommitter(Path outputPath, TaskAttemptContext context) throws IOException
outputPath
- the job's output pathcontext
- the task's contextIOException
public void setupJob(JobContext context) throws IOException
setupJob
in class OutputCommitter
context
- the job's contextIOException
- if temporary output could not be createdpublic void cleanupJob(JobContext context) throws IOException
cleanupJob
in class OutputCommitter
context
- the job's contextIOException
public void setupTask(TaskAttemptContext context) throws IOException
setupTask
in class OutputCommitter
context
- Context of the task whose output is being written.IOException
public void commitTask(TaskAttemptContext context) throws IOException
commitTask
in class OutputCommitter
context
- the task contextIOException
- if commit is notpublic void abortTask(TaskAttemptContext context)
abortTask
in class OutputCommitter
public boolean needsTaskCommit(TaskAttemptContext context) throws IOException
needsTaskCommit
in class OutputCommitter
context
- the task's contextIOException
public Path getWorkPath() throws IOException
IOException
Copyright © 2010 The Apache Software Foundation