Package | Description |
---|---|
org.apache.hadoop.mapred |
A software framework for easily writing applications which process vast
amounts of data (multi-terabyte data-sets) parallelly on large clusters
(thousands of nodes) built of commodity hardware in a reliable, fault-tolerant
manner.
|
org.apache.hadoop.mapreduce | |
org.apache.hadoop.mapreduce.lib.input | |
org.apache.hadoop.mapreduce.lib.output |
Modifier and Type | Class and Description |
---|---|
class |
TaskAttemptContext
Deprecated.
Use
TaskAttemptContext
instead. |
Modifier and Type | Method and Description |
---|---|
void |
OutputCommitter.abortTask(TaskAttemptContext taskContext)
Deprecated.
This method implements the new interface by calling the old method.
|
void |
OutputCommitter.commitTask(TaskAttemptContext taskContext)
Deprecated.
This method implements the new interface by calling the old method.
|
boolean |
OutputCommitter.needsTaskCommit(TaskAttemptContext taskContext)
Deprecated.
This method implements the new interface by calling the old method.
|
void |
OutputCommitter.setupTask(TaskAttemptContext taskContext)
Deprecated.
This method implements the new interface by calling the old method.
|
Modifier and Type | Class and Description |
---|---|
class |
MapContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
The context that is given to the
Mapper . |
class |
Mapper.Context |
class |
ReduceContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
The context passed to the
Reducer . |
class |
Reducer.Context |
class |
TaskInputOutputContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
A context object that allows input and output from the task.
|
Modifier and Type | Method and Description |
---|---|
abstract void |
OutputCommitter.abortTask(TaskAttemptContext taskContext)
Discard the task output
|
abstract void |
RecordWriter.close(TaskAttemptContext context)
Close this
RecordWriter to future operations. |
abstract void |
OutputCommitter.commitTask(TaskAttemptContext taskContext)
To promote the task's temporary output to final output location
The task's output is moved to the job's output directory.
|
abstract RecordReader<K,V> |
InputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context)
Create a record reader for a given split.
|
abstract OutputCommitter |
OutputFormat.getOutputCommitter(TaskAttemptContext context)
Get the output committer for this output format.
|
abstract RecordWriter<K,V> |
OutputFormat.getRecordWriter(TaskAttemptContext context)
Get the
RecordWriter for the given task. |
abstract void |
RecordReader.initialize(InputSplit split,
TaskAttemptContext context)
Called once at initialization.
|
abstract boolean |
OutputCommitter.needsTaskCommit(TaskAttemptContext taskContext)
Check whether task needs a commit
|
abstract void |
OutputCommitter.setupTask(TaskAttemptContext taskContext)
Sets up output for the task.
|
Modifier and Type | Method and Description |
---|---|
RecordReader<LongWritable,Text> |
TextInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
RecordReader<K,V> |
SequenceFileInputFormat.createRecordReader(InputSplit split,
TaskAttemptContext context) |
void |
LineRecordReader.initialize(InputSplit genericSplit,
TaskAttemptContext context) |
void |
SequenceFileRecordReader.initialize(InputSplit split,
TaskAttemptContext context) |
Modifier and Type | Method and Description |
---|---|
void |
FileOutputCommitter.abortTask(TaskAttemptContext context)
Delete the work directory
|
void |
TextOutputFormat.LineRecordWriter.close(TaskAttemptContext context) |
void |
FileOutputCommitter.commitTask(TaskAttemptContext context)
Move the files from the work directory to the job output directory
|
Path |
FileOutputFormat.getDefaultWorkFile(TaskAttemptContext context,
String extension)
Get the default path and filename for the output format.
|
OutputCommitter |
NullOutputFormat.getOutputCommitter(TaskAttemptContext context) |
OutputCommitter |
FileOutputFormat.getOutputCommitter(TaskAttemptContext context) |
RecordWriter<K,V> |
NullOutputFormat.getRecordWriter(TaskAttemptContext context) |
RecordWriter<K,V> |
SequenceFileOutputFormat.getRecordWriter(TaskAttemptContext context) |
RecordWriter<K,V> |
TextOutputFormat.getRecordWriter(TaskAttemptContext job) |
abstract RecordWriter<K,V> |
FileOutputFormat.getRecordWriter(TaskAttemptContext job) |
static String |
FileOutputFormat.getUniqueFile(TaskAttemptContext context,
String name,
String extension)
Generate a unique filename, based on the task id, name, and extension
|
boolean |
FileOutputCommitter.needsTaskCommit(TaskAttemptContext context)
Did this task write any files in the work directory?
|
void |
FileOutputCommitter.setupTask(TaskAttemptContext context)
No task setup required.
|
Constructor and Description |
---|
FileOutputCommitter(Path outputPath,
TaskAttemptContext context)
Create a file output committer
|
Copyright © 2010 The Apache Software Foundation