Package | Description |
---|---|
org.apache.hadoop.contrib.index.mapred | |
org.apache.hadoop.examples.terasort |
This package consists of 3 map/reduce applications for Hadoop to
compete in the annual terabyte sort
competition.
|
org.apache.hadoop.fs |
An abstract file system API.
|
org.apache.hadoop.fs.ftp | |
org.apache.hadoop.fs.kfs |
A client for the Kosmos filesystem (KFS)
|
org.apache.hadoop.fs.s3 |
A distributed, block-based implementation of
FileSystem that uses Amazon S3
as a backing store. |
org.apache.hadoop.fs.s3native |
A distributed implementation of
FileSystem for reading and writing files on
Amazon S3. |
org.apache.hadoop.io |
Generic i/o code for use when reading and writing data to the network,
to databases, and to files.
|
org.apache.hadoop.mapred |
A software framework for easily writing applications which process vast
amounts of data (multi-terabyte data-sets) parallelly on large clusters
(thousands of nodes) built of commodity hardware in a reliable, fault-tolerant
manner.
|
org.apache.hadoop.mapred.lib |
Library of generally useful mappers, reducers, and partitioners.
|
org.apache.hadoop.mapred.lib.db |
org.apache.hadoop.mapred.lib.db Package
|
org.apache.hadoop.mapreduce | |
org.apache.hadoop.util |
Common utilities.
|
Modifier and Type | Method and Description |
---|---|
RecordWriter<Shard,Text> |
IndexUpdateOutputFormat.getRecordWriter(FileSystem fs,
JobConf job,
String name,
Progressable progress) |
Modifier and Type | Method and Description |
---|---|
RecordWriter<Text,Text> |
TeraOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress) |
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
FilterFileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
FSDataOutputStream |
RawLocalFileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
abstract FSDataOutputStream |
FileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
FSDataOutputStream |
ChecksumFileSystem.append(Path f,
int bufferSize,
Progressable progress)
Append to an existing file (optional operation).
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite,
int bufferSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
RawLocalFileSystem.create(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.create(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FilterFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
RawLocalFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
HarFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
abstract FSDataOutputStream |
FileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
ChecksumFileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.create(Path f,
Progressable progress)
Create an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
FSDataOutputStream |
FileSystem.create(Path f,
short replication,
Progressable progress)
Opens an FSDataOutputStream at the indicated Path with write-progress
reporting.
|
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
FTPFileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
FTPFileSystem.create(Path file,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress)
A stream obtained via this call must be closed before using other APIs of
this class or else the invocation will block.
|
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
KosmosFileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
KosmosFileSystem.create(Path file,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
S3FileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
S3FileSystem.create(Path file,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
Modifier and Type | Method and Description |
---|---|
FSDataOutputStream |
NativeS3FileSystem.append(Path f,
int bufferSize,
Progressable progress)
This optional operation is not yet supported.
|
FSDataOutputStream |
NativeS3FileSystem.create(Path f,
FsPermission permission,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress) |
Modifier and Type | Method and Description |
---|---|
SequenceFile.Writer |
SequenceFile.Sorter.cloneFileAttributes(Path inputFile,
Path outputFile,
Progressable prog)
Clones the attributes (like compression of the input file and creates a
corresponding Writer
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
int bufferSize,
short replication,
long blockSize,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress,
SequenceFile.Metadata metadata)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress,
SequenceFile.Metadata metadata)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
Progressable progress)
Construct the preferred type of SequenceFile Writer.
|
void |
SequenceFile.Sorter.setProgressable(Progressable progressable)
Set the progressable object in order to report progress.
|
Constructor and Description |
---|
Writer(Configuration conf,
FileSystem fs,
String file,
Class<? extends Writable> valClass,
SequenceFile.CompressionType compress,
Progressable progress)
Create the named file for values of the named class.
|
Writer(Configuration conf,
FileSystem fs,
String dirName,
Class<? extends WritableComparable> keyClass,
Class<? extends Writable> valClass,
SequenceFile.CompressionType compress,
CompressionCodec codec,
Progressable progress) |
Writer(Configuration conf,
FileSystem fs,
String dirName,
Class<? extends WritableComparable> keyClass,
Class valClass,
SequenceFile.CompressionType compress,
CompressionCodec codec,
Progressable progress)
Create the named map for keys of the named class.
|
Writer(Configuration conf,
FileSystem fs,
String dirName,
Class<? extends WritableComparable> keyClass,
Class valClass,
SequenceFile.CompressionType compress,
Progressable progress) |
Writer(Configuration conf,
FileSystem fs,
String dirName,
Class<? extends WritableComparable> keyClass,
Class valClass,
SequenceFile.CompressionType compress,
Progressable progress)
Create the named map for keys of the named class.
|
Writer(Configuration conf,
FileSystem fs,
String dirName,
WritableComparator comparator,
Class valClass,
SequenceFile.CompressionType compress,
CompressionCodec codec,
Progressable progress) |
Writer(Configuration conf,
FileSystem fs,
String dirName,
WritableComparator comparator,
Class valClass,
SequenceFile.CompressionType compress,
CompressionCodec codec,
Progressable progress)
Create the named map using the named key comparator.
|
Writer(Configuration conf,
FileSystem fs,
String dirName,
WritableComparator comparator,
Class valClass,
SequenceFile.CompressionType compress,
Progressable progress) |
Writer(Configuration conf,
FileSystem fs,
String dirName,
WritableComparator comparator,
Class valClass,
SequenceFile.CompressionType compress,
Progressable progress)
Create the named map using the named key comparator.
|
Writer(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
SequenceFile.Metadata metadata)
Create the named file with write-progress reporter.
|
Writer(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
Progressable progress,
SequenceFile.Metadata metadata)
Create the named file with write-progress reporter.
|
Modifier and Type | Interface and Description |
---|---|
interface |
Reporter
A facility for Map-Reduce applications to report progress and update
counters, status information etc.
|
Modifier and Type | Method and Description |
---|---|
Progressable |
JobContext.getProgressible()
Deprecated.
Get the progress mechanism for reporting progress.
|
Progressable |
TaskAttemptContext.getProgressible()
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
RecordWriter<BytesWritable,BytesWritable> |
SequenceFileAsBinaryOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress) |
RecordWriter<K,V> |
SequenceFileOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress)
Deprecated.
|
RecordWriter<WritableComparable,Writable> |
MapFileOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress) |
RecordWriter<K,V> |
TextOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress)
Deprecated.
|
abstract RecordWriter<K,V> |
FileOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress) |
RecordWriter<K,V> |
OutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress)
Deprecated.
Get the
RecordWriter for the given job. |
Modifier and Type | Method and Description |
---|---|
protected abstract RecordWriter<K,V> |
MultipleOutputFormat.getBaseRecordWriter(FileSystem fs,
JobConf job,
String name,
Progressable arg3) |
protected RecordWriter<K,V> |
MultipleTextOutputFormat.getBaseRecordWriter(FileSystem fs,
JobConf job,
String name,
Progressable arg3) |
protected RecordWriter<K,V> |
MultipleSequenceFileOutputFormat.getBaseRecordWriter(FileSystem fs,
JobConf job,
String name,
Progressable arg3) |
RecordWriter<K,V> |
MultipleOutputFormat.getRecordWriter(FileSystem fs,
JobConf job,
String name,
Progressable arg3)
Create a composite record writer that can write key/value data to different
output files
|
RecordWriter<K,V> |
NullOutputFormat.getRecordWriter(FileSystem ignored,
JobConf job,
String name,
Progressable progress)
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
RecordWriter<K,V> |
DBOutputFormat.getRecordWriter(FileSystem filesystem,
JobConf job,
String name,
Progressable progress)
Get the
RecordWriter for the given job. |
Modifier and Type | Class and Description |
---|---|
class |
MapContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
The context that is given to the
Mapper . |
class |
Mapper.Context |
class |
ReduceContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
The context passed to the
Reducer . |
class |
Reducer.Context |
class |
TaskAttemptContext
The context for task attempts.
|
class |
TaskInputOutputContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
A context object that allows input and output from the task.
|
Modifier and Type | Field and Description |
---|---|
protected Progressable |
ReduceContext.reporter |
Modifier and Type | Method and Description |
---|---|
void |
QuickSort.sort(IndexedSortable s,
int p,
int r,
Progressable rep)
Same as
IndexedSorter.sort(IndexedSortable,int,int) , but indicate progress
periodically. |
void |
HeapSort.sort(IndexedSortable s,
int p,
int r,
Progressable rep)
Same as
IndexedSorter.sort(IndexedSortable,int,int) , but indicate progress
periodically. |
void |
IndexedSorter.sort(IndexedSortable s,
int l,
int r,
Progressable rep)
Same as
IndexedSorter.sort(IndexedSortable,int,int) , but indicate progress
periodically. |
Copyright © 2010 The Apache Software Foundation