Package | Description |
---|---|
org.apache.hadoop.io |
Generic i/o code for use when reading and writing data to the network,
to databases, and to files.
|
org.apache.hadoop.io.compress | |
org.apache.hadoop.mapred |
A software framework for easily writing applications which process vast
amounts of data (multi-terabyte data-sets) parallelly on large clusters
(thousands of nodes) built of commodity hardware in a reliable, fault-tolerant
manner.
|
org.apache.hadoop.mapreduce.lib.output |
Modifier and Type | Method and Description |
---|---|
CompressionCodec |
SequenceFile.Writer.getCompressionCodec()
Returns the compression codec of data in this file.
|
CompressionCodec |
SequenceFile.Reader.getCompressionCodec()
Returns the compression codec of data in this file.
|
Modifier and Type | Method and Description |
---|---|
static SequenceFile.Writer |
SequenceFile.createWriter(Configuration conf,
FSDataOutputStream out,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec)
Construct the preferred type of 'raw' SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(Configuration conf,
FSDataOutputStream out,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
SequenceFile.Metadata metadata)
Construct the preferred type of 'raw' SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
int bufferSize,
short replication,
long blockSize,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress,
SequenceFile.Metadata metadata)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress)
Construct the preferred type of SequenceFile Writer.
|
static SequenceFile.Writer |
SequenceFile.createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress,
SequenceFile.Metadata metadata)
Construct the preferred type of SequenceFile Writer.
|
Constructor and Description |
---|
Writer(Configuration conf,
FileSystem fs,
String dirName,
Class<? extends WritableComparable> keyClass,
Class<? extends Writable> valClass,
SequenceFile.CompressionType compress,
CompressionCodec codec,
Progressable progress) |
Writer(Configuration conf,
FileSystem fs,
String dirName,
Class<? extends WritableComparable> keyClass,
Class valClass,
SequenceFile.CompressionType compress,
CompressionCodec codec,
Progressable progress)
Create the named map for keys of the named class.
|
Writer(Configuration conf,
FileSystem fs,
String dirName,
WritableComparator comparator,
Class valClass,
SequenceFile.CompressionType compress,
CompressionCodec codec,
Progressable progress) |
Writer(Configuration conf,
FileSystem fs,
String dirName,
WritableComparator comparator,
Class valClass,
SequenceFile.CompressionType compress,
CompressionCodec codec,
Progressable progress)
Create the named map using the named key comparator.
|
Modifier and Type | Class and Description |
---|---|
class |
BZip2Codec
This class provides CompressionOutputStream and CompressionInputStream for
compression and decompression.
|
class |
DefaultCodec |
class |
GzipCodec
This class creates gzip compressors/decompressors.
|
Modifier and Type | Method and Description |
---|---|
CompressionCodec |
CompressionCodecFactory.getCodec(Path file)
Find the relevant compression codec for the given file based on its
filename suffix.
|
Modifier and Type | Method and Description |
---|---|
static List<Class<? extends CompressionCodec>> |
CompressionCodecFactory.getCodecClasses(Configuration conf)
Get the list of codecs listed in the configuration
|
Modifier and Type | Method and Description |
---|---|
static Compressor |
CodecPool.getCompressor(CompressionCodec codec)
Get a
Compressor for the given CompressionCodec from the
pool or a new one. |
static Decompressor |
CodecPool.getDecompressor(CompressionCodec codec)
Get a
Decompressor for the given CompressionCodec from the
pool or a new one. |
Modifier and Type | Method and Description |
---|---|
Class<? extends CompressionCodec> |
JobConf.getMapOutputCompressorClass(Class<? extends CompressionCodec> defaultValue)
Deprecated.
Get the
CompressionCodec for compressing the map outputs. |
static Class<? extends CompressionCodec> |
FileOutputFormat.getOutputCompressorClass(JobConf conf,
Class<? extends CompressionCodec> defaultValue)
Get the
CompressionCodec for compressing the job outputs. |
Modifier and Type | Method and Description |
---|---|
Class<? extends CompressionCodec> |
JobConf.getMapOutputCompressorClass(Class<? extends CompressionCodec> defaultValue)
Deprecated.
Get the
CompressionCodec for compressing the map outputs. |
static Class<? extends CompressionCodec> |
FileOutputFormat.getOutputCompressorClass(JobConf conf,
Class<? extends CompressionCodec> defaultValue)
Get the
CompressionCodec for compressing the job outputs. |
void |
JobConf.setMapOutputCompressorClass(Class<? extends CompressionCodec> codecClass)
Deprecated.
Set the given class as the
CompressionCodec for the map outputs. |
static void |
FileOutputFormat.setOutputCompressorClass(JobConf conf,
Class<? extends CompressionCodec> codecClass)
Set the
CompressionCodec to be used to compress job outputs. |
Modifier and Type | Method and Description |
---|---|
static Class<? extends CompressionCodec> |
FileOutputFormat.getOutputCompressorClass(JobContext job,
Class<? extends CompressionCodec> defaultValue)
Get the
CompressionCodec for compressing the job outputs. |
Modifier and Type | Method and Description |
---|---|
static Class<? extends CompressionCodec> |
FileOutputFormat.getOutputCompressorClass(JobContext job,
Class<? extends CompressionCodec> defaultValue)
Get the
CompressionCodec for compressing the job outputs. |
static void |
FileOutputFormat.setOutputCompressorClass(Job job,
Class<? extends CompressionCodec> codecClass)
Set the
CompressionCodec to be used to compress job outputs. |
Copyright © 2010 The Apache Software Foundation