alluxio.client.block.stream

Class BlockInStream

java.lang.Object

java.io.InputStream

alluxio.client.block.stream.BlockInStream

All Implemented Interfaces:

BoundedStream, PositionedReadable, alluxio.Positioned, alluxio.Seekable, Closeable, AutoCloseable

@NotThreadSafe
public class BlockInStream
extends InputStream
implements BoundedStream, alluxio.Seekable, PositionedReadable

Provides an InputStream implementation that is based on DataReaders to stream data chunk by chunk.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BlockInStream.BlockInStreamSource the source tracking where the block is from.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BlockInStream(DataReader.Factory dataReaderFactory, alluxio.wire.WorkerNetAddress address, BlockInStream.BlockInStreamSource blockSource, long id, long length) Creates an instance of BlockInStream.

Method Summary

Modifier and Type	Method and Description
void	close()
static BlockInStream	create(FileSystemContext context, alluxio.wire.BlockInfo info, alluxio.wire.WorkerNetAddress dataSource, BlockInStream.BlockInStreamSource dataSourceType, InStreamOptions options) Creates a BlockInStream.
static BlockInStream	createRemoteBlockInStream(FileSystemContext context, long blockId, alluxio.wire.WorkerNetAddress address, BlockInStream.BlockInStreamSource blockSource, long blockSize, alluxio.proto.dataserver.Protocol.OpenUfsBlockOptions ufsOptions) Creates a BlockInStream to read from a specific remote server.
alluxio.wire.WorkerNetAddress	getAddress()
long	getId()
long	getPos()
BlockInStream.BlockInStreamSource	getSource()
boolean	isShortCircuit()
int	positionedRead(long pos, byte[] b, int off, int len) Reads up to the specified number of bytes, from a given position within a file, and return the number of bytes read.
int	read()
int	read(byte[] b)
int	read(byte[] b, int off, int len)
long	remaining()
void	seek(long pos)
long	skip(long n)

Methods inherited from class java.io.InputStream

available, mark, markSupported, reset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BlockInStream

protected BlockInStream(DataReader.Factory dataReaderFactory,
                        alluxio.wire.WorkerNetAddress address,
                        BlockInStream.BlockInStreamSource blockSource,
                        long id,
                        long length)

Creates an instance of BlockInStream.

Parameters:

dataReaderFactory - the data reader factory

address - the address of the gRPC data server

blockSource - the source location of the block

id - the ID (either block ID or UFS file ID)

length - the length

Method Detail

create

public static BlockInStream create(FileSystemContext context,
                                   alluxio.wire.BlockInfo info,
                                   alluxio.wire.WorkerNetAddress dataSource,
                                   BlockInStream.BlockInStreamSource dataSourceType,
                                   InStreamOptions options)
                            throws IOException

Creates a BlockInStream. One of several read behaviors: 1. Domain socket - if the data source is the local worker and the local worker has a domain socket server 2. Short-Circuit - if the data source is the local worker 3. Local Loopback Read - if the data source is the local worker and short circuit is disabled 4. Read from remote worker - if the data source is a remote worker 5. UFS Read from worker - if the data source is UFS, read from the UFS policy's designated worker (ufs -> local or remote worker -> client)

Parameters:

context - the file system context

info - the block info

dataSource - the Alluxio worker which should read the data

dataSourceType - the source location of the block

options - the instream options

Returns:

the BlockInStream object

Throws:

IOException

createRemoteBlockInStream

public static BlockInStream createRemoteBlockInStream(FileSystemContext context,
                                                      long blockId,
                                                      alluxio.wire.WorkerNetAddress address,
                                                      BlockInStream.BlockInStreamSource blockSource,
                                                      long blockSize,
                                                      alluxio.proto.dataserver.Protocol.OpenUfsBlockOptions ufsOptions)

Creates a BlockInStream to read from a specific remote server. Should only be used in cases where the data source and method of reading is known, ie. worker - worker communication.

Parameters:

context - the file system context

blockId - the block id

address - the address of the gRPC data server

blockSource - the source location of the block

blockSize - the size of the block

ufsOptions - the ufs read options

Returns:

the BlockInStream created

getPos

public long getPos()

Specified by:

getPos in interface alluxio.Positioned

read

public int read()
         throws IOException

Specified by:

read in class InputStream

Throws:

IOException

read

public int read(byte[] b)
         throws IOException

Overrides:

read in class InputStream

Throws:

IOException

read

public int read(byte[] b,
                int off,
                int len)
         throws IOException

Overrides:

read in class InputStream

Throws:

IOException

positionedRead

public int positionedRead(long pos,
                          byte[] b,
                          int off,
                          int len)
                   throws IOException

Description copied from interface: PositionedReadable

Reads up to the specified number of bytes, from a given position within a file, and return the number of bytes read. This does not change the current offset of a file, and is thread-safe. Not all implementations meet the thread safety requirement. According to the documentation of the PositionedReadable interface in Hadoop, those that do not meet the requirement cannot be used as a backing store for some applications, such as Apache HBase. All implementations of this interface in Alluxio must meet the thread safety requirement.

Specified by:

positionedRead in interface PositionedReadable

Parameters:

pos - position within file

b - destination buffer

off - offset in the buffer

len - number of bytes to read

Returns:

actual number of bytes read; -1 means "EOF";

Throws:

IOException

remaining

public long remaining()

Specified by:

remaining in interface BoundedStream

Returns:

the remaining number of bytes left in the stream, starting at the current position

seek

public void seek(long pos)
          throws IOException

Specified by:

seek in interface alluxio.Seekable

Throws:

IOException

skip

public long skip(long n)
          throws IOException

Overrides:

skip in class InputStream

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class InputStream

Throws:

IOException

isShortCircuit

public boolean isShortCircuit()

Returns:

whether the reader is reading data directly from a local file

getAddress

public alluxio.wire.WorkerNetAddress getAddress()

Returns:

the address of the data server

getSource

public BlockInStream.BlockInStreamSource getSource()

Returns:

the source of the block location

getId

public long getId()

Returns:

the block ID