water.persist

Class Persist

java.lang.Object

water.persist.Persist

Direct Known Subclasses:

EagerPersistBase, PersistFS, PersistHex, PersistNFS

public abstract class Persist
extends java.lang.Object

Abstract class describing various persistence targets.

• store(Value v) - Store a Value, using storage space.
• load(Value v) - Load a previously stored Value.
• delete(Value v) - Free storage from a previously store Value.

This class is used to implement both user-mode swapping, and the initial load of files - typically raw text for parsing.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Persist.PersistEntry

Constructor Summary

Constructors

Constructor and Description
Persist()

Method Summary

Modifier and Type	Method and Description
abstract java.util.List<java.lang.String>	calcTypeaheadMatches(java.lang.String filter, int limit) Calculate typeahead matches for src
boolean	canHandle(java.lang.String path) Returns true if the persist layer understands given path.
abstract void	cleanUp() Delete persistent storage on startup and shutdown
java.io.OutputStream	create(java.lang.String path, boolean overwrite) Create a new file and return OutputStream for writing.
boolean	delete(java.lang.String path)
abstract void	delete(Value v) Reclaim space from a previously stored Value
boolean	exists(java.lang.String path)
java.lang.String	getHomeDirectory()
java.lang.String	getParent(java.lang.String path)
long	getTotalSpace() Total storage space, or -1 for unknown
long	getUsableSpace() Usable storage space, or -1 for unknown
abstract void	importFiles(java.lang.String path, java.lang.String pattern, java.util.ArrayList<java.lang.String> files, java.util.ArrayList<java.lang.String> keys, java.util.ArrayList<java.lang.String> fails, java.util.ArrayList<java.lang.String> dels)
boolean	isDirectory(java.lang.String path)
boolean	isSeekableOpenSupported() Indicates whether this Persist backend can natively support Seekable InputStreams
protected static java.lang.String	key2Str(Key k)
long	length(java.lang.String path)
Persist.PersistEntry[]	list(java.lang.String path)
byte[]	load(Key k, long skip, int max)
abstract byte[]	load(Value v) Load a previously stored Value
boolean	mkdirs(java.lang.String path)
java.io.InputStream	open(java.lang.String path)
java.io.InputStream	openSeekable(java.lang.String path) Creates a seekable Hadoop implementation of InputStream (FSDataInputStream) h2o-core doesn't depend on Hadoop libraries and can thus not declare the return type specifically
boolean	rename(java.lang.String fromPath, java.lang.String toPath)
abstract void	store(Value v) Store a Value into persistent storage, consuming some storage space.
abstract Key	uriToKey(java.net.URI uri) Transform given uri into file vector holding file name.
java.io.InputStream	wrapSeekable(Vec vec) Creates a Seekable InputStream for a given Vec

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Persist

public Persist()

Method Detail

store

public abstract void store(Value v)
                    throws java.io.IOException

Store a Value into persistent storage, consuming some storage space.

Throws:

java.io.IOException

load

public abstract byte[] load(Value v)
                     throws java.io.IOException

Load a previously stored Value

Throws:

java.io.IOException

load

public byte[] load(Key k,
                   long skip,
                   int max)
            throws java.io.IOException

Throws:

java.io.IOException

delete

public abstract void delete(Value v)

Reclaim space from a previously stored Value

getUsableSpace

public long getUsableSpace()

Usable storage space, or -1 for unknown

getTotalSpace

public long getTotalSpace()

Total storage space, or -1 for unknown

uriToKey

public abstract Key uriToKey(java.net.URI uri)
                      throws java.io.IOException

Transform given uri into file vector holding file name.

Throws:

java.io.IOException

cleanUp

public abstract void cleanUp()

Delete persistent storage on startup and shutdown

calcTypeaheadMatches

public abstract java.util.List<java.lang.String> calcTypeaheadMatches(java.lang.String filter,
                                                                      int limit)

Calculate typeahead matches for src

Parameters:

filter - Source string to match for typeahead

limit - Max number of entries to return

Returns:

List of matches

importFiles

public abstract void importFiles(java.lang.String path,
                                 java.lang.String pattern,
                                 java.util.ArrayList<java.lang.String> files,
                                 java.util.ArrayList<java.lang.String> keys,
                                 java.util.ArrayList<java.lang.String> fails,
                                 java.util.ArrayList<java.lang.String> dels)

key2Str

protected static java.lang.String key2Str(Key k)

getHomeDirectory

public java.lang.String getHomeDirectory()

list

public Persist.PersistEntry[] list(java.lang.String path)

exists

public boolean exists(java.lang.String path)

getParent

public java.lang.String getParent(java.lang.String path)

isDirectory

public boolean isDirectory(java.lang.String path)

length

public long length(java.lang.String path)

open

public java.io.InputStream open(java.lang.String path)

openSeekable

public java.io.InputStream openSeekable(java.lang.String path)

Creates a seekable Hadoop implementation of InputStream (FSDataInputStream) h2o-core doesn't depend on Hadoop libraries and can thus not declare the return type specifically

Parameters:

path - any H2O-3 allowed path

Returns:

instance of FSDataInputStream

isSeekableOpenSupported

public boolean isSeekableOpenSupported()

Indicates whether this Persist backend can natively support Seekable InputStreams

Returns:

true, if openSeekable can be safely called

wrapSeekable

public java.io.InputStream wrapSeekable(Vec vec)

Creates a Seekable InputStream for a given Vec

Parameters:

vec - any Vec in theory typically a ByteVec/FileVec

Returns:

instance of FSDataInputStream

mkdirs

public boolean mkdirs(java.lang.String path)

rename

public boolean rename(java.lang.String fromPath,
                      java.lang.String toPath)

create

public java.io.OutputStream create(java.lang.String path,
                                   boolean overwrite)

Create a new file and return OutputStream for writing. The method creates all directories which does not exists on the referenced path.

Parameters:

path - persist layer specific path

overwrite - overwrite destination file

Returns:

output stream

Throws:

FSIOException - in case of underlying FS error

delete

public boolean delete(java.lang.String path)

canHandle

public boolean canHandle(java.lang.String path)

Returns true if the persist layer understands given path.