Package de.dlr.proseo.storagemgr.s3

Class S3Storage

java.lang.Object
de.dlr.proseo.storagemgr.s3.S3Storage
All Implemented Interfaces:
Storage
public class S3Storage extends Object implements Storage
An implementation of the Storage interface for a S3-based file system, providing functionality to interact with the storage system, that is retrieving information about the storage, performing file operations, and path conversion.
Author:
Denys Chaykovskiy

  • Constructor Details

    • S3Storage

      public S3Storage(S3Configuration cfg) throws IOException
      Constructor with the s3 configuration
      Parameters:
      cfg - s3 configuration
      Throws:
      IOException - if an I/O exception occurs

  • Method Details

    • getStorageType

      public StorageType getStorageType()
      Gets the storage type.
      Specified by:
      getStorageType in interface Storage
      Returns:
      the storage type

    • getBasePath

      public String getBasePath()
      Gets the base path.
      Specified by:
      getBasePath in interface Storage
      Returns:
      the base path

    • getAbsoluteBasePath

      public String getAbsoluteBasePath()
      Gets the absolute base path (fs prefix + bucket + base path), depends on fs. In other words, it is the absolute path without the relative path in this S3 version. s3://bucket/
      Specified by:
      getAbsoluteBasePath in interface Storage
      Returns:
      the absolute base path

    • getSourcePath

      public String getSourcePath()
      Gets the source path.
      Specified by:
      getSourcePath in interface Storage
      Returns:
      the source path

    • setBucket

      public void setBucket(String bucket) throws IOException
      Sets the bucket.
      Specified by:
      setBucket in interface Storage
      Parameters:
      bucket - the bucket to set
      Throws:
      IOException - if an I/O exception occurs

    • getBucket

      public String getBucket()
      Gets the bucket.
      Specified by:
      getBucket in interface Storage
      Returns:
      the current bucket

    • getBuckets

      public List<String> getBuckets() throws IOException
      Gets the buckets from storage.
      Specified by:
      getBuckets in interface Storage
      Returns:
      the list of buckets
      Throws:
      IOException - if an I/O exception occurs

    • bucketExists

      public boolean bucketExists(String bucketName) throws IOException
      Checks if the bucket exists.
      Specified by:
      bucketExists in interface Storage
      Parameters:
      bucketName - the name of the bucket
      Returns:
      true if the bucket exists
      Throws:
      IOException - if an I/O exception occurs

    • deleteBucket

      public void deleteBucket(String bucketName) throws IOException
      Deletes the bucket in storage.
      Specified by:
      deleteBucket in interface Storage
      Parameters:
      bucketName - the bucket to delete
      Throws:
      IOException - if the bucket cannot be deleted

    • getRelativeFiles

      public List<String> getRelativeFiles(String folder) throws IOException
      Gets the files from storage with the given prefix (folder).
      Specified by:
      getRelativeFiles in interface Storage
      Parameters:
      folder - the prefix (folder) for search in storage
      Returns:
      the list of files with the given prefix
      Throws:
      IOException - if an I/O exception occurs

    • getRelativeFiles

      public List<String> getRelativeFiles() throws IOException
      Gets all files from storage.
      Specified by:
      getRelativeFiles in interface Storage
      Returns:
      the list of all files from storage
      Throws:
      IOException - if an I/O exception occurs

    • getAbsoluteFiles

      public List<String> getAbsoluteFiles(String relativePath) throws IOException
      Gets files (absolute paths) from storage with the given relative path.
      Specified by:
      getAbsoluteFiles in interface Storage
      Parameters:
      relativePath - the relative path for search in storage
      Returns:
      the list of files with the given prefix
      Throws:
      IOException - if an I/O exception occurs

    • getAbsoluteFiles

      public List<String> getAbsoluteFiles() throws IOException
      Gets all files (absolute paths) from storage.
      Specified by:
      getAbsoluteFiles in interface Storage
      Returns:
      the list of all files from storage
      Throws:
      IOException - if an I/O exception occurs

    • getRelativePath

      public String getRelativePath(String absolutePath)
      Gets the relative path from the absolute path by removing the S3 prefix, bucket, and left slash.
      Specified by:
      getRelativePath in interface Storage
      Parameters:
      absolutePath - the absolute path
      Returns:
      the relative path

    • getRelativePath

      public List<String> getRelativePath(List<String> absolutePaths)
      Gets the relative paths from the absolute paths by removing the S3 prefix, bucket, and left slash.
      Specified by:
      getRelativePath in interface Storage
      Parameters:
      absolutePaths - the absolute paths
      Returns:
      the relative paths

    • getAbsolutePath

      public String getAbsolutePath(String relativePath)
      Gets the absolute path (s3://<bucket>/<relativePath>).
      Specified by:
      getAbsolutePath in interface Storage
      Parameters:
      relativePath - the relative path
      Returns:
      the absolute file depending on the storage file system

    • getAbsolutePath

      public List<String> getAbsolutePath(List<String> relativePaths)
      Gets the absolute paths (s3://<bucket>/<relativePath>).
      Specified by:
      getAbsolutePath in interface Storage
      Parameters:
      relativePaths - the relative paths
      Returns:
      the absolute paths depending on the storage file system

    • getStorageFile

      public StorageFile getStorageFile(String relativePath)
      Gets the Storage File.
      Specified by:
      getStorageFile in interface Storage
      Parameters:
      relativePath - the relative path in storage to the file
      Returns:
      the storage file object

    • getStorageFiles

      public List<StorageFile> getStorageFiles() throws IOException
      Gets the storage files.
      Specified by:
      getStorageFiles in interface Storage
      Returns:
      the list of storage files
      Throws:
      IOException - if an I/O exception occurs

    • createStorageFile

      public StorageFile createStorageFile(String relativePath, String content) throws IOException
      Creates the physical storage file.
      Specified by:
      createStorageFile in interface Storage
      Parameters:
      relativePath - the relative path of the file
      content - the content of the file
      Returns:
      the storage file object of the created file
      Throws:
      IOException - if the file cannot be created

    • fileExists

      public boolean fileExists(StorageFile storageFile) throws IOException
      Checks if the file exists.
      Specified by:
      fileExists in interface Storage
      Parameters:
      storageFile - the storage file to check
      Returns:
      true if the file exists physically
      Throws:
      IOException - if an I/O exception occurs

    • isFile

      public boolean isFile(StorageFile storageFileOrDir) throws IOException
      Checks if the storage file or directory is a file (no slash at the end of the path).
      Specified by:
      isFile in interface Storage
      Parameters:
      storageFileOrDir - the storage file or directory
      Returns:
      true if the storage file is a file
      Throws:
      IOException - if an I/O exception occurs

    • isDirectory

      public boolean isDirectory(StorageFile storageFileOrDir) throws IOException
      Checks if the storage file or directory is a directory.
      Specified by:
      isDirectory in interface Storage
      Parameters:
      storageFileOrDir - the storage file or directory
      Returns:
      true if the storage file is a file
      Throws:
      IOException - if an I/O exception occurs

    • getFileSize

      public long getFileSize(StorageFile storageFile) throws IOException
      Gets the file size.
      Specified by:
      getFileSize in interface Storage
      Parameters:
      storageFile - the storage file
      Returns:
      the file size of the storage file
      Throws:
      IOException - if an I/O exception occurs

    • getFileContent

      public String getFileContent(StorageFile storageFile) throws IOException
      Gets the file content.
      Specified by:
      getFileContent in interface Storage
      Parameters:
      storageFile - the storage file
      Returns:
      the file content
      Throws:
      IOException - if an error occurs while getting the file content

    • uploadFile

      public String uploadFile(StorageFile sourceFile, StorageFile targetFileOrDir) throws IOException
      Uploads the file to storage.
      Specified by:
      uploadFile in interface Storage
      Parameters:
      sourceFile - the source file to upload
      targetFileOrDir - the target file or directory in the storage
      Returns:
      the uploaded storage file
      Throws:
      IOException - if the file cannot be uploaded

    • upload

      public List<String> upload(StorageFile sourceFileOrDir, StorageFile targetFileOrDir) throws IOException
      Uploads the file or directory recursively to storage.
      Specified by:
      upload in interface Storage
      Parameters:
      sourceFileOrDir - the source file or directory
      targetFileOrDir - the target file or directory in the storage
      Returns:
      the list of uploaded files
      Throws:
      IOException - if the file or directory cannot be uploaded

    • upload

      public List<String> upload(StorageFile sourceFileOrDir) throws IOException
      Uploads the file or directory recursively to the storage.
      Specified by:
      upload in interface Storage
      Parameters:
      sourceFileOrDir - the source file or dir to upload
      Returns:
      the list of uploaded files
      Throws:
      IOException - if the file or directory cannot be uploaded

    • uploadFile

      public String uploadFile(StorageFile sourceFile) throws IOException
      Uploads the file to the storage.
      Specified by:
      uploadFile in interface Storage
      Parameters:
      sourceFile - the source file to upload
      Returns:
      the path of the uploaded file
      Throws:
      IOException - if the file cannot be uploaded

    • uploadSourceFileOrDir

      public List<String> uploadSourceFileOrDir(String relativeSourceFileOrDir) throws IOException
      Uploads the source file or directory recursively to the storage.
      Specified by:
      uploadSourceFileOrDir in interface Storage
      Parameters:
      relativeSourceFileOrDir - the relative path to source file or directory
      Returns:
      the list of uploaded to storage source files
      Throws:
      IOException - if the file cannot be uploaded

    • uploadSourceFile

      public String uploadSourceFile(String relativeSourceFile) throws IOException
      Uploads the source file.
      Specified by:
      uploadSourceFile in interface Storage
      Parameters:
      relativeSourceFile - the relative path to source file
      Returns:
      the uploaded to storage source file
      Throws:
      IOException - if the file cannot be uploaded

    • download

      public List<String> download(StorageFile sourceFileOrDir, StorageFile targetFileOrDir) throws IOException
      Downloads the file or directory recursively.
      Specified by:
      download in interface Storage
      Parameters:
      sourceFileOrDir - the source file or directory in the storage
      targetFileOrDir - the target file or directory
      Returns:
      the list of downloaded files
      Throws:
      IOException - if the file cannot be downloaded

    • downloadFile

      public String downloadFile(StorageFile sourceFile, StorageFile targetFileOrDir) throws IOException
      Downloads the file from storage.
      Specified by:
      downloadFile in interface Storage
      Parameters:
      sourceFile - the storage source file to download
      targetFileOrDir - the target file or directory
      Returns:
      the Storage File object with the downloaded file
      Throws:
      IOException - if the file cannot be downloaded

    • delete

      public List<String> delete(StorageFile storageFileOrDir) throws IOException
      Deletes the file or directory recursively from the storage.
      Specified by:
      delete in interface Storage
      Parameters:
      storageFileOrDir - the Storage file or directory to delete
      Returns:
      the list of deleted files from storage
      Throws:
      IOException - if the file or directory cannot be deleted

    • deleteFile

      public String deleteFile(StorageFile storageFile) throws IOException
      Deletes the storage file from storage.
      Specified by:
      deleteFile in interface Storage
      Parameters:
      storageFile - the storage file to delete
      Returns:
      the deleted storage file
      Throws:
      IOException - if the file cannot be deleted

    • delete

      public List<String> delete(String relativeFileOrDir) throws IOException
      Deletes the file or directory recursively from the storage.
      Specified by:
      delete in interface Storage
      Parameters:
      relativeFileOrDir - the Storage file or directory to delete
      Returns:
      the list of deleted files from storage
      Throws:
      IOException - if the file or directory cannot be deleted

    • addFSPrefix

      public String addFSPrefix(String path)
      Adds the file system prefix to the path.
      Specified by:
      addFSPrefix in interface Storage
      Parameters:
      path - the path to extend
      Returns:
      the file system prefix + "|" + path

    • addFSPrefix

      public List<String> addFSPrefix(List<String> paths)
      Adds the file system prefix to paths.
      Specified by:
      addFSPrefix in interface Storage
      Parameters:
      paths - the paths to extend
      Returns:
      the list of file system prefix + path

    • getInputStream

      public InputStream getInputStream(StorageFile storageFile) throws IOException
      Gets the input stream from the file.
      Specified by:
      getInputStream in interface Storage
      Parameters:
      storageFile - the storage file
      Returns:
      the input stream from the file
      Throws:
      IOException - if an error occurs while getting the input stream