Package de.dlr.proseo.storagemgr.s3

Class S3DAL

java.lang.Object
de.dlr.proseo.storagemgr.s3.S3DAL
public class S3DAL extends Object
A data access layer for interacting with a S3-based storage system. It provides various methods to perform operations such as retrieving files, checking file or directory existence, obtaining file content, uploading files, downloading files, and deleting files or directories within the S3 storage system. This class utilizes the Amazon S3 SDK v2 for S3 client operations and requires AWS credentials and a S3 configuration to be initialized.
Author:
Denys Chaykovskiy

  • Constructor Details

  • Method Details

    • getConfiguration

      public S3Configuration getConfiguration()
      Gets the S3 configuration used by this S3DAL instance.
      Returns:
      the S3Configuration object

    • initS3ClientV1

      public void initS3ClientV1() throws IOException
      Initializes the S3 v1 client. Note that this method relies on the configuration passed to the current object upon construction. It is assumed that this configuration is derived from the application configuration without modification, so it is the same for every S3DAL object.
      Throws:
      IOException - if an I/O exception occurs

    • initS3ClientV2

      public void initS3ClientV2() throws IOException
      Initializes the S3 v2 client. Note that this method relies on the configuration passed to the current object upon construction. It is assumed that this configuration is derived from the application configuration without modification, so it is the same for every S3DAL object.
      Throws:
      IOException - if an I/O exception occurs

    • getAllFiles

      public List<String> getAllFiles() throws IOException
      Retrieves a list of all files in the default bucket.
      Returns:
      a list of all file paths
      Throws:
      IOException - if an I/O exception occurs

    • getAllFiles

      public List<String> getAllFiles(String bucket) throws IOException
      Retrieves a list of all files in the named bucket.
      Parameters:
      bucket - the bucket to search in
      Returns:
      a list of all file paths
      Throws:
      IOException - if an I/O exception occurs

    • getFiles

      public List<String> getFiles(String folder) throws IOException
      Retrieves a list of files from the default bucket that match the specified folder (prefix).
      Parameters:
      folder - the folder (prefix) to match
      Returns:
      a list of file paths that match the folder
      Throws:
      IOException - if an I/O exception occurs

    • getFiles

      public List<String> getFiles(String bucket, String folder) throws IOException
      Retrieves a list of files from the given bucket that match the specified folder (prefix).
      Parameters:
      bucket - the bucket to search in
      folder - the folder (prefix) to match
      Returns:
      a list of file paths that match the folder
      Throws:
      IOException - if an I/O exception occurs

    • fileExists

      public boolean fileExists(String filePath) throws IOException
      Checks if a file exists in the default bucket.
      Parameters:
      filePath - the file path
      Returns:
      true if the file exists, false otherwise
      Throws:
      IOException - if an I/O exception occurs

    • fileExists

      public boolean fileExists(String bucket, String filePath) throws IOException
      Checks if a file exists in the named S3 bucket.
      Parameters:
      bucket - the bucket to check
      filePath - the file path
      Returns:
      true if the file exists, false otherwise
      Throws:
      IOException - if an I/O exception occurs

    • isFile

      public boolean isFile(String path)
      Checks if the specified path represents a file (not a directory).
      Parameters:
      path - the path to check
      Returns:
      true if the path represents a file, false otherwise

    • getFileName

      public String getFileName(String path)
      Gets the name of the file from the specified path.
      Parameters:
      path - the file path
      Returns:
      the name of the file

    • getFileSize

      public long getFileSize(String filePath) throws IOException
      Retrieves the size of the file in the default bucket specified by the file path.
      Parameters:
      filePath - the file path
      Returns:
      the size of the file in bytes
      Throws:
      IOException - if an I/O exception occurs

    • getFileSize

      public long getFileSize(String bucket, String filePath) throws IOException
      Retrieves the size of the file in the named bucket specified by the file path.
      Parameters:
      bucket - the bucket to check
      filePath - the file path
      Returns:
      the size of the file in bytes
      Throws:
      IOException - if an I/O exception occurs

    • getFileContent

      public String getFileContent(String filePath) throws IOException
      Retrieves the content of the file in the default bucket specified by the file path.
      Parameters:
      filePath - the file path
      Returns:
      the content of the file
      Throws:
      IOException - if an I/O exception occurs

    • getFileContent

      public String getFileContent(String bucket, String filePath) throws IOException
      Retrieves the content of the file in the named bucket specified by the file path.
      Parameters:
      bucket - the bucket to retrieve the file from
      filePath - the file path
      Returns:
      the content of the file
      Throws:
      IOException - if an I/O exception occurs

    • uploadFile

      public String uploadFile(String sourceFile, String targetFileOrDir) throws IOException
      Uploads a file to the default bucket in the storage system.
      Parameters:
      sourceFile - the source file to upload
      targetFileOrDir - the target file or directory in the storage system
      Returns:
      the uploaded file path in the storage system
      Throws:
      IOException - if an I/O exception occurs if the file cannot be uploaded

    • uploadFile

      public String uploadFile(String sourceFile, String bucket, String targetFileOrDir) throws IOException
      Uploads a file to the named bucket in the storage system.
      Parameters:
      sourceFile - the source file to upload
      bucket - the bucket to upload to
      targetFileOrDir - the target file or directory in the storage system
      Returns:
      the uploaded file path in the storage system
      Throws:
      IOException - if an I/O exception occurs if the file cannot be uploaded

    • uploadFileOrDir

      public List<String> uploadFileOrDir(String sourceFileOrDir, String targetFileOrDir) throws IOException
      Uploads a file or directory to the default bucket in the storage system.
      Parameters:
      sourceFileOrDir - the source file or directory to upload
      targetFileOrDir - the target file or directory in the storage system
      Returns:
      a list of uploaded file paths in the storage system
      Throws:
      IOException - if an I/O exception occurs if the file or directory cannot be uploaded

    • uploadFileOrDir

      public List<String> uploadFileOrDir(String sourceFileOrDir, String bucket, String targetFileOrDir) throws IOException
      Uploads a file or directory to the named bucket in the storage system.
      Parameters:
      sourceFileOrDir - the source file or directory to upload
      bucket - the bucket to upload to
      targetFileOrDir - the target file or directory in the storage system
      Returns:
      a list of uploaded file paths in the storage system
      Throws:
      IOException - if an I/O exception occurs if the file or directory cannot be uploaded

    • downloadFile

      public String downloadFile(String sourceFile, String targetFileOrDir) throws IOException
      Downloads a file from the default bucket of the storage system.
      Parameters:
      sourceFile - the source file in the storage system to download
      targetFileOrDir - the target file or directory to download to
      Returns:
      the downloaded file path
      Throws:
      IOException - if an I/O exception occurs if the file cannot be downloaded

    • downloadFile

      public String downloadFile(String bucket, String sourceFile, String targetFileOrDir) throws IOException
      Downloads a file from the named bucket in the storage system.
      Parameters:
      bucket - the bucket to download from
      sourceFile - the source file in the storage system to download
      targetFileOrDir - the target file or directory to download to
      Returns:
      the downloaded file path
      Throws:
      IOException - if an I/O exception occurs if the file cannot be downloaded

    • download

      public List<String> download(String sourceFileOrDir, String targetFileOrDir) throws IOException
      Downloads a file or directory from the default bucket of the storage system.
      Parameters:
      sourceFileOrDir - the source file or directory in the storage system to download
      targetFileOrDir - the target file or directory to download to
      Returns:
      a list of downloaded file or directory paths
      Throws:
      IOException - if an I/O exception occurs if the file or directory cannot be downloaded

    • download

      public List<String> download(String bucket, String sourceFileOrDir, String targetFileOrDir) throws IOException
      Downloads a file or directory from the named bucket in the storage system.
      Parameters:
      bucket - the bucket to download from
      sourceFileOrDir - the source file or directory in the storage system to download
      targetFileOrDir - the target file or directory to download to
      Returns:
      a list of downloaded file or directory paths
      Throws:
      IOException - if an I/O exception occurs if the file or directory cannot be downloaded

    • downloadByPrefix

      public List<String> downloadByPrefix(String prefixFileOrDir) throws IOException
      Downloads a file or directory with a prefix match from the default bucket of the storage system.
      Parameters:
      prefixFileOrDir - the prefix file or directory to match
      Returns:
      a list of downloaded file paths
      Throws:
      IOException - if an I/O exception occurs if the file or directory cannot be downloaded

    • downloadByPrefix

      public List<String> downloadByPrefix(String bucket, String prefixFileOrDir) throws IOException
      Downloads a file or directory with a prefix match from the named bucket in the storage system.
      Parameters:
      bucket - the bucket to download from
      prefixFileOrDir - the prefix file or directory to match
      Returns:
      a list of downloaded file paths
      Throws:
      IOException - if an I/O exception occurs if the file or directory cannot be downloaded

    • delete

      public List<String> delete(String fileOrDir) throws IOException
      Deletes a file or directory from the default bucket.
      Parameters:
      fileOrDir - the file or directory to delete
      Returns:
      a list of deleted file paths
      Throws:
      IOException - if an I/O exception occurs

    • delete

      public List<String> delete(String bucket, String fileOrDir) throws IOException
      Deletes a file or directory from the named bucket.
      Parameters:
      bucket - the bucket to delete the file or directory from
      fileOrDir - the file or directory to delete
      Returns:
      a list of deleted file paths
      Throws:
      IOException - if an I/O exception occurs

    • deleteFile

      public String deleteFile(String filepath) throws IOException
      Deletes a file from the default bucket.
      Parameters:
      filepath - the file path
      Returns:
      the deleted file path
      Throws:
      IOException - if an I/O exception occurs

    • deleteFile

      public String deleteFile(String bucket, String filepath) throws IOException
      Deletes a file from the named bucket.
      Parameters:
      bucket - the bucket to delete the file from
      filepath - the file path
      Returns:
      the deleted file path
      Throws:
      IOException - if an I/O exception occurs

    • deleteFiles

      public List<String> deleteFiles() throws IOException
      Deletes files in the default bucket.
      Returns:
      a list of deleted file paths
      Throws:
      IOException - if an I/O exception occurs

    • deleteFiles

      public List<String> deleteFiles(String bucket) throws IOException
      Deletes files in the named bucket.
      Parameters:
      bucket - the bucket to delete files from
      Returns:
      a list of deleted file paths
      Throws:
      IOException - if an I/O exception occurs

    • deleteFiles

      public List<String> deleteFiles(List<String> toDeleteList)
      Deletes named files from the default bucket.
      Parameters:
      toDeleteList - a list of file or directory paths to delete
      Returns:
      a list of deleted file paths

    • deleteFiles

      public List<String> deleteFiles(String bucket, List<String> toDeleteList)
      Deletes named files from the given bucket.
      Parameters:
      bucket - the bucket to delete files from
      toDeleteList - a list of file or directory paths to delete
      Returns:
      a list of deleted file paths

    • setDefaultBucket

      public void setDefaultBucket(String bucket) throws IOException
      Sets the default bucket to use for operations.
      Parameters:
      bucket - the bucket name
      Throws:
      IOException - if an I/O exception occurs

    • getDefaultBucket

      public String getDefaultBucket()
      Gets the currently default bucket.
      Returns:
      the default bucket name

    • getBuckets

      public List<String> getBuckets() throws IOException
      Gets the list of available buckets.
      Returns:
      a list of bucket names
      Throws:
      IOException - if an I/O exception occurs

    • bucketExists

      public boolean bucketExists(String bucketName) throws IOException
      Checks if a bucket exists in the storage system.
      Parameters:
      bucketName - the bucket name
      Returns:
      true if the bucket exists, false otherwise
      Throws:
      IOException - if an I/O exception occurs

    • deleteBucket

      public void deleteBucket(String bucketName) throws IOException
      Deletes a bucket from the storage system.
      Parameters:
      bucketName - the bucket name
      Throws:
      IOException - if an I/O exception occurs

    • getInputStream

      public InputStream getInputStream(String relativePath) throws IOException
      Gets an input stream from a file in the default bucket of the storage system.
      Parameters:
      relativePath - the relative file path
      Returns:
      an InputStream object for reading the file content
      Throws:
      IOException - if an I/O exception occurs

    • getInputStream

      public InputStream getInputStream(String bucket, String relativePath) throws IOException
      Gets an input stream from a file in the named bucket of the storage system.
      Parameters:
      bucket - the bucket to retrieve the file from
      relativePath - the relative file path
      Returns:
      an InputStream object for reading the file content
      Throws:
      IOException - if an I/O exception occurs