File Storage

CoreWallet provides generic file storage that can be used for various processes, such as generated reports, user uploads, system files, etc.

Extension point

CoreWallet uses the interface com.trimplement.wallet.server.common.file.transmission.FileTransmissionService for all internal file handling. It contains methods for storing, retrieving and deleting files.

The file content is streamed during storing and retrieving, so that big files can be processed without loading the full file content into memory. This content streaming is even supported over HttpInvoker, when a traditional 2-layer deployment is used.

It currently has 2 implementations: an AWS S3 compatible implementation, and a local file system implementation.

AWS S3 implementation

The AWS S3 implementation for FileTransmissionService allows to store file content in an S3 bucket.

It is strongly recommended to store files only in private S3 buckets. In addition, each file is AES256-encrypted with a file-specific key. Each file-specific key is AES256-encrypted by a master-key, and stored as a header with each file. The master-key is provided as a configuration property to the application, and can be versioned.

During write the SHA256 checksum is calculated of the content, so the caller can store it for integrity validation during subsequent reads.

Keep in mind that AWS S3 bucket names must be globally unique: https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html 

Testing with MinIO

For local testing of the AWS S3 implementation the compatible implementation MinIO can be used, both for tests and local deployments.

File system implementation

The file system implementation for FileTransmissionService allows to store files on the local file system.

The file content is not encrypted.

During write the SHA256 checksum is calculated of the content, so the caller can store it for integrity validation during subsequent reads.

The file-location and file-name must be compatible with AWS S3 naming conventions to allow for easy replacement with an AWS S3 implementation.