Output Store - Generic S3

Describes the capabilities of the Generic S3 Storage Option within Telestream Cloud

The Generic S3 storage option for Telestream Cloud is a capability that allows interaction with Object Storage providers who implement the S3 REST API.
Amazon has established a well known/understood REST API for interaction with their Simple Storage Service (S3). Other vendors who implement this same API may be interacted with via a generic implementation of this S3 API.

622622

The image above visualizes the different implementations of the S3 protocol. The AWS Implementation corresponds to the capabilities which would be available in AWS S3 storage component present in Telestream Cloud. The implementations of the S3 protocol by other vendors is what the Generic S3 Storage capability was intended to serve.

Constraints

The Generic S3 storage capability is only intended/supported when utilized with the CloudPort service. No other Telestream Cloud services are supported at this time (despite the Generic S3 option potentially appearing as an output in the listing for those services).

Since the Generic S3 option is an opaque implementation of the S3 API/Protocol; there can be no Vendor Specific capabilities assumed by the Generic S3 storage option. In other words, capabilities unique to a specific vendor that fall outside of the S3 API/Protocol will not be accessible in the Generic S3 storage option.
For example, the Telestream Cloud AWS S3 option has the ability to use both Access Key/Secret Key or IAM settings/roles for interaction with an AWS S3 bucket. Since this is quite specific to the AWS implementation; and relies on other components in an AWS account; the ONLY credential mode supported by the Generic S3 option is the Access Key/Secret Key capability.

Likewise, more targeted Storage options (again take the AWS S3 option as an example); allow Telestream Cloud users to specify the store as an INPUT store (and make use of Watch rules). This capability relies on capabilities outside of the S3 protocol (for example, the AWS S3 storage implementation requires the use of AWS Queues and the SNS notification service); these components do not exist in any other provider; as such there is NO Input store support (or watch folder support) for stores which are defined using the Generic S3 option.

Lastly, the specification of an output store often designates where Telestream Cloud jobs shall be executed. However, since this component often refers to locations outside of supported Compute environments (or even to environments which have no indigenous compute); the use of the Generic S3 shall result in any job which references this as running in the AWS Region US-EAST-1 (unless explicitly called out). Experiments/Testing should be performed to verify where particular jobs are run to ensure that unwanted/unexpected egress costs are not incurred because the Generic S3 output store forces jobs to run in a fixed region.

Configuration

The Generic S3 option only supports the use of Access Key and Secret Key for credentials.
The following image depicts the options presented for configuring a Generic S3 output store:

791791

The 'Store Name' field corresponds to the well-known name by which this store is identified.
The Access Key and Secret Access Key are the credentials required to access the bucket.
The Server is the URL that corresponds to the HOST of the Storage you wish to associate with this new entry.
Finally, the Path entry corresponds to the Bucket Name that shall be referred to.

The configuration of the Generic S3 has been patterned after the Cyberduck application. In fact, the use of Cyberduck to validate the storage that is to be used in this capacity is highly recommended.
If the S3 storage you wish to interact with is accessible via Cyberduck; then there is a good chance it will operate via the Generic S3 storage within Telestream Cloud.

The following image illustrates accessing a S3 endpoint via Cyberduck:

600600

To configure this location as a Generic S3 Output store in Telestream Cloud, the following values would be used:

503503