How to use S3

Getty Images

Technology generates, consumes and stores a massive amount of data -- dozens of zettabytes. Data management, analysis and use are more challenging than ever.

One technology that's emerged to help manage data is object storage. Amazon S3 launched more than a decade ago and made object the standard for cloud data storage. It's a flexible, secure and cost-effective way to store, manage and use data, no matter the type or usage. But users don't have to go directly to Amazon to take advantage because several vendors offer S3-compatible storage.

Object storage is an ideal way to store data in the cloud because it's unstructured, defining data as an object with metadata and a unique identifier to locate and access each data unit. Most S3 data stores enable custom metadata for objects so users can manage them more finely.

S3 is the largest and most popular public cloud storage service, so the tech world has embraced it for interface development and applications that use data. The term S3-compatible means the data store uses the S3 API to communicate with any connected device or system. It originally referred to data stored in a public cloud; however, S3-compatible storage has extended to on-premises and private cloud deployments.

Systems, devices and applications that use the S3 API should be able to connect easily with S3-compatible storage deployments, regardless of location or use.

S3 stores data as objects within buckets, which are containers for objects. Each bucket is typically stored in a designated region, whether in a public cloud or a private cloud within a single data center facility. Data does not leave the region unless specifically moved or replicated in another region. This structure helps customers with latency, cost and compliance.

Users can typically store any number of objects in a bucket, but many S3 data stores limit the number of buckets per account. Bucket policies enable access control to the data for management of systems, applications and users. They can add or deny permissions to objects based on specific criteria, such as requester, S3 actions and IP address.

A key is the unique identifier of an object within a bucket. Each object in a bucket has one key, which is combined with the object metadata and bucket information to create a unique object identifier.

Depending on the S3 storage product, users can also enable object version IDs to preserve, retrieve and restore every version of every object stored in buckets.

Vendors offer a range of S3-compatible products and services:

Part of: Using AWS storage

There are several AWS storage types, but these four offerings cover file, block and object storage needs. Explore how the cloud storage choices compare with each other.

Amazon S3, FSx, Elastic File System and Elastic Block Store have different price structures, but data use remains the top factor in determining the cost of cloud storage.

Public bucket access is a prevalent and discussed S3 security issue. However, there are several other important security measures to take, including monitoring and MFA.

Amazon S3 is a popular platform for object storage. Through the use of APIs, though, storage vendors can offer their own take on S3, potentially improving management and cost.

Elastic Block Store and Elastic File System are two of Amazon's most popular storage types. Compare how they store data before deciding which one to use.