Amazon S3 Buckets (Simple Storage Service) are used to store objects and flat files in the Cloud.
There is unlimited storage available, across 100 buckets, and files can be from 0 bytes to 5TB.
Amazon S3 is one of the oldest services AWS offers and is incredibly flexible with multiple ways to use it.
Analytics / Data Lake
Temporary data storage before being loading into AWS Redshift.
How data is stored
Each S3 bucket needs a unique name and is formatted as:
Each object consists of:
- Key (the name of the object),
- Value (the data in the file itself made of bytes),
Amazon S3 provides read after write consistently and eventual consistency for updates and deletes. Data is being replicated across multiple data centres and may take time to flow through.
Storage Class Options
- The most expensive but most durable and reliable option for ‘hot’ data.
- Cloud apps, big data analytics, websites, content distribution.
S3: Infrequent Access
- For storing non-critical data that CANNOT be easily reproduced and needs to be retrieved quickly.
- Disaster recovery, backups.
S3: Infrequent Access – One Zone
- For storing non-critical data that CAN be easily reproduced and needs to be retrieved quickly.
- Secondary backups as this will only be stored in one zone.
- For long-term storage with a 3 – 5 hour retrieval time for ‘cold’ data.
Deep Glacier (NEW)
- For long-term storage with a 12 hour retrieval time for ‘cold’ data.
- Documents that need to be kept for compliance reasons for 7+ years.
- Data is encrypted by the client and uploaded to Amazon S3 already encrypted.
- Encrypts as the data is written and decrypts when it is being used.
- Versioning allows for older copies of a file to be seen, and “deleted” files to be restored.
- Deleted files have a delete tag added which hides the file. To restore the file, delete the tag.
- Each version takes up storage space, so a 1GB file edited three times with versioning on takes up 3GB of space.
- Once turned on versioning can only be suspended, not removed.
- Versions that are deleted on the other hand are actually deleted. Enabling Versioning MFA Delete gives extra protection as it requires MFA before a version can be deleted.
- Cross-Region Replication lets you automatically replicate the contents of a bucket from one region to another.
- Existing files won’t be copied until there’s been a new version, which will also replicate all previous versions and permission.
Get started with Amazon S3 with the Free Tier. It offers 12 months of free storage:
5 GB of Standard Storage
20,000 Get Requests
2,000 Put Requests