On this page

S3 Storage

Checking access...

Amazon Simple Storage Service (S3) is a highly durable, scalable object storage service. Objects are stored in buckets and accessed via a flat namespace using unique keys.

Buckets and Objects

Buckets are containers for objects, with globally unique names across all AWS accounts. Objects consist of data (any file type) and metadata.

# Create a bucket (globally unique name required)
aws s3 mb s3://my-app-assets-2024 --region us-east-1

# Upload an object
aws s3 cp index.html s3://my-app-assets-2024/web/index.html

# List objects with metadata
aws s3api list-objects-v2 --bucket my-app-assets-2024

Tip

Bucket names must be DNS-compliant: 3-63 characters, lowercase, no underscores, starting with letter or number. Use a naming convention like {project}-{environment}-{purpose}-{region}.

Storage Classes

S3 offers multiple storage classes to optimize cost based on access patterns:

Class	Durability	Availability	Min Duration	Retrieval	Use Case
S3 Standard	99.999999999%	99.99%	None	Instant	Frequently accessed data
S3 Intelligent-Tiering	99.999999999%	99.99%	None	Instant	Unknown or changing patterns
S3 Standard-IA	99.999999999%	99.99%	30 days	Instant	Infrequent access
S3 One Zone-IA	99.999999999%	99.5%	30 days	Instant	Recreatable data
S3 Glacier Instant	99.999999999%	99.99%	90 days	Milliseconds	Long-term archive, instant access
S3 Glacier Flexible	99.999999999%	99.99%	90 days	1-5 min	Archive backups
S3 Glacier Deep Archive	99.999999999%	99.99%	180 days	12 hours	Compliance archives

Versioning

Versioning protects against accidental deletions and overwrites by preserving all object versions.

# Enable versioning
aws s3api put-bucket-versioning --bucket my-app-assets-2024 --versioning-configuration Status=Enabled

resource "aws_s3_bucket" "assets" {
  bucket = "my-app-assets-2024"
}

resource "aws_s3_bucket_versioning" "assets_versioning" {
  bucket = aws_s3_bucket.assets.id
  versioning_configuration {
    status = "Enabled"
  }
}

Danger

Once enabled, versioning cannot be disabled — only suspended. You are billed for all stored versions. Configure lifecycle policies to clean up old versions.

Lifecycle Policies

Lifecycle rules automate transitions between storage classes and expirations:

resource "aws_s3_bucket_lifecycle_configuration" "assets_lifecycle" {
  bucket = aws_s3_bucket.assets.id

  rule {
    id     = "archive-logs"
    status = "Enabled"

    filter {
      prefix = "logs/"
    }

    transition {
      days          = 30
      storage_class = "STANDARD_IA"
    }

    transition {
      days          = 90
      storage_class = "GLACIER"
    }

    expiration {
      days = 365
    }
  }

  rule {
    id     = "cleanup-old-versions"
    status = "Enabled"

    noncurrent_version_expiration {
      noncurrent_days = 90
    }
  }
}

Presigned URLs

Presigned URLs grant temporary access to private objects without requiring AWS credentials:

# Generate a presigned URL for private object (expires in 3600 seconds)
aws s3 presign s3://my-app-assets-2024/reports/q1-report.pdf --expires-in 3600

import boto3
from datetime import timedelta

s3 = boto3.client('s3')
url = s3.generate_presigned_url(
    'get_object',
    Params={'Bucket': 'my-app-assets-2024', 'Key': 'reports/q1-report.pdf'},
    ExpiresIn=3600
)
print(url)

S3 Event Notifications

S3 can send events to trigger workflows when objects are created, deleted, or restored:

resource "aws_s3_bucket_notification" "assets_notification" {
  bucket = aws_s3_bucket.assets.id

  lambda_function {
    lambda_function_arn = aws_lambda_function.process_image.arn
    events              = ["s3:ObjectCreated:*"]
    filter_prefix       = "uploads/images/"
    filter_suffix       = ".jpg"
  }

  queue {
    queue_arn     = aws_sqs_queue.resized.arn
    events        = ["s3:ObjectCreated:*"]
    filter_prefix = "resized/"
  }
}

Key Takeaways

S3 is a flat namespace — objects are stored by key (path-like string), not in directories; folder-like prefixes are a UI convention
Six storage classes optimize cost: Standard (frequent), Intelligent-Tiering (auto), Standard-IA (30d infrequent), One Zone-IA (recreatable), Glacier (archive), Glacier Deep Archive (compliance)
Versioning protects against accidental overwrites and deletes — once enabled it can only be suspended, and all versions incur storage costs
Lifecycle policies automate tier transitions and expirations using rules with prefix filters and transition/expiration actions
Presigned URLs grant temporary (1-36000s) access to private objects without exposing AWS credentials — use for downloads, uploads, and sharing
Event notifications integrate with Lambda, SQS, and SNS to trigger workflows on object creation, deletion, or restore events