S3
S3 Fundamentals
S3 Fundamentals Amazon S3 is an object storage service with virtually unlimited capacity. Objects are stored in buckets, identified by keys, and organized into …
S3 Fundamentals
Amazon S3 is an object storage service with virtually unlimited capacity. Objects are stored in buckets, identified by keys, and organized into storage classes. The AWS CLI makes it easy to interact with S3 programmatically.
Buckets, Objects & Keys
# Bucket names: globally unique, 3-63 chars, lowercase letters/numbers/hyphens
# Object key: the full path within the bucket (e.g. "images/2024/photo.jpg")
# URL: https://<bucket>.s3.<region>.amazonaws.com/<key>
# Create a bucket (in a specific region)
aws s3 mb s3://my-app-assets-prod --region us-east-1
aws s3 mb s3://my-app-backups --region us-west-2
# List buckets
aws s3 ls
# List objects in a bucket
aws s3 ls s3://my-app-assets-prod
aws s3 ls s3://my-app-assets-prod/images/ # List a "folder" (prefix)
aws s3 ls s3://my-app-assets-prod --recursive # All objects recursively
aws s3 ls s3://my-app-assets-prod --recursive --human-readable --summarize
# Copy objects
aws s3 cp local-file.txt s3://my-app-assets-prod/uploads/file.txt
aws s3 cp s3://my-bucket/file.txt ./local-file.txt # Download
aws s3 cp s3://src-bucket/key.txt s3://dst-bucket/key.txt # Copy between buckets
# Copy with metadata and storage class
aws s3 cp large-file.zip s3://my-bucket/ \
--storage-class STANDARD_IA \
--metadata "version=1.2,author=alice" \
--content-type "application/zip"
# Sync local directory to S3 (delta — only changed files)
aws s3 sync ./dist s3://my-app-assets-prod/ \
--delete \
--exclude ".DS_Store" \
--exclude "*.map"
# Sync S3 to local (backup)
aws s3 sync s3://my-app-backups/db/ ./backups/
# Move (copy + delete)
aws s3 mv s3://my-bucket/old-key.txt s3://my-bucket/new-key.txt
# Delete objects
aws s3 rm s3://my-bucket/file.txt
aws s3 rm s3://my-bucket/uploads/ --recursive # Delete prefix
# Remove empty bucket
aws s3 rb s3://my-bucket
# Remove non-empty bucket (force)
aws s3 rb s3://my-bucket --forceStorage Classes
# Storage class | Durability | Availability | Min duration | Use case
# ─────────────────────────────────────────────────────────────────────────────
# Standard | 11 9s | 99.99% | None | Frequently accessed data
# Standard-IA | 11 9s | 99.9% | 30 days | Infrequent access, retrieval fee
# One Zone-IA | 11 9s | 99.5% | 30 days | Non-critical, infrequent access
# Intelligent-Tier | 11 9s | 99.9% | None | Unknown access patterns (auto-moves)
# Glacier Instant | 11 9s | 99.9% | 90 days | Archives, millisecond retrieval
# Glacier Flexible | 11 9s | 99.99% | 90 days | Archives, 1min-12hr retrieval
# Glacier Deep Arc | 11 9s | 99.99% | 180 days | Long-term archive, 12-48hr retrieval
# Upload to Glacier Instant Retrieval
aws s3 cp old-data.tar.gz s3://my-archive-bucket/ \
--storage-class GLACIER_IR
# Change storage class of existing object
aws s3 cp s3://my-bucket/file.txt s3://my-bucket/file.txt \
--storage-class STANDARD_IA --metadata-directive COPY
# Check storage class of an object
aws s3api head-object \
--bucket my-bucket \
--key file.txt \
--query "StorageClass"Presigned URLs & Metadata
# Presigned URLs — temporary, signed URLs for private objects (no auth required by caller)
# Use for: client-side downloads of private files, client-side uploads directly to S3
# Generate presigned GET URL (default 1 hour, max 7 days with SigV4)
aws s3 presign s3://my-private-bucket/report.pdf --expires-in 3600
# Generate presigned PUT URL (for client-side uploads — keeps AWS credentials server-side)
aws s3 presign s3://my-uploads-bucket/user/123/avatar.jpg \
--expires-in 300 \
--region us-east-1
# Using presigned URL for upload from client (curl example)
# curl -X PUT "<presigned-put-url>" \
# -H "Content-Type: image/jpeg" \
# --upload-file avatar.jpg
# Node.js SDK presigned URL
# import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3";
# import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
# const url = await getSignedUrl(
# s3Client,
# new GetObjectCommand({ Bucket: "my-bucket", Key: "report.pdf" }),
# { expiresIn: 3600 }
# );
# Object metadata — key-value pairs stored with the object
# User-defined metadata keys must be prefixed with x-amz-meta-
aws s3api put-object \
--bucket my-bucket \
--key docs/report.pdf \
--body report.pdf \
--content-type "application/pdf" \
--metadata "author=alice,version=2.1,project=myapp"
# Retrieve metadata (HEAD request — no body download)
aws s3api head-object --bucket my-bucket --key docs/report.pdf