Cloud Storage: gsutil CLI, Lifecycle & Signed URLs

gsutil (Legacy CLI)

gsutil is the older Cloud Storage CLI, still widely used. The newer gcloud storage commands are faster for large transfers.

# Sync (like rsync — only uploads changed files)
gsutil -m rsync -r ./local-dir/ gs://my-bucket/prefix/
gsutil -m rsync -r -d gs://my-bucket/prefix/ ./local/  # delete files not in source

# -m flag: parallel multi-threading (much faster for many files)
gsutil -m cp -r ./images/ gs://my-bucket/images/

# Set metadata
gsutil setmeta -h "Cache-Control:public, max-age=3600" gs://my-bucket/**.js
gsutil setmeta -h "Content-Type:image/webp" gs://my-bucket/image.webp

# Set CORS
# cors.json: [{"origin": ["*"], "method": ["GET"], "maxAgeSeconds": 3600}]
gsutil cors set cors.json gs://my-bucket

# Get bucket info
gsutil ls -L -b gs://my-bucket

# Move/rename
gsutil mv gs://my-bucket/old-name.txt gs://my-bucket/new-name.txt

Lifecycle Rules

Lifecycle rules automatically transition or delete objects based on age, storage class, or other conditions.

{
  "rule": [
    {
      "action": {"type": "SetStorageClass", "storageClass": "NEARLINE"},
      "condition": {"age": 30}
    },
    {
      "action": {"type": "SetStorageClass", "storageClass": "COLDLINE"},
      "condition": {"age": 90}
    },
    {
      "action": {"type": "Delete"},
      "condition": {"age": 365}
    },
    {
      "action": {"type": "Delete"},
      "condition": {
        "numNewerVersions": 3,
        "isLive": false
      }
    }
  ]
}

# Apply lifecycle policy
gcloud storage buckets update gs://my-bucket   --lifecycle-file=lifecycle.json

# View current lifecycle
gcloud storage buckets describe gs://my-bucket --format="yaml(lifecycle)"

# Enable object versioning
gcloud storage buckets update gs://my-bucket --versioning

# List all versions
gcloud storage ls -a gs://my-bucket/

Signed URLs

Signed URLs grant temporary access to a private object without requiring a Google account. Use for letting users download/upload directly from/to Cloud Storage.

# Generate signed URL (requires service account key or signing authority)
gcloud storage sign-url gs://my-bucket/private-file.pdf   --duration=1h   --private-key-file=service-account-key.json

# Or using Python client
from google.cloud import storage
import datetime

client = storage.Client()
bucket = client.bucket("my-bucket")
blob = bucket.blob("private-file.pdf")

url = blob.generate_signed_url(
    expiration=datetime.timedelta(hours=1),
    method="GET",
    version="v4",
)
print(url)  # share this URL — valid for 1 hour

# Signed URL for upload (PUT)
upload_url = blob.generate_signed_url(
    expiration=datetime.timedelta(minutes=15),
    method="PUT",
    content_type="application/pdf",
    version="v4",
)
# Client uploads directly: PUT upload_url with file body

Best Practices

• Use uniform bucket-level access — simpler IAM management, no per-object ACL complexity.

• Enable Object Versioning for critical data — protects against accidental deletes and overwrites.

• Use lifecycle rules to auto-archive aging data to Nearline/Coldline/Archive classes.

• Prefer gcloud storage over gsutil for new scripts — parallel transfers, better performance.

• For public static websites: enable website configuration (MainPageSuffix + NotFoundPage), add allUsers viewer.

• Use Pub/Sub notifications on bucket changes to trigger Cloud Functions for event-driven processing.

• Customer-managed encryption keys (CMEK): use Cloud KMS to control encryption keys for compliance.

• Transfer Service: move large datasets from S3, Azure Blob, or HTTP/HTTPS sources efficiently.