AWS S3 CLI Tutorial: Upload, Sync, and Manage Buckets from the Terminal

Overview

The AWS CLI is the fastest way to work with S3. It handles bulk uploads, cross-region copies, lifecycle policies, and scripted backups. This tutorial covers installation, credentials, and the commands you will use most.

Install the AWS CLI

# macOS
brew install awscli

# Linux (v2)
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

# Windows — download the MSI from the AWS CLI page

Precompiled installers for all platforms are on the AWS CLI official page.

aws --version

Configure Credentials

aws configure

You will be prompted for:

• AWS Access Key ID
• AWS Secret Access Key
• Default region (e.g., us-east-1)
• Default output format (json, yaml, table)

Credentials are stored in ~/.aws/credentials. For production, prefer IAM roles on EC2 or SSO over long-lived access keys.

Bucket Operations

# List all buckets
aws s3 ls

# Create a bucket
aws s3 mb s3://my-unique-bucket-name --region us-east-1

# Delete an empty bucket
aws s3 rb s3://my-unique-bucket-name

# List objects in a bucket
aws s3 ls s3://my-unique-bucket-name/

# Recursive listing with sizes and dates
aws s3 ls s3://my-unique-bucket-name/ --recursive --human-readable --summarize

Uploading Files

Syncing Directories

s3 sync uploads only new or changed files. It is the right tool for backups and static site deployments.

# Upload local directory to S3
aws s3 sync ./dist s3://my-site-bucket/ --delete

# Download from S3 to local
aws s3 sync s3://my-site-bucket/ ./restore

# Preview without making changes
aws s3 sync ./dist s3://my-site-bucket/ --dryrun

The --delete flag removes files in the destination that no longer exist in the source. Use it with care.

Copying Between Buckets

aws s3 cp s3://source-bucket/data.csv s3://dest-bucket/data.csv

aws s3 sync s3://source-bucket/logs/ s3://archive-bucket/logs/ \
  --storage-class GLACIER

Storage Classes

Managing Objects

# Remove a single object
aws s3 rm s3://bucket/file.txt

# Remove a prefix recursively
aws s3 rm s3://bucket/logs/ --recursive

# Move (copy then delete)
aws s3 mv s3://bucket/a.txt s3://bucket/archive/a.txt

# Presign a temporary download URL (valid for 1 hour)
aws s3 presign s3://bucket/private.pdf --expires-in 3600

Bucket Policies and Permissions

Make a bucket public for static hosting (use with caution):

aws s3api put-bucket-policy \
  --bucket my-site-bucket \
  --policy file://public-read.json

Where public-read.json contains:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "PublicRead",
    "Effect": "Allow",
    "Principal": "*",
    "Action": "s3:GetObject",
    "Resource": "arn:aws:s3:::my-site-bucket/*"
  }]
}

Lifecycle Rules

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-bucket \
  --lifecycle-configuration file://lifecycle.json

Example rule that moves objects to Glacier after 30 days and deletes them after 365:

{
  "Rules": [{
    "ID": "archive-then-delete",
    "Status": "Enabled",
    "Filter": { "Prefix": "logs/" },
    "Transitions": [
      { "Days": 30, "StorageClass": "GLACIER" }
    ],
    "Expiration": { "Days": 365 }
  }]
}

Server-Side Encryption

aws s3 cp file.txt s3://bucket/ \
  --sse AES256

aws s3 cp file.txt s3://bucket/ \
  --sse aws:kms \
  --sse-kms-key-id alias/my-key

Performance Tips

• Use aws s3 sync rather than cp --recursive for incremental uploads.
• Enable transfer acceleration for cross-continent uploads.
• Increase concurrency with --cli-read-timeout and aws configure set max_concurrent_requests.
• For very large datasets, use the AWS DataSync service or S3 Batch Operations.

Common Pitfalls