Sunday, 14 February 2016

How to Upload Files and Folders to Amazon S3 from PowerShell

Overview

If you need to upload files or entire folder structures to Amazon S3 from PowerShell, you no longer need to write your own recursive upload function unless you have very specific custom logic.

Today, there are two practical approaches:

  • AWS CLI v2 – the simplest and most flexible option for most developers.
  • AWS Tools for PowerShell – a good choice if you want to stay fully inside PowerShell cmdlets.

For most cases, I recommend AWS CLI v2.

Important Note About “Folders” in S3

Amazon S3 does not store folders the same way a normal file system does. S3 stores objects by key, and folder-like paths are created through key prefixes such as images/logo.png or releases/v1/app.zip.

That means you do not need to create folders manually before uploading files. If you upload an object to a key like photos/abc.png, S3 will display that path as a folder structure in the console.

Option 1. Use AWS CLI v2 (Recommended)

The modern way to upload files and folders to S3 from PowerShell is to use AWS CLI version 2.

This approach is faster, easier to maintain, and much better than building your own recursive upload script.

Install AWS CLI

Install the latest AWS CLI v2 on your Windows machine before continuing.

Configure Credentials

The old approach of hardcoding AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in scripts is no longer a good default.

Instead, use a named profile:

aws configure --profile my-s3-profile

If your organisation uses AWS IAM Identity Center (SSO), use:

aws configure sso --profile my-sso-profile
aws sso login --profile my-sso-profile

This keeps credentials out of your script and makes your setup easier to reuse.

Upload a Single File

aws s3 cp "C:\Files\logo.png" "s3://my-bucket/assets/logo.png" --profile my-s3-profile

Upload an Entire Folder

For whole directories, sync is usually better than cp --recursive because it only uploads new or changed files.

aws s3 sync "C:\MyDirectory" "s3://my-bucket/releases/v1/" --profile my-s3-profile

Upload Only Certain File Types

aws s3 sync "C:\MyDirectory" "s3://my-bucket/images/" --exclude "*" --include "*.jpg" --include "*.png" --profile my-s3-profile

Preview Before Uploading

If you want to see what will happen before making changes, use --dryrun:

aws s3 sync "C:\MyDirectory" "s3://my-bucket/releases/v1/" --dryrun --profile my-s3-profile

Mirror a Folder Exactly

If you want S3 to match your local folder exactly, add --delete:

aws s3 sync "C:\MyDirectory" "s3://my-bucket/releases/v1/" --delete --profile my-s3-profile

Be careful: --delete removes files in the target that no longer exist in the source.

Use a Profile in PowerShell

If you do not want to pass --profile on every command, set it in your current PowerShell session:

$env:AWS_PROFILE = "my-s3-profile"
aws s3 sync "C:\MyDirectory" "s3://my-bucket/releases/v1/"

Option 2. Use AWS Tools for PowerShell

If you prefer native PowerShell cmdlets instead of the AWS CLI, you can use AWS Tools for PowerShell.

Install the S3 Module

Install-Module AWS.Tools.S3 -Scope CurrentUser

Use an Existing AWS Profile

Set-AWSCredential -ProfileName my-s3-profile

Upload a Single File

Write-S3Object -BucketName "my-bucket" -Key "assets/logo.png" -File "C:\Files\logo.png"

Upload a Folder Recursively

Write-S3Object -BucketName "my-bucket" -Folder "C:\MyDirectory" -KeyPrefix "releases/v1/" -Recurse

Which Option Should You Use?

  • Use AWS CLI v2 if you want the most common and portable approach.
  • Use AWS Tools for PowerShell if you prefer PowerShell cmdlets and are already working in PowerShell-heavy automation.

Final Thoughts

If your goal is simply to upload or sync files and folders to S3, the built-in AWS tools are much better than writing your own recursive function. They are faster, safer, easier to read, and easier to maintain.

My recommendation today would be simple: use aws s3 sync for folders, use aws s3 cp for single files, and avoid storing access keys directly inside scripts unless you have a very specific reason to do so.

About the Author

Adnan Zameer, Principal Developer at Optimizely UK, is a certified Microsoft professional, specializing in web app architecture. His expertise includes Optimizely CMS and Azure, showcasing proficiency in crafting robust and efficient solutions.

0 comments :

Post a Comment