Home

Support for SSE-C

This fork adds support for Server Side Encryption with a Customer provided key (SSE-C). See the Amazon SSE-C documentation regarding this new feature.

Two new options are added to the command line. The man page has been updated.

• --sse-customer-key This is a 32 character string with defines the AES256 key
• --sse-copy-source-customer-key Used for copy or move operations

The --configure option will now ask for a an SSE-C key and will save it in .s3cfg.

Note: Adding sse_customer_key to the .s3cfg file means all operations will use SSE-C. If you want to do some operations without SSE-C, do not add the key to .s3cfg. Use the --sse-customer-key command line option to specify a key when required.

Caveats when using SSE-C

• For put and sync operations MD5 checking is automatically disabled because the returned ETag does not contain an MD5 checksum of the uploaded file as it usually does. Instead, a check is made that the returned MD5 of the customer key matches the one that was used. If it doesn't match a retry is attempted.
• Even though MD5 checking is removed from sync_checks, the md5 option is retained in preserve_attrs_list. A checksum is calculated for each uploaded file and included in s3cmd-attrs.
• The get operation works as before. The s3cmd-attrs metadata includes the MD5 checksum and this is compared against the file that is downloaded.
• Because the ETags differ, local to remote sync operations cannot use MD5 to detect changed files. s3cmd still tests file size and I've added an additional test used only during SSE-C sync which compares the local file mtime against the S3 timestamp (upload time). Files are flagged for upload if the mtime is after the last upload.
• Remote to local sync always downloads all files. This appears to be the way it works before my changes.
• Remote to remote sync is untested.

Some observations about metadata.

When examining objects using the S3 Management Console or other S3 browsing tools, the metadata of SSE-C objects is not visible. When the object is downloaded via the GET API with appropriate SSE-C headers, its correct metadata is included. SSE-C objects cannot be downloaded through the Management Console or via a static website. You must use the GET API and provide the correct AES256 key or you get an error.

You cannot copy (cp) an SSE-C object unless you provide the key used to encrypt it via --sse-copy-source-customer-key option. The copy destination object will be unencrypted unless you provide a key with --sse-customer-key.

It appears that copies of SSE-C objects lose their metadata, whether the copy target is encrypted or not. This includes the s3cmd-attrs metadata, so file attributes and MD5 checksum is lost.

Examples

The AES256 key for encryption is specified via a 32 character string (256 bits).

s3cmd --sse-customer-key=12345678901234567890123456789012 sync mydir s3://mybucket.example.com/
s3cmd --sse-customer-key=12345678901234567890123456789012 get s3://mybucket.example.com/mydir/myfile
s3cmd --sse-copy-source-customer-key=12345678901234567890123456789012 cp s3://mybucket.example.com/mydir/myfile s3://myotherbucket.example.com/myfile

In last example the copy will not be encrypted unless you also specify --sse-customer-key.

The key can be placed in the .s3cfg file (in the sse_customer_key parameter) so its not necessary to include it on the command line. The copy-source key can only be specified on the command line.