High-throughput transfers to and from Amazon Simple Storage Service (Amazon S3) often require manual tuning of threads, connections, and memory. Transfer Manager for Amazon S3, a high-level transfer library for Rust, handles this work for you. It is now generally available (GA).

Since the developer preview launch in March 2025, we have redesigned the library. The GA release adds a scheduler-driven execution engine, a shared buffer pool that bounds memory use, and a download pipeline that copies less data.

What’s new since developer preview

The redesign focuses on four areas: transfer speed, memory use, visibility into transfer behavior, and WebAssembly support.

Throughput defaults adapt to your instance

With the default settings, the transfer manager aims to use all available network bandwidth. At startup, it detects the instance’s cores and runs one worker thread for each core. It reuses connections across transfers and rotates them across Amazon S3 endpoints. You don’t need to configure thread pools, connection limits, or DNS caching.

A new fair scheduler coordinates all active transfers and aims to prevent large transfers from starving small ones. On instances with multiple network interfaces, you can configure the transfer manager to distribute work across all of them.

Downloads also copy less data. The transfer manager writes each part directly to its final position in memory or on disk, then atomically renames a completed file into place. For streaming consumers, the transfer manager delivers parts in order with backpressure, so the application controls how fast parts arrive.

Memory stays bounded and predictable

By default, the transfer manager sizes its memory budget to the instance, using up to a quarter of available memory (capped at 32 GiB). Every upload and download draws from one shared buffer pool, which grows as needed up to that limit. To change the limit, set a fixed byte limit or a fraction of available memory.

Runtime diagnostics help you understand transfer behavior

When throughput is lower than expected, runtime diagnostics show what is happening. Without changing your code, you can get buffer pool snapshots and traces that show how each transfer moves through its stages. To turn them on, set the AWS_S3_TM_DIAGNOSTICS environment variable. The transfer manager emits diagnostics as debug-level logs, so you also need to set RUST_LOG. Before you run the following command, complete the prerequisites in the Getting started section. Run the following command from a clone of the repository, and replace my-source and my-destination with real paths:

AWS_S3_TM_DIAGNOSTICS=memory.snapshot=1000ms,transfer.detail=1 \ RUST_LOG=aws_s3_transfer=debug \ cargo run --release --example cp -- my-source my-destination

In this command, memory.snapshot=1000ms sets the interval between memory pool snapshots, and transfer.detail=1 turns on transfer diagnostics, which summarize how each upload or download progressed through its lifecycle. This example transfers real objects, so standard Amazon S3 charges apply. To avoid ongoing storage charges, delete the copied objects from your bucket when you finish. For details, see Amazon S3 pricing.

Transfers run in WebAssembly environments

You can now build for wasm32-wasi* targets and run Amazon S3 transfers in WebAssembly-based environments.

Getting started

Before you begin, make sure that you have the following:

  • An AWS account
  • AWS credentials that from_env() can find in your environment, with s3:GetObject, s3:PutObject and s3:ListBucket permissions on your bucket
  • A Rust project that uses the Tokio runtime
  • An Amazon S3 bucket for the example transfers
  • A clone of the transfer manager repository, for running the cp example

To set up the transfer manager, complete the following steps:

  1. Add the dependency to your Cargo.toml:
    [dependencies] aws-s3-transfer = "1.0"
  2. Create a transfer manager client. from_env() loads the configuration and AWS credentials from your environment, and Client::new creates the tm client. Run this code inside an async function, such as one that is marked with #[tokio::main]:
    let config = aws_s3_transfer::from_env().load().await; let tm = aws_s3_transfer::Client::new(config);

For upload and download examples, see the developer guide.

Conclusion

This post introduced the GA release of Transfer Manager for Amazon S3 in Rust: faster default throughput, bounded memory use, runtime diagnostics, and WebAssembly support. The transfer manager handles parallel multipart uploads, byte-range downloads, and directory transfers, so your application logic can stay the focus. You can get the aws-s3-transfer crate from crates.io. Share feedback through GitHub issues. If you have questions, leave a comment on this post.

For more information, see the following resources: