Add `Credentials` to `aws-config` (#678)

vNext (Month Day, Year)

-----------------------

This release adds support for three commonly requested features:

- More powerful credential chain

- Support for constructing multiple clients from the same configuration

- Support for transcribe streaming and S3 Select

In addition, this overhauls client configuration which lead to a number of breaking changes. Detailed changes are inline.

Current Credential Provider Support:

- [x] Environment variables

- [x] Web Identity Token Credentials

- [ ] Profile file support (partial)

  - [ ] Credentials

    - [ ] SSO

    - [ ] ECS Credential source

    - [ ] IMDS credential source

    - [x] Assume role from source profile

    - [x] Static credentials source profile

    - [x] WebTokenIdentity provider

  - [ ] Region

- [ ] IMDS

- [ ] ECS

## Upgrade Guide

### If you use `<sdk>::Client::from_env`

`from_env` loaded region & credentials from environment variables _only_. Default sources have been removed from the generated

SDK clients and moved to the `aws-config` package. Note that the `aws-config` package default chain adds support for

profile file and web identity token profiles.

1. Add a dependency on `aws-config`:

     ```toml

     [dependencies]

     aws-config = { git = "https://github.com/awslabs/aws-sdk-rust", tag = "v0.0.17-alpha" }

     ```

2. Update your client creation code:

   ```rust

   // `shared_config` can be used to construct multiple different service clients!

   let shared_config = aws_config::load_from_env().await;

   // before: <service>::Client::from_env();

   let client = <service>::Client::new(&shared_config)

   ```

### If you used `<client>::Config::builder()`

`Config::build()` has been modified to _not_ fallback to a default provider. Instead, use `aws-config` to load and modify

the default chain. Note that when you switch to `aws-config`, support for profile files and web identity tokens will be added.

1. Add a dependency on `aws-config`:

     ```toml

     [dependencies]

     aws-config = { git = "https://github.com/awslabs/aws-sdk-rust", tag = "v0.0.17-alpha" }

     ```

2. Update your client creation code:

   ```rust

   fn before() {

     let region = aws_types::region::ChainProvider::first_try(<1 provider>).or_default_provider();

     let config = <service>::Config::builder().region(region).build();

     let client = <service>::Client::from_conf(&config);

   }

   async fn after() {

     use aws_config::meta::region::RegionProviderChain;

     let region_provider = RegionProviderChain::first_try(<1 provider>).or_default_provider();

     // `shared_config` can be used to construct multiple different service clients!

     let shared_config = aws_config::from_env().region(region_provider).load().await;

     let client = <service>::Client::new(&shared_config)

   }

   ```

### If you used `aws-auth-providers`

All credential providers that were in `aws-auth-providers` have been moved to `aws-config`. Unless you have a specific use case

for a specific credential provider, you should use the default provider chain:

```rust

 let shared_config = aws_config::load_from_env().await;

 let client = <service>::Client::new(&shared_config);

```

### If you maintain your own credential provider

`AsyncProvideCredentials` has been renamed to `ProvideCredentials`. The trait has been moved from `aws-auth` to `aws-types`.

The original `ProvideCredentials` trait has been removed. The return type has been changed to by a custom future.

For synchronous use cases:

```rust

use aws_types::credentials::{ProvideCredentials, future};

#[derive(Debug)]

struct CustomCreds;

impl ProvideCredentials for CustomCreds {

  fn provide_credentials<'a>(&'a self) -> future::ProvideCredentials<'a>

    where

            Self: 'a,

  {

    // if your credentials are synchronous, use `::ready`

    // if your credentials are loaded asynchronously, use `::new`

    future::ProvideCredentials::ready(todo!()) // your credentials go here

  }

}

```

For asynchronous use cases:

```rust

use aws_types::credentials::{ProvideCredentials, future, Result};

#[derive(Debug)]

struct CustomAsyncCreds;

impl CustomAsyncCreds {

  async fn load_credentials(&self) -> Result {

    Ok(Credentials::from_keys("my creds...", "secret", None))

  }

}

impl ProvideCredentials for CustomCreds {

  fn provide_credentials<'a>(&'a self) -> future::ProvideCredentials<'a>

    where

            Self: 'a,

  {

    future::ProvideCredentials::new(self.load_credentials())

  }

}

```

**Breaking Changes**

- Credential providers from `aws-auth-providers` have been moved to `aws-config` (#678)

- `AsyncProvideCredentials` has been renamed to `ProvideCredentials`. The original non-async provide credentials has been

  removed. See the migration guide above.

- `<sevicename>::from_env()` has been removed (#675). A drop-in replacement is available:

  1. Add a dependency on `aws-config`:

     ```toml

**New This Week**

- Add AssumeRoleProvider parser implementation. (#632)

- The closure passed to `async_provide_credentials_fn` can now borrow values (#637)

- The closure passed to `provide_credentials_fn` can now borrow values (#637)

- Add `Sender`/`Receiver` implementations for Event Stream (#639)

- Bring in the latest AWS models (#630)

- :tada: Add support for EBS (#567)

- :tada: Add support for Cognito (#573)

- :tada: Add support for Snowball (#579, @landonxjames)

- Make it possible to asynchronously provide credentials with `async_provide_credentials_fn` (#572, #577)

- Make it possible to asynchronously provide credentials with `provide_credentials_fn` (#572, #577)

- Improve RDS, QLDB, Polly, and KMS examples (#561, #560, #558, #556, #550)

- Update AWS SDK models (#575)

- :bug: Bugfix: Fill in message from error response even when it doesn't match the modeled case format (#565)

    "aws-sigv4"

]

exclude = ["aws-auth-providers", "aws-config"]

exclude = ["aws-config"]

[package]

name = "aws-auth-providers"

version = "0.1.0"

authors = ["AWS Rust SDK Team <aws-sdk-rust@amazon.com>", "Russell Cohen <rcoh@amazon.com>"]

edition = "2018"

[features]

rustls = ["smithy-client/rustls"]

native-tls = ["smithy-client/native-tls"]

rt-tokio = ["smithy-async/rt-tokio"]

default = ["rustls", "rt-tokio"]

[dependencies]

aws-auth = { path = "../../sdk/build/aws-sdk/aws-auth" }

aws-config = { path = "../../sdk/build/aws-sdk/aws-config"}

aws-types = { path = "../../sdk/build/aws-sdk/aws-types" }

aws-sdk-sts = { path = "../../sdk/build/aws-sdk/sts"}

aws-hyper = { path = "../../sdk/build/aws-sdk/aws-hyper"}

smithy-async = { path = "../../sdk/build/aws-sdk/smithy-async" }

tracing = "0.1"

smithy-client = { path = "../../sdk/build/aws-sdk/smithy-client" }

[dev-dependencies]

serde = { version = "1", features = ["derive"] }

serde_json = "1"

smithy-client = { path = "../../sdk/build/aws-sdk/smithy-client", features = ["test-util", "hyper-rustls"]}

tokio = { version = "1", features = ["full"]}

tracing-test = "0.1.0"

/*

 * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

 * SPDX-License-Identifier: Apache-2.0.

 */

use std::borrow::Cow;

use aws_auth::provider::env::EnvironmentVariableCredentialsProvider;

use aws_auth::provider::lazy_caching::LazyCachingCredentialsProvider;

use aws_auth::provider::BoxFuture;

use aws_auth::provider::{AsyncProvideCredentials, CredentialsResult};

use aws_hyper::DynConnector;

use aws_types::os_shim_internal::{Env, Fs};

use aws_types::region::Region;

use smithy_async::rt::sleep::AsyncSleep;

/// Default AWS Credential Provider Chain

///

/// Resolution order:

/// 1. Environment variables: [`EnvironmentVariableCredentialsProvider`](aws_auth::provider::env::EnvironmentVariableCredentialsProvider)

/// 2. Shared config (`~/.aws/config`, `~/.aws/credentials`): [`SharedConfigCredentialsProvider`](crate::profile::ProfileFileCredentialProvider)

///

/// The outer provider is wrapped in a refreshing cache.

///

/// More providers are a work in progress.

///

/// ## Example:

/// Create a default chain with a custom region:

/// ```rust

/// use aws_types::region::Region;

/// let credentials_provider = aws_auth_providers::DefaultProviderChain::builder()

///     .region(Region::new("us-west-1"))

///     .build();

/// ```

///

/// Create a default chain with no overrides:

/// ```rust

/// let credentials_provider = aws_auth_providers::default_provider();

/// ```

pub struct DefaultProviderChain(LazyCachingCredentialsProvider);

impl DefaultProviderChain {

    pub fn builder() -> Builder {

        Builder::default()

    }

}

impl AsyncProvideCredentials for DefaultProviderChain {

    fn provide_credentials<'a>(&'a self) -> BoxFuture<'a, CredentialsResult>

    where

        Self: 'a,

    {

        self.0.provide_credentials()

    }

}

/// Builder for [`DefaultProviderChain`](DefaultProviderChain)

#[derive(Default)]

pub struct Builder {

    profile_file_builder: crate::profile::Builder,

    web_identity_builder: crate::web_identity_token::Builder,

    credential_cache: aws_auth::provider::lazy_caching::builder::Builder,

    env: Option<Env>,

}

impl Builder {

    /// Set the region used when making requests to AWS services (eg. STS) as part of the provider chain

    ///

    /// When unset, the default region resolver chain will be used.

    pub fn region(mut self, region: Region) -> Self {

        self.set_region(Some(region));

        self

    }

    pub fn set_region(&mut self, region: Option<Region>) -> &mut Self {

        self.profile_file_builder.set_region(region.clone());

        self.web_identity_builder.set_region(region);

        self

    }

    /// Override the HTTPS connector used for this provider

    ///

    /// If a connector other than Hyper is used or if the Tokio/Hyper features have been disabled

    /// this method MUST be used to specify a custom connector.

    pub fn connector(mut self, connector: DynConnector) -> Self {

        self.profile_file_builder

            .set_connector(Some(connector.clone()));

        self.web_identity_builder.set_connector(Some(connector));

        self

    }

    /// Override the sleep implementation used for this provider

    ///

    /// By default, Tokio will be used to support async sleep during credentials for timeouts

    /// and reloading credentials. If the tokio default feature has been disabled, a custom

    /// sleep implementation must be provided.

    pub fn sleep(mut self, sleep: impl AsyncSleep + 'static) -> Self {

        self.credential_cache = self.credential_cache.sleep(sleep);

        self

    }

    /// Add an additional credential source for the ProfileProvider

    ///

    /// Assume role profiles may specify named credential sources:

    /// ```ini

    /// [default]

    /// role_arn = arn:aws:iam::123456789:role/RoleA

    /// credential_source = MyCustomProvider

    /// ```

    ///

    /// Typically, these are built-in providers like `Environment`, however, custom sources may

    /// also be used. Using custom sources must be registered:

    /// ```rust

    /// use aws_auth::provider::{ProvideCredentials, CredentialsError};

    /// use aws_auth::Credentials;

    /// use aws_auth_providers::DefaultProviderChain;

    /// struct MyCustomProvider;

    /// // there is a blanket implementation for `AsyncProvideCredentials` on ProvideCredentials

    /// impl ProvideCredentials for MyCustomProvider {

    ///   fn provide_credentials(&self) -> Result<Credentials, CredentialsError> {

    ///     todo!()

    ///   }

    /// }

    /// // assume role can now use `MyCustomProvider` when maed

    /// let provider_chain = DefaultProviderChain::builder()

    ///     .with_custom_credential_source("MyCustomProvider", MyCustomProvider)

    ///     .build();

    /// ```

    pub fn with_custom_credential_source(

        mut self,

        name: impl Into<Cow<'static, str>>,

        provider: impl AsyncProvideCredentials + 'static,

    ) -> Self {

        self.profile_file_builder = self

            .profile_file_builder

            .with_custom_provider(name, provider);

        self

    }

    #[doc(hidden)]

    /// Override the filesystem used for this provider

    ///

    /// This method exists primarily for testing credential providers

    pub fn fs(mut self, fs: Fs) -> Self {

        self.profile_file_builder.set_fs(fs.clone());

        self.web_identity_builder.set_fs(fs);

        self

    }

    #[doc(hidden)]

    /// Override the environment used for this provider

    ///

    /// This method exists primarily for testing credential providers

    pub fn env(mut self, env: Env) -> Self {

        self.env = Some(env.clone());

        self.profile_file_builder.set_env(env.clone());

        self.web_identity_builder.set_env(env);

        self

    }

    pub fn build(self) -> DefaultProviderChain {

        let profile_provider = self.profile_file_builder.build();

        let env_provider =

            EnvironmentVariableCredentialsProvider::new_with_env(self.env.unwrap_or_default());

        let web_identity_token_provider = self.web_identity_builder.build();

        let provider_chain = crate::chain::ChainProvider::first_try("Environment", env_provider)

            .or_else("Profile", profile_provider)

            .or_else("WebIdentityToken", web_identity_token_provider);

        let cached_provider = self.credential_cache.load(provider_chain);

        DefaultProviderChain(cached_provider.build())

    }

}

#[cfg(test)]

mod test {

    macro_rules! make_test {

        ($name: ident) => {

            #[traced_test]

            #[tokio::test]

            async fn $name() {

                crate::test_case::TestEnvironment::from_dir(concat!(

                    "./test-data/default-provider-chain/",

                    stringify!($name)

                ))

                .unwrap()

                .execute(|fs, env, conn| {

                    crate::default_provider_chain::Builder::default()

                        .env(env)

                        .fs(fs)

                        .region(Region::from_static("us-east-1"))

                        .connector(conn)

                        .build()

                })

                .await

            }

        };

    }

    use aws_sdk_sts::Region;

    use tracing_test::traced_test;

    make_test!(prefer_environment);

    make_test!(profile_static_keys);

    make_test!(web_identity_token_env);

    make_test!(web_identity_source_profile_no_env);

    make_test!(web_identity_token_invalid_jwt);

    make_test!(web_identity_token_source_profile);

    make_test!(web_identity_token_profile);

    make_test!(profile_overrides_web_identity);

    /// Helper that uses `execute_and_update` instead of execute

    ///

    /// If you run this, it will add another HTTP traffic log which re-records the request

    /// data

    #[tokio::test]

    #[ignore]

    async fn update_test() {

        crate::test_case::TestEnvironment::from_dir(concat!(

            "./test-data/default-provider-chain/web_identity_token_source_profile",

        ))

        .unwrap()

        .execute_and_update(|fs, env, conn| {

            crate::default_provider_chain::Builder::default()

                .env(env)

                .fs(fs)

                .region(Region::from_static("us-east-1"))

                .connector(conn)

                .build()

        })

        .await

    }

}

/*

 * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

 * SPDX-License-Identifier: Apache-2.0.

 */

use aws_auth::provider::AsyncProvideCredentials;

use aws_hyper::DynConnector;

pub use default_provider_chain::DefaultProviderChain;

pub mod default_provider_chain;

pub mod profile;

/// Credentials Provider that evaluates a series of providers

pub mod chain;

mod sts_util;

mod test_case;

pub mod web_identity_token;

// create a default connector given the currently enabled cargo features.

// rustls  | native tls | result

// -----------------------------

// yes     | yes        | rustls

// yes     | no         | rustls

// no      | yes        | native_tls

// no      | no         | no default

fn must_have_connector() -> DynConnector {

    default_connector().expect("A connector was not available. Either set a custom connector or enable the `rustls` and `native-tls` crate features.")

}

#[cfg(feature = "rustls")]

fn default_connector() -> Option<DynConnector> {

    Some(DynConnector::new(smithy_client::conns::https()))

}

#[cfg(all(not(feature = "rustls"), feature = "native-tls"))]

fn default_connector() -> Option<DynConnector> {

    Some(DynConnector::new(smithy_client::conns::native_tls()))

}

#[cfg(not(any(feature = "rustls", feature = "native-tls")))]

fn default_connector() -> Option<DynConnector> {

    None

}

// because this doesn't provide any configuration, a runtime and connector must be provided.

#[cfg(all(any(feature = "native-tls", feature = "rustls"), feature = "rt-tokio"))]

/// Default AWS provider chain

///

/// This provider chain will use defaults for all settings. The region will be resolved with the default

/// provider chain. To construct a custom provider, use [`default_provider_chain::Builder`](default_provider_chain::Builder).

pub async fn default_provider() -> impl AsyncProvideCredentials {

    use aws_config::meta::region::ProvideRegion;

    let resolved_region = aws_config::default_provider::region::default_provider()

        .region()

        .await;

    let mut builder = default_provider_chain::Builder::default();

    builder.set_region(resolved_region);

    builder.build()

}