Unverified Commit 0242158f authored by Russell Cohen's avatar Russell Cohen Committed by GitHub
Browse files

Add `Credentials` to `aws-config` (#678)

* Add ProvideCredentials to aws-types

* Integrate ProvideCredentials into SDK runtime and codegen

* Add CredentialsChainProvider to aws-config

* Move lazy_caching and async_provide_credentials_fn to config

* Migrate profile & web identity token providers

* Complete refactoring of aws_auth_providers

* Update examples

* Delete profile parser from aws-types and move fuzz tests

* Cleanups

* Fix some test failures

* Fix docs

* Fix select-object-content example

* convert tests, remove STS

* Fix doc comment

* remove usages of test-env-log

* take 2 fix doc comment

* Update changelog

* rename asycn_provide_credentials_fn
parent 8676d8dc
Loading
Loading
Loading
Loading
+127 −2
Original line number Diff line number Diff line
vNext (Month Day, Year)
-----------------------
This release adds support for three commonly requested features:
- More powerful credential chain
- Support for constructing multiple clients from the same configuration
- Support for transcribe streaming and S3 Select

In addition, this overhauls client configuration which lead to a number of breaking changes. Detailed changes are inline.

Current Credential Provider Support:
- [x] Environment variables
- [x] Web Identity Token Credentials
- [ ] Profile file support (partial)
  - [ ] Credentials
    - [ ] SSO
    - [ ] ECS Credential source
    - [ ] IMDS credential source
    - [x] Assume role from source profile
    - [x] Static credentials source profile
    - [x] WebTokenIdentity provider
  - [ ] Region
- [ ] IMDS
- [ ] ECS

## Upgrade Guide
### If you use `<sdk>::Client::from_env`
`from_env` loaded region & credentials from environment variables _only_. Default sources have been removed from the generated
SDK clients and moved to the `aws-config` package. Note that the `aws-config` package default chain adds support for
profile file and web identity token profiles.

1. Add a dependency on `aws-config`:
     ```toml
     [dependencies]
     aws-config = { git = "https://github.com/awslabs/aws-sdk-rust", tag = "v0.0.17-alpha" }
     ```
2. Update your client creation code:
   ```rust
   // `shared_config` can be used to construct multiple different service clients!
   let shared_config = aws_config::load_from_env().await;
   // before: <service>::Client::from_env();
   let client = <service>::Client::new(&shared_config)
   ```

### If you used `<client>::Config::builder()`
`Config::build()` has been modified to _not_ fallback to a default provider. Instead, use `aws-config` to load and modify
the default chain. Note that when you switch to `aws-config`, support for profile files and web identity tokens will be added.

1. Add a dependency on `aws-config`:
     ```toml
     [dependencies]
     aws-config = { git = "https://github.com/awslabs/aws-sdk-rust", tag = "v0.0.17-alpha" }
     ```

2. Update your client creation code:

   ```rust
   fn before() {
     let region = aws_types::region::ChainProvider::first_try(<1 provider>).or_default_provider();
     let config = <service>::Config::builder().region(region).build();
     let client = <service>::Client::from_conf(&config);
   }

   async fn after() {
     use aws_config::meta::region::RegionProviderChain;
     let region_provider = RegionProviderChain::first_try(<1 provider>).or_default_provider();
     // `shared_config` can be used to construct multiple different service clients!
     let shared_config = aws_config::from_env().region(region_provider).load().await;
     let client = <service>::Client::new(&shared_config)
   }
   ```

### If you used `aws-auth-providers`
All credential providers that were in `aws-auth-providers` have been moved to `aws-config`. Unless you have a specific use case
for a specific credential provider, you should use the default provider chain:

```rust
 let shared_config = aws_config::load_from_env().await;
 let client = <service>::Client::new(&shared_config);
```

### If you maintain your own credential provider
`AsyncProvideCredentials` has been renamed to `ProvideCredentials`. The trait has been moved from `aws-auth` to `aws-types`.
The original `ProvideCredentials` trait has been removed. The return type has been changed to by a custom future.

For synchronous use cases:
```rust
use aws_types::credentials::{ProvideCredentials, future};

#[derive(Debug)]
struct CustomCreds;
impl ProvideCredentials for CustomCreds {
  fn provide_credentials<'a>(&'a self) -> future::ProvideCredentials<'a>
    where
            Self: 'a,
  {
    // if your credentials are synchronous, use `::ready`
    // if your credentials are loaded asynchronously, use `::new`
    future::ProvideCredentials::ready(todo!()) // your credentials go here
  }
}
```

For asynchronous use cases:
```rust
use aws_types::credentials::{ProvideCredentials, future, Result};

#[derive(Debug)]
struct CustomAsyncCreds;
impl CustomAsyncCreds {
  async fn load_credentials(&self) -> Result {
    Ok(Credentials::from_keys("my creds...", "secret", None))
  }
}

impl ProvideCredentials for CustomCreds {
  fn provide_credentials<'a>(&'a self) -> future::ProvideCredentials<'a>
    where
            Self: 'a,
  {
    future::ProvideCredentials::new(self.load_credentials())
  }
}
```

**Breaking Changes**
- Credential providers from `aws-auth-providers` have been moved to `aws-config` (#678)
- `AsyncProvideCredentials` has been renamed to `ProvideCredentials`. The original non-async provide credentials has been
  removed. See the migration guide above.
- `<sevicename>::from_env()` has been removed (#675). A drop-in replacement is available:
  1. Add a dependency on `aws-config`:
     ```toml
@@ -93,7 +218,7 @@ v0.20 (August 10th, 2021)
**New This Week**

- Add AssumeRoleProvider parser implementation. (#632)
- The closure passed to `async_provide_credentials_fn` can now borrow values (#637)
- The closure passed to `provide_credentials_fn` can now borrow values (#637)
- Add `Sender`/`Receiver` implementations for Event Stream (#639)
- Bring in the latest AWS models (#630)

@@ -175,7 +300,7 @@ v0.16 (July 6th 2021)
- :tada: Add support for EBS (#567)
- :tada: Add support for Cognito (#573)
- :tada: Add support for Snowball (#579, @landonxjames)
- Make it possible to asynchronously provide credentials with `async_provide_credentials_fn` (#572, #577)
- Make it possible to asynchronously provide credentials with `provide_credentials_fn` (#572, #577)
- Improve RDS, QLDB, Polly, and KMS examples (#561, #560, #558, #556, #550)
- Update AWS SDK models (#575)
- :bug: Bugfix: Fill in message from error response even when it doesn't match the modeled case format (#565)
+1 −1
Original line number Diff line number Diff line
@@ -13,4 +13,4 @@ members = [
    "aws-sigv4"
]

exclude = ["aws-auth-providers", "aws-config"]
exclude = ["aws-config"]
+0 −28
Original line number Diff line number Diff line
[package]
name = "aws-auth-providers"
version = "0.1.0"
authors = ["AWS Rust SDK Team <aws-sdk-rust@amazon.com>", "Russell Cohen <rcoh@amazon.com>"]
edition = "2018"

[features]
rustls = ["smithy-client/rustls"]
native-tls = ["smithy-client/native-tls"]
rt-tokio = ["smithy-async/rt-tokio"]
default = ["rustls", "rt-tokio"]

[dependencies]
aws-auth = { path = "../../sdk/build/aws-sdk/aws-auth" }
aws-config = { path = "../../sdk/build/aws-sdk/aws-config"}
aws-types = { path = "../../sdk/build/aws-sdk/aws-types" }
aws-sdk-sts = { path = "../../sdk/build/aws-sdk/sts"}
aws-hyper = { path = "../../sdk/build/aws-sdk/aws-hyper"}
smithy-async = { path = "../../sdk/build/aws-sdk/smithy-async" }
tracing = "0.1"
smithy-client = { path = "../../sdk/build/aws-sdk/smithy-client" }

[dev-dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
smithy-client = { path = "../../sdk/build/aws-sdk/smithy-client", features = ["test-util", "hyper-rustls"]}
tokio = { version = "1", features = ["full"]}
tracing-test = "0.1.0"
+0 −234
Original line number Diff line number Diff line
/*
 * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
 * SPDX-License-Identifier: Apache-2.0.
 */

use std::borrow::Cow;

use aws_auth::provider::env::EnvironmentVariableCredentialsProvider;
use aws_auth::provider::lazy_caching::LazyCachingCredentialsProvider;
use aws_auth::provider::BoxFuture;
use aws_auth::provider::{AsyncProvideCredentials, CredentialsResult};
use aws_hyper::DynConnector;
use aws_types::os_shim_internal::{Env, Fs};
use aws_types::region::Region;
use smithy_async::rt::sleep::AsyncSleep;

/// Default AWS Credential Provider Chain
///
/// Resolution order:
/// 1. Environment variables: [`EnvironmentVariableCredentialsProvider`](aws_auth::provider::env::EnvironmentVariableCredentialsProvider)
/// 2. Shared config (`~/.aws/config`, `~/.aws/credentials`): [`SharedConfigCredentialsProvider`](crate::profile::ProfileFileCredentialProvider)
///
/// The outer provider is wrapped in a refreshing cache.
///
/// More providers are a work in progress.
///
/// ## Example:
/// Create a default chain with a custom region:
/// ```rust
/// use aws_types::region::Region;
/// let credentials_provider = aws_auth_providers::DefaultProviderChain::builder()
///     .region(Region::new("us-west-1"))
///     .build();
/// ```
///
/// Create a default chain with no overrides:
/// ```rust
/// let credentials_provider = aws_auth_providers::default_provider();
/// ```
pub struct DefaultProviderChain(LazyCachingCredentialsProvider);

impl DefaultProviderChain {
    pub fn builder() -> Builder {
        Builder::default()
    }
}

impl AsyncProvideCredentials for DefaultProviderChain {
    fn provide_credentials<'a>(&'a self) -> BoxFuture<'a, CredentialsResult>
    where
        Self: 'a,
    {
        self.0.provide_credentials()
    }
}

/// Builder for [`DefaultProviderChain`](DefaultProviderChain)
#[derive(Default)]
pub struct Builder {
    profile_file_builder: crate::profile::Builder,
    web_identity_builder: crate::web_identity_token::Builder,
    credential_cache: aws_auth::provider::lazy_caching::builder::Builder,
    env: Option<Env>,
}

impl Builder {
    /// Set the region used when making requests to AWS services (eg. STS) as part of the provider chain
    ///
    /// When unset, the default region resolver chain will be used.
    pub fn region(mut self, region: Region) -> Self {
        self.set_region(Some(region));
        self
    }

    pub fn set_region(&mut self, region: Option<Region>) -> &mut Self {
        self.profile_file_builder.set_region(region.clone());
        self.web_identity_builder.set_region(region);
        self
    }

    /// Override the HTTPS connector used for this provider
    ///
    /// If a connector other than Hyper is used or if the Tokio/Hyper features have been disabled
    /// this method MUST be used to specify a custom connector.
    pub fn connector(mut self, connector: DynConnector) -> Self {
        self.profile_file_builder
            .set_connector(Some(connector.clone()));
        self.web_identity_builder.set_connector(Some(connector));
        self
    }

    /// Override the sleep implementation used for this provider
    ///
    /// By default, Tokio will be used to support async sleep during credentials for timeouts
    /// and reloading credentials. If the tokio default feature has been disabled, a custom
    /// sleep implementation must be provided.
    pub fn sleep(mut self, sleep: impl AsyncSleep + 'static) -> Self {
        self.credential_cache = self.credential_cache.sleep(sleep);
        self
    }

    /// Add an additional credential source for the ProfileProvider
    ///
    /// Assume role profiles may specify named credential sources:
    /// ```ini
    /// [default]
    /// role_arn = arn:aws:iam::123456789:role/RoleA
    /// credential_source = MyCustomProvider
    /// ```
    ///
    /// Typically, these are built-in providers like `Environment`, however, custom sources may
    /// also be used. Using custom sources must be registered:
    /// ```rust
    /// use aws_auth::provider::{ProvideCredentials, CredentialsError};
    /// use aws_auth::Credentials;
    /// use aws_auth_providers::DefaultProviderChain;
    /// struct MyCustomProvider;
    /// // there is a blanket implementation for `AsyncProvideCredentials` on ProvideCredentials
    /// impl ProvideCredentials for MyCustomProvider {
    ///   fn provide_credentials(&self) -> Result<Credentials, CredentialsError> {
    ///     todo!()
    ///   }
    /// }
    /// // assume role can now use `MyCustomProvider` when maed
    /// let provider_chain = DefaultProviderChain::builder()
    ///     .with_custom_credential_source("MyCustomProvider", MyCustomProvider)
    ///     .build();
    /// ```
    pub fn with_custom_credential_source(
        mut self,
        name: impl Into<Cow<'static, str>>,
        provider: impl AsyncProvideCredentials + 'static,
    ) -> Self {
        self.profile_file_builder = self
            .profile_file_builder
            .with_custom_provider(name, provider);
        self
    }

    #[doc(hidden)]
    /// Override the filesystem used for this provider
    ///
    /// This method exists primarily for testing credential providers
    pub fn fs(mut self, fs: Fs) -> Self {
        self.profile_file_builder.set_fs(fs.clone());
        self.web_identity_builder.set_fs(fs);
        self
    }

    #[doc(hidden)]
    /// Override the environment used for this provider
    ///
    /// This method exists primarily for testing credential providers
    pub fn env(mut self, env: Env) -> Self {
        self.env = Some(env.clone());
        self.profile_file_builder.set_env(env.clone());
        self.web_identity_builder.set_env(env);
        self
    }

    pub fn build(self) -> DefaultProviderChain {
        let profile_provider = self.profile_file_builder.build();
        let env_provider =
            EnvironmentVariableCredentialsProvider::new_with_env(self.env.unwrap_or_default());
        let web_identity_token_provider = self.web_identity_builder.build();
        let provider_chain = crate::chain::ChainProvider::first_try("Environment", env_provider)
            .or_else("Profile", profile_provider)
            .or_else("WebIdentityToken", web_identity_token_provider);
        let cached_provider = self.credential_cache.load(provider_chain);
        DefaultProviderChain(cached_provider.build())
    }
}

#[cfg(test)]
mod test {

    macro_rules! make_test {
        ($name: ident) => {
            #[traced_test]
            #[tokio::test]
            async fn $name() {
                crate::test_case::TestEnvironment::from_dir(concat!(
                    "./test-data/default-provider-chain/",
                    stringify!($name)
                ))
                .unwrap()
                .execute(|fs, env, conn| {
                    crate::default_provider_chain::Builder::default()
                        .env(env)
                        .fs(fs)
                        .region(Region::from_static("us-east-1"))
                        .connector(conn)
                        .build()
                })
                .await
            }
        };
    }

    use aws_sdk_sts::Region;

    use tracing_test::traced_test;

    make_test!(prefer_environment);
    make_test!(profile_static_keys);
    make_test!(web_identity_token_env);
    make_test!(web_identity_source_profile_no_env);
    make_test!(web_identity_token_invalid_jwt);
    make_test!(web_identity_token_source_profile);
    make_test!(web_identity_token_profile);
    make_test!(profile_overrides_web_identity);

    /// Helper that uses `execute_and_update` instead of execute
    ///
    /// If you run this, it will add another HTTP traffic log which re-records the request
    /// data
    #[tokio::test]
    #[ignore]
    async fn update_test() {
        crate::test_case::TestEnvironment::from_dir(concat!(
            "./test-data/default-provider-chain/web_identity_token_source_profile",
        ))
        .unwrap()
        .execute_and_update(|fs, env, conn| {
            crate::default_provider_chain::Builder::default()
                .env(env)
                .fs(fs)
                .region(Region::from_static("us-east-1"))
                .connector(conn)
                .build()
        })
        .await
    }
}
+0 −60
Original line number Diff line number Diff line
/*
 * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
 * SPDX-License-Identifier: Apache-2.0.
 */
use aws_auth::provider::AsyncProvideCredentials;
use aws_hyper::DynConnector;

pub use default_provider_chain::DefaultProviderChain;

pub mod default_provider_chain;
pub mod profile;

/// Credentials Provider that evaluates a series of providers
pub mod chain;
mod sts_util;
mod test_case;
pub mod web_identity_token;

// create a default connector given the currently enabled cargo features.
// rustls  | native tls | result
// -----------------------------
// yes     | yes        | rustls
// yes     | no         | rustls
// no      | yes        | native_tls
// no      | no         | no default

fn must_have_connector() -> DynConnector {
    default_connector().expect("A connector was not available. Either set a custom connector or enable the `rustls` and `native-tls` crate features.")
}

#[cfg(feature = "rustls")]
fn default_connector() -> Option<DynConnector> {
    Some(DynConnector::new(smithy_client::conns::https()))
}

#[cfg(all(not(feature = "rustls"), feature = "native-tls"))]
fn default_connector() -> Option<DynConnector> {
    Some(DynConnector::new(smithy_client::conns::native_tls()))
}

#[cfg(not(any(feature = "rustls", feature = "native-tls")))]
fn default_connector() -> Option<DynConnector> {
    None
}

// because this doesn't provide any configuration, a runtime and connector must be provided.
#[cfg(all(any(feature = "native-tls", feature = "rustls"), feature = "rt-tokio"))]
/// Default AWS provider chain
///
/// This provider chain will use defaults for all settings. The region will be resolved with the default
/// provider chain. To construct a custom provider, use [`default_provider_chain::Builder`](default_provider_chain::Builder).
pub async fn default_provider() -> impl AsyncProvideCredentials {
    use aws_config::meta::region::ProvideRegion;
    let resolved_region = aws_config::default_provider::region::default_provider()
        .region()
        .await;
    let mut builder = default_provider_chain::Builder::default();
    builder.set_region(resolved_region);
    builder.build()
}
Loading