Unverified Commit 0c5fc5d2 authored by Russell Cohen's avatar Russell Cohen Committed by GitHub
Browse files

add initial profile file provider support (#640) 

* add initial profile file provider support

This commit establishes baseline support for profile file providers. The only named credential source currently supported is `Environment` but we'll be adding more sources in the future. To support testing, the `dvr` module was added to smithy-client. There are some limitations in this which will be addressed going forward, primarily around time control so we can do request verification.

* Fix missed rename

* CR feedback: add doc comments, cleanup tests

* Improve take requests API

* Fix missed refactoring

* Add missing test data

* Fix intradoc link

* Update changelog

* resolve clippy errors
parent 13e955ac

CHANGELOG.md

+68 −38
Original line number Diff line number Diff line 
vNext (Month Day, Year)

-----------------------

**New this week**



- (When complete) Add profile file provider for region (#594, #xyz)

- Add experimental `dvr` module to smithy-client. This will enable easier testing of HTTP traffic. (#640)

- Add profile file credential provider implementation. This implementation currently does not support credential sources

  for assume role providers other than environment variables. (#640)



v0.20 (August 10th, 2021)

--------------------------



**Breaking changes**



- (#635) The `config()`, `config_mut()`, `request()`, and `request_mut()` methods on `operation::Request` have been renamed to `properties()`, `properties_mut()`, `http()`, and `http_mut()` respectively.

- (#635) The `Response` type on Tower middleware has been changed from `http::Response<SdkBody>` to `operation::Response`. The HTTP response is still available from the `operation::Response` using its `http()` and `http_mut()` methods.

- (#635) The `ParseHttpResponse` trait's `parse_unloaded()` method now takes an `operation::Response` rather than an `http::Response<SdkBody>`.

- (#626) `ParseHttpResponse` no longer has a generic argument for the body type, but instead, always uses `SdkBody`. This may cause compilation failures for you if you are using Smithy generated types to parse JSON or XML without using a client to request data from a service. The fix should be as simple as removing `<SdkBody>` in the example below:

- (#635) The `config()`, `config_mut()`, `request()`, and `request_mut()` methods on `operation::Request` have been

  renamed to `properties()`, `properties_mut()`, `http()`, and `http_mut()` respectively.

- (#635) The `Response` type on Tower middleware has been changed from `http::Response<SdkBody>`

  to `operation::Response`. The HTTP response is still available from the `operation::Response` using its `http()`

  and `http_mut()` methods.

- (#635) The `ParseHttpResponse` trait's `parse_unloaded()` method now takes an `operation::Response` rather than

  an `http::Response<SdkBody>`.

- (#626) `ParseHttpResponse` no longer has a generic argument for the body type, but instead, always uses `SdkBody`.

  This may cause compilation failures for you if you are using Smithy generated types to parse JSON or XML without using

  a client to request data from a service. The fix should be as simple as removing `<SdkBody>` in the example below:



  Before:

  ```rust
@@ -23,6 +34,7 @@ v0.20 (August 10th, 2021) 
  ```



**New This Week**



- Add AssumeRoleProvider parser implementation. (#632)

- The closure passed to `async_provide_credentials_fn` can now borrow values (#637)

- Add `Sender`/`Receiver` implementations for Event Stream (#639)
@@ -33,18 +45,19 @@ v0.19 (August 3rd, 2021) 


IoT Data Plane is now available! If you discover it isn't functioning as expected, please let us know!



This week also sees the addition of a robust async caching credentials provider.

Take a look at the

This week also sees the addition of a robust async caching credentials provider. Take a look at the

[STS example](https://github.com/awslabs/smithy-rs/blob/7fa4af4a9367aeca6d55e26fc4d4ba93093b90c4/aws/sdk/examples/sts/src/bin/credentials-provider.rs)

to see how to use it.



**New This Week**



- :tada: Add IoT Data Plane (#624)

- :tada: Add LazyCachingCredentialsProvider to aws-auth for use with expiring credentials, such as STS AssumeRole. Update STS example to use this new provider (#578, #595)

- :tada: Add LazyCachingCredentialsProvider to aws-auth for use with expiring credentials, such as STS AssumeRole.

  Update STS example to use this new provider (#578, #595)

- :bug: Correctly encode HTTP Checksums using base64 instead of hex. Fixes aws-sdk-rust#164. (#615)

- Update SDK gradle build logic to use gradle properties (#620)

- Overhaul serialization/deserialization of numeric/boolean types. This resolves issues around serialization of NaN/Infinity and should also reduce the number of allocations required during serialization. (#618)

- Overhaul serialization/deserialization of numeric/boolean types. This resolves issues around serialization of

  NaN/Infinity and should also reduce the number of allocations required during serialization. (#618)

- Update SQS example to clarify usage of FIFO vs. standard queues (#622, @trevorrobertsjr)

- Implement Event Stream frame encoding/decoding (#609, #619)
@@ -54,39 +67,38 @@ Thank you for your contributions! :heart: 


- @trevorrobertsjr (#622)





v0.18.1 (July 27th 2021)

------------------------



- Remove timestreamwrite and timestreamquery from the generated services (#613)





v0.18 (July 27th 2021)

----------------------



**Breaking changes**



- `test-util` has been made an optional dependency and has moved from

  aws-hyper to smithy-http. If you were relying on `aws_hyper::TestConnection`, add `smithy-client` as a dependency

  and enable the optional `test-util` feature. This prunes some unnecessary dependencies on `roxmltree` and `serde_json`

- `test-util` has been made an optional dependency and has moved from aws-hyper to smithy-http. If you were relying

  on `aws_hyper::TestConnection`, add `smithy-client` as a dependency and enable the optional `test-util` feature. This

  prunes some unnecessary dependencies on `roxmltree` and `serde_json`

  for most users. (#608)



**New This Week**



- :tada: Release all but three remaining AWS services! Glacier, IoT Data Plane and Transcribe streaming will be available in a future release. If you discover that a service isn't functioning as expected please let us know! (#607)

- :tada: Release all but three remaining AWS services! Glacier, IoT Data Plane and Transcribe streaming will be

  available in a future release. If you discover that a service isn't functioning as expected please let us know! (#607)

- :bug: Bugfix: Fix parsing bug where parsing XML incorrectly stripped whitespace (#590, aws-sdk-rust#153)

- Establish common abstraction for environment variables (#594)

- Add windows to the test matrix (#594)

- :bug: Bugfix: Constrain RFC-3339 timestamp formatting to microsecond precision (#596)





v0.17 (July 15th 2021)

----------------------



**New this Week**



- :tada: Add support for Autoscaling (#576, #582)

- `AsyncProvideCredentials` now introduces an additional lifetime parameter, simplifying bridging it with `#[async_trait]` interfaces

- `AsyncProvideCredentials` now introduces an additional lifetime parameter, simplifying bridging it

  with `#[async_trait]` interfaces

- Fix S3 bug when content type was set explicitly (aws-sdk-rust#131, #566, @eagletmt)



**Contributions**
@@ -95,13 +107,13 @@ Thank you for your contributions! :heart: 


- @eagletmt (#566)





v0.16 (July 6th 2021)

---------------------



**New this Week**



- :warning: **Breaking Change:** `ProvideCredentials` and `CredentialError` were both moved into `aws_auth::provider` when they were previously in `aws_auth` (#572)

- :warning: **Breaking Change:** `ProvideCredentials` and `CredentialError` were both moved into `aws_auth::provider`

  when they were previously in `aws_auth` (#572)

- :tada: Add support for AWS Config (#570)

- :tada: Add support for EBS (#567)

- :tada: Add support for Cognito (#573)
@@ -123,12 +135,11 @@ Thank you for your contributions! :heart: 


- landonxjames (#579)





v0.15 (June 29th 2021)

----------------------



This week, we've added EKS, ECR and Cloudwatch. The JSON deserialization implementation has been replaced, please be

on the lookout for potential issues.

This week, we've added EKS, ECR and Cloudwatch. The JSON deserialization implementation has been replaced, please be on

the lookout for potential issues.



**New this Week**
@@ -136,12 +147,14 @@ on the lookout for potential issues. 
- :tada: Add support for Cloudwatch (#554)

- :tada: Add support for EKS (#553)

- :warn: **Breaking Change:** httpLabel no longer causes fields to be non-optional. (#537)

- :warn: **Breaking Change:** `Exception` is not renamed to `Error`. Code may need to be updated to replace `exception` with `error`

- :warn: **Breaking Change:** `Exception` is not renamed to `Error`. Code may need to be updated to replace `exception`

  with `error`

- Add more SES examples, and improve examples for Batch.

- Improved error handling ergonomics: Errors now provide `is_<variantname>()` methods to simplify error handling

- :bug: Bugfix: fix bug where invalid query strings could be generated (#531, @eagletmt)



**Internal Changes**



- Pin CI version to 1.52.1 (#532)

- New JSON deserializer implementation (#530)

- Fix numerous namespace collision bugs (#539)
@@ -153,27 +166,31 @@ Thank you for your contributions! :heart: 


- @eagletmt (#531)





v0.14 (June 22nd 2021)

----------------------



This week, we've added CloudWatch Logs support and fixed several bugs in the generated S3 clients.

There are a few breaking changes this week.

This week, we've added CloudWatch Logs support and fixed several bugs in the generated S3 clients. There are a few

breaking changes this week.



**New this Week**



- :tada: Add support for CloudWatch Logs (#526)

- :warning: **Breaking Change:** The `set_*` functions on generated Builders now always take an `Option` (#506)

- :warning: **Breaking Change:** Unions with Documents will see the inner document type change from `Option<Document>` to `Document` (#520)

- :warning: **Breaking Change:** The `as_*` functions on unions now return `Result` rather than `Option` to clearly indicate what the actual value is (#527)

- Add more S3 examples, and improve SNS, SQS, and SageMaker examples. Improve example doc comments (#490, #508, #509, #510, #511, #512, #513, #524)

- :warning: **Breaking Change:** Unions with Documents will see the inner document type change from `Option<Document>`

  to `Document` (#520)

- :warning: **Breaking Change:** The `as_*` functions on unions now return `Result` rather than `Option` to clearly

  indicate what the actual value is (#527)

- Add more S3 examples, and improve SNS, SQS, and SageMaker examples. Improve example doc comments (#490, #508, #509,

  #510, #511, #512, #513, #524)

- :bug: Bugfix: Show response body in trace logs for calls that don't return a stream (#514)

- :bug: Bugfix: Correctly parse S3's GetBucketLocation response (#516)

- :bug: Bugfix: Correctly URL-encode tilde characters before SigV4 signing (#519)

- :bug: Bugfix: Fix S3 PutBucketLifecycle operation by adding support for the `@httpChecksumRequired` Smithy trait (#523)

- :bug: Bugfix: Fix S3 PutBucketLifecycle operation by adding support for the `@httpChecksumRequired` Smithy trait (

  #523)

- :bug: Bugfix: Correctly parse non-list headers with commas in them (#525, @eagletmt)



**Internal Changes**



- Reduce name collisions in generated code (#502)

- Combine individual example packages into per-service example packages with multiple binaries (#481, #490)

- Re-export HyperAdapter in smithy-client (#515, @zekisherif)
@@ -189,7 +206,8 @@ Thank you for your contributions! :heart: 
v0.13 (June 15th 2021)

----------------------



Smithy-rs now has codegen support for all AWS services! This week, we've added CloudFormation, SageMaker, EC2, and SES. More details below.

Smithy-rs now has codegen support for all AWS services! This week, we've added CloudFormation, SageMaker, EC2, and SES.

More details below.



**New this Week**
@@ -204,11 +222,13 @@ Smithy-rs now has codegen support for all AWS services! This week, we've added C 


**Internal Changes**



- Combine individual example packages into per-service example packages with multiple binaries (#477, #480, #482, #484, #485, #486, #487, #491)

- Combine individual example packages into per-service example packages with multiple binaries (#477, #480, #482, #484,

  #485, #486, #487, #491)

- Work towards JSON deserialization overhaul (#474)

- Make deserializer function naming consistent between XML and JSON deserializers (#497)



Contributors:



- @Doug-AWS

- @jdisanti

- @rcoh
@@ -220,18 +240,22 @@ Thanks!! 
v0.12 (June 8th 2021)

---------------------



Starting this week, smithy-rs now has codegen support for all AWS services except EC2. This week we’ve added MediaLive, MediaPackage, SNS, Batch, STS, RDS, RDSData, Route53, and IAM. More details below.

Starting this week, smithy-rs now has codegen support for all AWS services except EC2. This week we’ve added MediaLive,

MediaPackage, SNS, Batch, STS, RDS, RDSData, Route53, and IAM. More details below.



**New this Week**



- :tada: Add support for MediaLive and MediaPackage (#449, @alastaim)

- :tada: Add support for SNS (#450)

- :tada: Add support for Batch (#452, @alistaim)

- :tada: Add support for STS. **Note:** This does not include support for an STS-based credential provider although an example is provided. (#453)

- :tada: Add support for STS. **Note:** This does not include support for an STS-based credential provider although an

  example is provided. (#453)

- :tada: Add support for RDS (#455) and RDS-Data (#470). (@LMJW)

- :tada: Add support for Route53 (#457, @alistaim)

- Support AWS Endpoints & Regions. With this update, regions like `iam-fips` and `cn-north-1` will now resolve to the correct endpoint. Please report any issues with endpoint resolution. (#468)

- :bug: Bugfix: Primitive numerics and booleans are now filtered from serialization when they are 0 and not marked as required. This resolves issues where maxResults needed to be set even though it is optional. (#451)

- Support AWS Endpoints & Regions. With this update, regions like `iam-fips` and `cn-north-1` will now resolve to the

  correct endpoint. Please report any issues with endpoint resolution. (#468)

- :bug: Bugfix: Primitive numerics and booleans are now filtered from serialization when they are 0 and not marked as

  required. This resolves issues where maxResults needed to be set even though it is optional. (#451)

- :bug: Bugfix: S3 Head Object returned the wrong error when the object did not exist (#460, fixes #456)



**Internal Changes**
@@ -242,6 +266,7 @@ Starting this week, smithy-rs now has codegen support for all AWS services excep 
- Work towards JSON deserialization overhaul (#454, #462)



Contributors:



- @rcoh

- @jdisanti

- @alistaim
@@ -254,15 +279,19 @@ v0.11 (June 1st, 2021) 


**New this week:**



- :tada: Add support for SQS. SQS is our first service to use the awsQuery protocol. Please report any issues you may encounter.

- :tada: Add support for SQS. SQS is our first service to use the awsQuery protocol. Please report any issues you may

  encounter.

- :tada: Add support for ECS.

- **Breaking Change**: Refactored `smithy_types::Error` to be more flexible. Internal fields of `Error` are now private and can now be accessed accessor functions. (#426)

- **Breaking Change**: Refactored `smithy_types::Error` to be more flexible. Internal fields of `Error` are now private

  and can now be accessed accessor functions. (#426)

- `ByteStream::from_path` now accepts `implications AsRef<Path>` (@LMJW)

- Add support for S3 extended request id (#429)

- Add support for the awsQuery protocol. smithy-rs can now add support for all services except EC2.

- **Bugfix**: Timestamps that fell precisely on minute boundaries were not properly formatted (#435)

- Improve documentation for `ByteStream` & add `pub use` (#443)

- Add support for `EndpointPrefix` used by [`s3::WriteGetObjectResponse`](https://awslabs.github.io/aws-sdk-rust/aws_sdk_s3/operation/struct.WriteGetObjectResponse.html) (#420)

- Add support for `EndpointPrefix` used

  by [`s3::WriteGetObjectResponse`](https://awslabs.github.io/aws-sdk-rust/aws_sdk_s3/operation/struct.WriteGetObjectResponse.html) (

  #420)



**Smithy Internals**
@@ -271,6 +300,7 @@ v0.11 (June 1st, 2021) 
- **Bugfix:** Idempotency tokens were not properly generated when operations were used by resources



Contributors:



- @jdisanti

- @rcoh

- @LMJW

aws/rust-runtime/aws-auth-providers/Cargo.toml

+3 −0
Original line number Diff line number Diff line
@@ -14,3 +14,6 @@ tracing = "0.1" 
[dev-dependencies]

serde = { version = "1", features = ["derive"] }

serde_json = "1"

smithy-client = { path = "../../sdk/build/aws-sdk/smithy-client", features = ["test-util", "hyper-rustls"]}

tokio = { version = "1", features = ["full"]}

tracing-test = "0.1.0"

aws/rust-runtime/aws-auth-providers/src/lib.rs

+0 −1
Original line number Diff line number Diff line
@@ -2,5 +2,4 @@ 
 * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

 * SPDX-License-Identifier: Apache-2.0.

 */



pub mod profile;

aws/rust-runtime/aws-auth-providers/src/profile.rs

+402 −5

File changed.

Preview size limit exceeded, changes collapsed.

aws/rust-runtime/aws-auth-providers/src/profile/exec.rs

+184 −1
Original line number Diff line number Diff line
@@ -3,4 +3,187 @@ 
 * SPDX-License-Identifier: Apache-2.0.

 */



//! Future home of the execution side of Profile Provider execution
 
use std::sync::Arc;



use aws_auth::provider::{AsyncProvideCredentials, CredentialsError, CredentialsResult};

use aws_auth::Credentials;

use aws_hyper::StandardClient;

use aws_sdk_sts::operation::AssumeRole;

use aws_sdk_sts::Config;

use aws_types::region::Region;



use crate::profile::repr::BaseProvider;

use crate::profile::ProfileFileError;



use super::repr;

use std::fmt::{Debug, Formatter};



#[derive(Debug)]

pub struct AssumeRoleProvider {

    role_arn: String,

    external_id: Option<String>,

    session_name: Option<String>,

}



pub struct ClientConfiguration {

    pub core_client: StandardClient,

    pub region: Option<Region>,

}



impl AssumeRoleProvider {

    pub async fn credentials(

        &self,

        input_credentials: Credentials,

        client_config: &ClientConfiguration,

    ) -> CredentialsResult {

        let config = Config::builder()

            .credentials_provider(input_credentials)

            .region(client_config.region.clone())

            .build();

        let operation = AssumeRole::builder()

            .role_arn(&self.role_arn)

            .set_external_id(self.external_id.clone())

            .role_session_name(

                self.session_name

                    .as_deref()

                    .unwrap_or("assume-role-provider-session"),

            )

            .build()

            .expect("operation is valid")

            .make_operation(&config)

            .expect("valid operation");

        let assume_role_creds = client_config

            .core_client

            .call(operation)

            .await

            .map_err(|err| CredentialsError::Unhandled(err.into()))?

            .credentials

            .ok_or_else(|| {

                CredentialsError::Unhandled(

                    "assume role provider did not return credentials".into(),

                )

            })?;

        let expiration = assume_role_creds

            .expiration

            .ok_or_else(|| CredentialsError::Unhandled("missing expiration".into()))?;

        let expiration = expiration.to_system_time().ok_or_else(|| {

            CredentialsError::Unhandled(

                format!("expiration is before unix epoch: {:?}", &expiration).into(),

            )

        })?;

        Ok(Credentials::new(

            assume_role_creds.access_key_id.ok_or_else(|| {

                CredentialsError::Unhandled("access key id missing from result".into())

            })?,

            assume_role_creds

                .secret_access_key

                .ok_or_else(|| CredentialsError::Unhandled("secret access token missing".into()))?,

            assume_role_creds.session_token,

            Some(expiration),

            "AssumeRoleProvider",

        ))

    }

}



pub(crate) struct ProviderChain {

    base: Arc<dyn AsyncProvideCredentials>,

    chain: Vec<AssumeRoleProvider>,

}



impl Debug for ProviderChain {

    fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {

        // TODO: AsyncProvideCredentials should probably mandate debug

        f.debug_struct("ProviderChain").finish()

    }

}



impl ProviderChain {

    pub fn base(&self) -> &dyn AsyncProvideCredentials {

        self.base.as_ref()

    }



    pub fn chain(&self) -> &[AssumeRoleProvider] {

        &self.chain.as_slice()

    }

}



impl ProviderChain {

    pub fn from_repr(

        repr: repr::ProfileChain,

        factory: &named::NamedProviderFactory,

    ) -> Result<Self, ProfileFileError> {

        let base = match repr.base() {

            BaseProvider::NamedSource(name) => {

                factory

                    .provider(name)

                    .ok_or(ProfileFileError::UnknownProvider {

                        name: name.to_string(),

                    })?

            }

            BaseProvider::AccessKey(key) => Arc::new(key.clone()),

        };

        tracing::info!(base = ?repr.base(), "first credentials will be loaded from {:?}", repr.base());

        let chain = repr

            .chain()

            .iter()

            .map(|role_arn| {

                tracing::info!(role_arn = ?role_arn, "which will be used to assume a role");

                AssumeRoleProvider {

                    role_arn: role_arn.role_arn.into(),

                    external_id: role_arn.external_id.map(|id| id.into()),

                    session_name: role_arn.session_name.map(|id| id.into()),

                }

            })

            .collect();

        Ok(ProviderChain { base, chain })

    }

}



pub mod named {

    use std::collections::HashMap;

    use std::sync::Arc;



    use aws_auth::provider::AsyncProvideCredentials;



    pub struct NamedProviderFactory {

        providers: HashMap<String, Arc<dyn AsyncProvideCredentials>>,

    }



    impl NamedProviderFactory {

        pub fn new(providers: HashMap<String, Arc<dyn AsyncProvideCredentials>>) -> Self {

            Self { providers }

        }



        pub fn provider(&self, name: &str) -> Option<Arc<dyn AsyncProvideCredentials>> {

            self.providers.get(name).cloned()

        }

    }

}



#[cfg(test)]

mod test {

    use crate::profile::exec::named::NamedProviderFactory;

    use crate::profile::exec::ProviderChain;

    use crate::profile::repr::{BaseProvider, ProfileChain};

    use std::collections::HashMap;



    #[test]

    fn error_on_unknown_provider() {

        let factory = NamedProviderFactory::new(HashMap::new());

        let chain = ProviderChain::from_repr(

            ProfileChain {

                base: BaseProvider::NamedSource("floozle"),

                chain: vec![],

            },

            &factory,

        );

        let err = chain.expect_err("no source by that name");

        assert!(

            format!("{}", err).contains(

                "profile referenced `floozle` provider but that provider is not supported"

            ),

            "`{}` did not match expected error",

            err

        );

    }

}