Create CredentialsCache and DefaultResolver only once in test/bench (#2620)
## Motivation and Context Addresses https://github.com/awslabs/smithy-rs/pull/2593#discussion_r1170703265 and https://github.com/awslabs/smithy-rs/pull/2593#discussion_r1170703694. Also ports the changes made in #2592. ## Description Prior to this PR, `sra_manual_test` and the bench `middleware_vs_orchestrator` created `CredentialsCache` and `aws_sdk_s3::endpoint::DefaultResolver` every time a request for `ListObjectsV2` was dispatched. This is not the case in production where those two types are created once during the construction of a service config ([here](https://github.com/awslabs/aws-sdk-rust/blob/main/sdk/s3/src/config.rs#L623-L625) and [here](https://github.com/awslabs/aws-sdk-rust/blob/main/sdk/s3/src/config.rs#L635-L652) ) and reused for subsequent request dispatches. The PR will make `sra_manual_test` and `middleware_vs_orchestrator` do the same, creating `CredentialsCache` and `aws_sdk_s3::endpoint::DefaultResolver` only once before the test/benchmark starts running and reusing them thereafter. The change will help bring the performance for `orchestrator` closer to that for `middleware`. - In the `main` branch with `DefaultEndpointResolver` commented in and `StaticUriEndpointResolver` commented out: ``` middleware time: [20.924 µs 20.943 µs 20.964 µs] change: [-1.0107% -0.7357% -0.4827%] (p = 0.00 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 1 (1.00%) high mild 6 (6.00%) high severe orchestrator time: [933.68 µs 940.11 µs 945.82 µs] change: [+2735.7% +2754.5% +2770.9%] (p = 0.00 < 0.05) Performance has regressed. Found 17 outliers among 100 measurements (17.00%) 14 (14.00%) low mild 2 (2.00%) high mild 1 (1.00%) high severe ``` - With the change in this PR: ``` middleware time: [21.161 µs 21.194 µs 21.232 µs] change: [-0.8973% -0.6397% -0.3758%] (p = 0.00 < 0.05) Change within noise threshold. Found 8 outliers among 100 measurements (8.00%) 5 (5.00%) high mild 3 (3.00%) high severe orchestrator time: [56.038 µs 56.182 µs 56.349 µs] change: [-0.7921% -0.5552% -0.3157%] (p = 0.00 < 0.05) Change within noise threshold. Found 5 outliers among 100 measurements (5.00%) 2 (2.00%) high mild 3 (3.00%) high severe ``` ## Testing Executed the following without any errors: ``` ➜ smithy-rs git:(ysaito/create-creds-cache-and-ep-resolver-only-once) ✗ ./gradlew aws:sra-test:assemble ➜ aws-sdk-s3 git:(ysaito/create-creds-cache-and-ep-resolver-only-once) cargo t ➜ aws-sdk-s3 git:(ysaito/create-creds-cache-and-ep-resolver-only-once) cargo bench ``` ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._ --------- Co-authored-by:Yuki Saito <awsaito@amazon.com>
Loading
Please register or sign in to comment