Add a middleware vs. orchestrator performance comparison benchmark (#2593)
## Motivation and Context This PR adds a benchmark suite to the `aws-sdk-s3` test in `aws/sra-test` so that the performance of client middleware can be compared against the new orchestrator approach. The comparison is not yet apples to apples since the orchestrator isn't complete enough to be comparable, but this is a benchmark that we can check regularly as they become more comparable. Initially, the performance was really bad (43x slower than middleware), but that turned out to be a trivial mistake in orchestrator implementation that was causing some configuration JSON to get reparsed for every request. Fixing that makes the orchestrator only about 1.5x slower, and there is another trivial optimization that can be made to bring that down more. The benchmark can be run with the following: ```bash smithy-rs$ ./gradlew aws:sra-test:assemble $ cd aws/sra-test/integration-tests/aws-sdk-s3 $ cargo bench ``` ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._
Loading
Please register or sign in to comment