Unverified Commit ab69f196 authored by Aaron Todd's avatar Aaron Todd Committed by GitHub
Browse files

add https proxy support (#4253)

## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
https://github.com/awslabs/aws-sdk-rust/issues/169

## Description
* Adds support for automatic support for common proxy env vars
(`HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, `NO_PROXY` and their
lowercase versions) behind a new `BehaviorVersion`
* relocates the TLS providers to their own modules, these are largely
the same with the biggest change being `wrap_connector` returns
dedicated connector types for `rustls` and `s2n` and the addition of new
`connect` modules containing the proxy tunnel logic for each TLS
provider.
* This results in a bit of repeated logic to handle proxy tunneling but
I couldn't find a way to unify the implementations and abstract out just
the manual TLS handshake due to type system constraints
 * updates dependency versions in `aws-smithy-http-client`
* added a new `build_with_connector_fn` that allows creating a
`SharedHttpClient` by passing your own function for creating connectors.
We didn't have a way to configure `aws_smithy_http_client::Connector`
and turn it into a `SharedHttpClient` without going through
[http_client_fn](https://github.com/smithy-lang/smithy-rs/blob/release-2025-08-04/rust-runtime/aws-smithy-runtime-api/src/client/http.rs#L102).
That works but it doesn't re-use the same caching logic for connectors
as we do when going through `aws_smithy_http_client::Builder`. The
alternative would be to make the client builder in
aws-smithy-http-client behavior version aware. Doing it the way it is in
the PR allows us to externalize this though and handle it in
`aws-smithy-runtime` when creating the [default https
client](https://github.com/smithy-lang/smithy-rs/blob/74964aac66ce6833a422d2be446e1aa8d01ba5cc/rust-runtime/aws-smithy-runtime/src/client/http.rs#L87)


## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
* integration tests with mock proxy over localhost
* tested against `mitmproxy` using different combinations of env
variables and behavior versions

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
parent e4ab0a60
Loading
Loading
Loading
Loading
+13 −0
Original line number Diff line number Diff line
---
applies_to:
- client
- aws-sdk-rust
authors:
- aajtodd
references:
- aws-sdk-rust#169
breaking: false
new_feature: true
bug_fix: false
---
Add support for proxy environment variables (`HTTP_PROXY, `HTTPS_PROXY`, `ALL_PROXY`, `NO_PROXY`). Service clients will now automatically respect these proxy environment variables on the latest `BehaviorVersion`. Older behavior versions do not automatically detect these environment variables and will require manually building a `aws_smithy_http_client::Connector` with a proxy config explicitly set to use this feature.
+4 −4
Original line number Diff line number Diff line
@@ -231,7 +231,7 @@ dependencies = [

[[package]]
name = "aws-smithy-checksums"
version = "0.63.6"
version = "0.63.7"
dependencies = [
 "aws-smithy-http",
 "aws-smithy-types",
@@ -276,7 +276,7 @@ dependencies = [

[[package]]
name = "aws-smithy-http-client"
version = "1.0.7"
version = "1.1.0"
dependencies = [
 "aws-smithy-async",
 "aws-smithy-runtime-api",
@@ -319,7 +319,7 @@ dependencies = [

[[package]]
name = "aws-smithy-runtime"
version = "1.8.6"
version = "1.9.0"
dependencies = [
 "aws-smithy-async",
 "aws-smithy-http",
@@ -341,7 +341,7 @@ dependencies = [

[[package]]
name = "aws-smithy-runtime-api"
version = "1.8.7"
version = "1.9.0"
dependencies = [
 "aws-smithy-async",
 "aws-smithy-types",
+19 −3
Original line number Diff line number Diff line
@@ -248,7 +248,7 @@ dependencies = [

[[package]]
name = "aws-smithy-http-client"
version = "1.0.6"
version = "1.1.0"
dependencies = [
 "aws-smithy-async",
 "aws-smithy-protocol-test",
@@ -273,6 +273,7 @@ dependencies = [
 "serde",
 "serde_json",
 "tokio",
 "tokio-rustls",
 "tower",
 "tracing",
]
@@ -318,7 +319,7 @@ dependencies = [

[[package]]
name = "aws-smithy-runtime"
version = "1.8.6"
version = "1.9.0"
dependencies = [
 "aws-smithy-async",
 "aws-smithy-http",
@@ -341,7 +342,7 @@ dependencies = [

[[package]]
name = "aws-smithy-runtime-api"
version = "1.8.7"
version = "1.9.0"
dependencies = [
 "aws-smithy-async",
 "aws-smithy-types",
@@ -409,6 +410,12 @@ dependencies = [
 "windows-targets 0.52.6",
]

[[package]]
name = "base64"
version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"

[[package]]
name = "base64-simd"
version = "0.8.0"
@@ -1025,6 +1032,7 @@ version = "0.1.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d9b05277c7e8da2c93a568989bb6207bef0112e8d17df7a6eda4a3cf143bc5e"
dependencies = [
 "base64",
 "bytes",
 "futures-channel",
 "futures-core",
@@ -1032,7 +1040,9 @@ dependencies = [
 "http 1.3.1",
 "http-body 1.0.1",
 "hyper 1.6.0",
 "ipnet",
 "libc",
 "percent-encoding",
 "pin-project-lite",
 "socket2 0.6.0",
 "tokio",
@@ -1169,6 +1179,12 @@ dependencies = [
 "libc",
]

[[package]]
name = "ipnet"
version = "2.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "469fb0b9cefa57e3ef31275ee7cacb78f2fdca44e4765491884a2b119d4eb130"

[[package]]
name = "itertools"
version = "0.12.1"
+14 −11
Original line number Diff line number Diff line
@@ -1258,12 +1258,13 @@ version = "0.60.3"

[[package]]
name = "aws-smithy-http-client"
version = "1.0.6"
version = "1.1.0"
dependencies = [
 "aws-smithy-async",
 "aws-smithy-protocol-test",
 "aws-smithy-runtime-api",
 "aws-smithy-types",
 "base64 0.22.1",
 "bytes",
 "h2 0.3.27",
 "h2 0.4.12",
@@ -1286,8 +1287,10 @@ dependencies = [
 "rustls-pki-types",
 "s2n-tls",
 "s2n-tls-hyper",
 "s2n-tls-tokio",
 "serde",
 "serde_json",
 "serial_test",
 "tokio",
 "tokio-rustls 0.26.2",
 "tower",
@@ -1379,7 +1382,7 @@ dependencies = [

[[package]]
name = "aws-smithy-runtime"
version = "1.8.6"
version = "1.9.0"
dependencies = [
 "approx",
 "aws-smithy-async",
@@ -1407,7 +1410,7 @@ dependencies = [

[[package]]
name = "aws-smithy-runtime-api"
version = "1.8.7"
version = "1.9.0"
dependencies = [
 "aws-smithy-async",
 "aws-smithy-types",
@@ -3862,9 +3865,9 @@ checksum = "28d3b2b1366ec20994f1fd18c3c594f05c5dd4bc44d8bb0c1c632c8d6829481f"

[[package]]
name = "s2n-tls"
version = "0.3.12"
version = "0.3.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bca784a7bd2303b31dee13fe5d8617bd27c9e983342b6f07277d7bc07a2f32ea"
checksum = "902398e2a0bd7ed40ecc61ba85f5985139213b86bb6decb547120367733c9545"
dependencies = [
 "errno",
 "hex",
@@ -3875,9 +3878,9 @@ dependencies = [

[[package]]
name = "s2n-tls-hyper"
version = "0.0.4"
version = "0.0.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "de97dea004ba2d03d6f1618f0ee770b4a63e3ba98d1764a0fecd2e520123f9ec"
checksum = "d5bfd127ef11ae746efd789549ee807f5370d5cb04e2955324420f361b2c2bbe"
dependencies = [
 "http 1.3.1",
 "hyper 1.6.0",
@@ -3889,9 +3892,9 @@ dependencies = [

[[package]]
name = "s2n-tls-sys"
version = "0.3.12"
version = "0.3.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ac0a964de7522dd815253bea5f17e70fd382920ab20c69783ad5294519f4712"
checksum = "c17994d4b67dcca7c7c01120ad30a639c1500c589d695478490e7ad6584683fd"
dependencies = [
 "aws-lc-rs",
 "cc",
@@ -3900,9 +3903,9 @@ dependencies = [

[[package]]
name = "s2n-tls-tokio"
version = "0.3.12"
version = "0.3.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ef2b3f1b676575bf221cb8fe5c0bb5720736b98957e7dc9d873cd50412cbc7a"
checksum = "84e990a681d8c701037c14d475c239803b353cdd6dcec9c8d5ba4216d4a58937"
dependencies = [
 "errno",
 "libc",
+15 −12
Original line number Diff line number Diff line
@@ -330,7 +330,7 @@ dependencies = [

[[package]]
name = "aws-smithy-checksums"
version = "0.63.6"
version = "0.63.7"
dependencies = [
 "aws-smithy-http",
 "aws-smithy-types",
@@ -424,12 +424,13 @@ version = "0.60.3"

[[package]]
name = "aws-smithy-http-client"
version = "1.0.7"
version = "1.1.0"
dependencies = [
 "aws-smithy-async",
 "aws-smithy-protocol-test",
 "aws-smithy-runtime-api",
 "aws-smithy-types",
 "base64 0.22.1",
 "bytes",
 "h2 0.3.27",
 "h2 0.4.12",
@@ -452,8 +453,10 @@ dependencies = [
 "rustls-pki-types",
 "s2n-tls",
 "s2n-tls-hyper",
 "s2n-tls-tokio",
 "serde",
 "serde_json",
 "serial_test",
 "tokio",
 "tokio-rustls 0.26.2",
 "tower 0.5.2",
@@ -614,7 +617,7 @@ dependencies = [

[[package]]
name = "aws-smithy-runtime"
version = "1.8.6"
version = "1.9.0"
dependencies = [
 "approx",
 "aws-smithy-async",
@@ -642,7 +645,7 @@ dependencies = [

[[package]]
name = "aws-smithy-runtime-api"
version = "1.8.7"
version = "1.9.0"
dependencies = [
 "aws-smithy-async",
 "aws-smithy-types",
@@ -3168,9 +3171,9 @@ checksum = "28d3b2b1366ec20994f1fd18c3c594f05c5dd4bc44d8bb0c1c632c8d6829481f"

[[package]]
name = "s2n-tls"
version = "0.3.12"
version = "0.3.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bca784a7bd2303b31dee13fe5d8617bd27c9e983342b6f07277d7bc07a2f32ea"
checksum = "902398e2a0bd7ed40ecc61ba85f5985139213b86bb6decb547120367733c9545"
dependencies = [
 "errno",
 "hex",
@@ -3181,9 +3184,9 @@ dependencies = [

[[package]]
name = "s2n-tls-hyper"
version = "0.0.4"
version = "0.0.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "de97dea004ba2d03d6f1618f0ee770b4a63e3ba98d1764a0fecd2e520123f9ec"
checksum = "d5bfd127ef11ae746efd789549ee807f5370d5cb04e2955324420f361b2c2bbe"
dependencies = [
 "http 1.3.1",
 "hyper 1.6.0",
@@ -3195,9 +3198,9 @@ dependencies = [

[[package]]
name = "s2n-tls-sys"
version = "0.3.12"
version = "0.3.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ac0a964de7522dd815253bea5f17e70fd382920ab20c69783ad5294519f4712"
checksum = "c17994d4b67dcca7c7c01120ad30a639c1500c589d695478490e7ad6584683fd"
dependencies = [
 "aws-lc-rs",
 "cc",
@@ -3206,9 +3209,9 @@ dependencies = [

[[package]]
name = "s2n-tls-tokio"
version = "0.3.12"
version = "0.3.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ef2b3f1b676575bf221cb8fe5c0bb5720736b98957e7dc9d873cd50412cbc7a"
checksum = "84e990a681d8c701037c14d475c239803b353cdd6dcec9c8d5ba4216d4a58937"
dependencies = [
 "errno",
 "libc",
Loading