Add middleware & update design docs (#175)

# Summary

- [Http Operations](./operation.md)

- [HTTP middleware](./middleware.md)

# HTTP middleware

Signing, endpoint specification, and logging are all handled as middleware. The Rust SDK takes a minimalist approach to middleware:

Middleware is defined as minimally as possible, then adapted into the middleware system used by the IO layer. Tower is the de facto standard for HTTP middleware in Rust—we will probably use it. But we also want to make our middleware usable for users who aren't using Tower (or if we decide to not use Tower in the long run).

Because of this, rather than implementing all our middleware as "Tower Middleware", we implement it narrowly (eg. as a function that operates on `operation::Request`), then define optional adapters to make our middleware tower compatible.

an `Operation<Output>`. This codifies the base HTTP request & all the configuration and middleware layers required to modify and dispatch the request.

```rust,ignore

pub struct Operation<H> {

pub struct Operation<H, R> {

    request: Request,

    response_handler: Box<H>,

    response_handler: H,

    _retry_policy: R,

}

pub struct Request {

    let op = BatchExecuteStatement::new(BatchExecuteStatementInput {

        statements: self.statements,

    });

    let mut request = operation::Request::new(

        op.build_http_request()

            .map(|body| operation::SdkBody::from(body)),

    );

    use operation::signing_middleware::SigningConfigExt;

    request

        .config

        .insert_signingconfig(SigningConfig::default_config(

            auth::ServiceConfig {

                service: config.signing_service().into(),

                region: config.region.clone().into(),

            },

            auth::RequestConfig {

                request_ts: || std::time::SystemTime::now(),

            },

        ));

    use operation::signing_middleware::CredentialProviderExt;

    request

        .config

        .insert_credentials_provider(config.credentials_provider.clone());

    use operation::endpoint::EndpointProviderExt;

    request

        .config

        .insert_endpoint_provider(config.endpoint_provider.clone());

    Operation::new(request, op)

    let req = op.build_http_request().map(SdkBody::from);

    let mut req = operation::Request::new(req);

    let mut conf = req.config_mut();

    conf.insert_signing_config(config.signing_service());

    conf.insert_endpoint_provider(config.endpoint_provider.clone());

    Operation::new(req)

}

```

### Operation Dispatch and Middleware

The Rust SDK endeavors to behave as predictably as possible. This means that if at all possible we will not dispatch extra HTTP requests during the dispatch of normal operation. Making this work is covered in more detail in the design of credentials providers & endpoint resolution.

The upshot is that we will always prefer a design where the user has explicit control of when credentials are loaded and endpoints are resolved. This doesn't mean that users can't use easy-to-use options (We will provide an automatically refreshing credentials provider), however, the credential provider won't load requests during the dispatch of an individual request.

## Operation Parsing and Response Loading

The fundamental trait for HTTP-based protocols is `ParseHttpResponse`

pub mod base64;

pub mod body;

pub mod label;

pub mod middleware;

pub mod operation;

mod pin_util;

pub mod property_bag;

pub mod query;

pub mod response;

pub mod result;

/*

 * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

 * SPDX-License-Identifier: Apache-2.0.

 */

//! This modules defines the core, framework agnostic, HTTP middleware interface

//! used by the SDK

//!

//! smithy-middleware-tower provides Tower-specific middleware utilities (todo)

use crate::operation;

use crate::pin_mut;

use crate::response::ParseHttpResponse;

use crate::result::{SdkError, SdkSuccess};

use bytes::{Buf, Bytes};

use http_body::Body;

use std::error::Error;

type BoxError = Box<dyn Error + Send + Sync>;

/// [`MapRequest`] defines a synchronous middleware that transforms an [`operation::Request`].

///

/// Typically, these middleware will read configuration from the `PropertyBag` and use it to

/// augment the request. Most fundamental middleware is expressed as `MapRequest`, including

/// signing & endpoint resolution.

///

/// ```rust

/// # use smithy_http::middleware::MapRequest;

/// # use std::convert::Infallible;

/// # use smithy_http::operation;

/// use http::header::{HeaderName, HeaderValue};

/// struct AddHeader(HeaderName, HeaderValue);

/// /// Signaling struct added to the request property bag if a header should be added

/// struct NeedsHeader;

/// impl MapRequest for AddHeader {

///     type Error = Infallible;

///     fn apply(&self, request: operation::Request) -> Result<operation::Request, Self::Error> {

///         request.augment(|mut request, properties| {

///             if properties.get::<NeedsHeader>().is_some() {

///                 request.headers_mut().append(

///                     self.0.clone(),

///                     self.1.clone(),

///                 );

///             }

///             Ok(request)

///         })

///     }

/// }

/// ```

pub trait MapRequest {

    /// The Error type returned by this operation.

    ///

    /// If this middleware never fails use [std::convert::Infallible] or similar.

    type Error: Into<BoxError>;

    /// Apply this middleware to a request.

    ///

    /// Typically, implementations will use [`request.augment`](crate::operation::Request::augment)

    /// to be able to transform an owned `http::Request`.

    fn apply(&self, request: operation::Request) -> Result<operation::Request, Self::Error>;

}

/// Load a response using `handler` to parse the results.

///

/// This function is intended to be used on the response side of a middleware chain.

///

/// Success and failure will be split and mapped into `SdkSuccess` and `SdkError`.

/// Generic Parameters:

/// - `B`: The Response Body

/// - `O`: The Http response handler that returns `Result<T, E>`

/// - `T`/`E`: `Result<T, E>` returned by `handler`.

pub async fn load_response<B, T, E, O>(

    mut response: http::Response<B>,

    handler: &O,

) -> Result<SdkSuccess<T, B>, SdkError<E, B>>

where

    B: http_body::Body + Unpin,

    B: From<Bytes> + 'static,

    B::Error: Into<BoxError>,

    O: ParseHttpResponse<B, Output = Result<T, E>>,

{

    if let Some(parsed_response) = handler.parse_unloaded(&mut response) {

        return sdk_result(parsed_response, response);

    }

    let body = match read_body(response.body_mut()).await {

        Ok(body) => body,

        Err(e) => {

            return Err(SdkError::ResponseError {

                raw: response,

                err: e.into(),

            });

        }

    };

    let response = response.map(|_| Bytes::from(body));

    let parsed = handler.parse_loaded(&response);

    sdk_result(parsed, response.map(B::from))

}

async fn read_body<B: http_body::Body>(body: B) -> Result<Vec<u8>, B::Error> {

    let mut output = Vec::new();

    pin_mut!(body);

    while let Some(buf) = body.data().await {

        let mut buf = buf?;

        while buf.has_remaining() {

            output.extend_from_slice(buf.chunk());

            buf.advance(buf.chunk().len())

        }

    }

    Ok(output)

}

/// Convert a `Result<T, E>` into an `SdkResult` that includes the raw HTTP response

fn sdk_result<T, E, B>(

    parsed: Result<T, E>,

    raw: http::Response<B>,

) -> Result<SdkSuccess<T, B>, SdkError<E, B>> {

    match parsed {

        Ok(parsed) => Ok(SdkSuccess { raw, parsed }),

        Err(err) => Err(SdkError::ServiceError { raw, err }),

    }

}