Unverified Commit cc2b9474 authored by ysaito1001's avatar ysaito1001 Committed by GitHub
Browse files

Update release tooling to handle both stable and unstable crates (#3082) 

## Motivation and Context
This PR updates our smithy-rs release tooling to prepare for handling
both stable and unstable crates. Merging this PR to `main` does not make
any of the runtime crates stable, instead we will just have a supporting
mechanism in place, ready to be turned on by implementing `TODO(GA)`.
 
## Description
At a high level, this PR will
- update the `Release smithy-rs` workflow UI to have an additional input
`Stable semantic version`:
![Screenshot 2023-10-24 at 10 19 07
PM](https://github.com/awslabs/smithy-rs/assets/15333866/097502e6-3193-43f6-b39b-dd231c2d14d7)
- split an existing gradle property `smithy.rs.runtime.crate.version`
into `smithy.rs.runtime.crate.stable.version` and
`smithy.rs.runtime.crate.unstable.version`.
- make `publisher upgrade-runtime-crates-version` take a new, optional
(for backwards compatibility) command line argument `--stable-version`
to use the said gradle property
`smithy.rs.runtime.crate.stable.version`. This will allow the
`publisher` to set a value passed via a workflow input `Stable semantic
version` to `smithy.rs.runtime.crate.stable.version` in
`gradle.properties` (and the same goes for unstable crates).
- update `fixRuntimeCrateVersions` so that it fixes up runtime crate
versions based on properties `smithy.rs.runtime.crate.stable.version`
and `smithy.rs.runtime.crate.unstable.version`.

**NOTE**
There is a guard in place. When this PR gets merged and then we enter a
stable crate version in the `Stable semantic version` text box, it will
be overwritten by a version in `Unstable semantic version`, so that
1.x.y version will NOT be published to `crates.io` before GA and that
also replicates today's release workflow's behavior (only publishing
unstable crates). Just make sure we specify a 0.x.y in the `Untable
semantic version` text box because that does get shipped.

### What happens once `TODO(GA)` has been implemented?
Roughly, it will look like this. When we run the `Release smithy-rs`
workflow (not a dry-run), providing a stable version (say 1.0.0) and a
unstable version (say 0.57.0) in the workflow text boxes,
1. the workflow will create a new release branch
`smithy-rs-release-1.x.y`.
2. the workflow will set 1.0.0 to
`smithy.rs.runtime.crate.stable.version` and 0.57.0 to
`smithy.rs.runtime.crate.unstable.version` in `gradle.properties`.
3. for whatever smithy runtime crates whose package metadata says
```
[package.metadata.smithy-rs-release-tooling]
stable = true
```
their `Cargo.toml` will include `version = 1.0.0` (and `version =
0.57.0` if `smithy-rs-release-tooling` is not specified or if its value
is `stable = false`).

4. the workflow will publish smithy runtime crates accordingly to
`crates.io`.
5. releasing `aws-sdk-rust` subsequently will render `version = 1.0.0`
in `Cargo.toml` for stable AWS runtime crates (again as specified by
`smithy-rs-release-tooling`), will render `version = 1.1.0` for SDK
crates (we will not go into details here as to why it's not `1.0.0`),
and will publish those crates to `crates.io`.
 
## Testing
In a [separate
branch](https://github.com/awslabs/smithy-rs/tree/ysaito/stable-and-unstable-crates

)
that implemented `TODO(GA)`, I verified that
- our internal release pipeline was executed without errors
- a generated AWS SDK had following manifests (showing excerpts from
arbitrary crates)
```
[package]
name = "aws-sdk-s3"
version = "1.1.0"
```
```
[package]
name = "aws-smithy-types"
version = "1.0.0"
```
```
[package]
name = "aws-smithy-http"
version = "0.57.0"
```
```
[package]
name = "aws-types"
version = "1.0.0"
```
```
[package]
name = "aws-config"
version = "1.0.0"
```
----
 
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: default avatarJohn DiSanti <jdisanti@amazon.com>
parent a42c818b

.github/workflows/release.yml

+17 −8
Original line number Diff line number Diff line
@@ -13,22 +13,29 @@ env: 
  rust_version: 1.70.0



name: Release smithy-rs

run-name: ${{ github.workflow }} ${{ inputs.semantic_version }} (${{ inputs.commit_sha }}) - ${{ inputs.dry_run && 'Dry run' || 'Production run' }}

run-name: ${{ inputs.dry_run && 'Dry run' || 'Prod run' }} - ${{ github.workflow }} ${{ inputs.stable_semantic_version }}/${{ inputs.unstable_semantic_version }} (${{ inputs.commit_sha }})

on:

  workflow_dispatch:

    inputs:

      commit_sha:

        description: |

          The SHA of the git commit that you want to release.

          Commit SHA: The SHA of the git commit that you want to release.

          You must use the non-abbreviated SHA (e.g. b2318b0 won't work!).

        required: true

        type: string

      semantic_version:

        description: The semver tag that you want to release (e.g. 0.52.1)

      stable_semantic_version:

        description: |

          Stable semantic version: The semver tag that you want to release for stable crates (e.g. 1.0.2)

        required: true

        type: string

      unstable_semantic_version:

        description: |

          Unstable semantic version: The semver tag that you want to release for unstable crates (e.g. 0.52.1)

        required: true

        type: string

      dry_run:

        description: Dry runs will only produce release artifacts, but they will not cut a release tag in GitHub nor publish to crates.io

        description: |

          Dry run: When selected, it only produces release artifacts, but will not cut a release tag in GitHub or publish to crates.io

        required: true

        type: boolean

        default: true
@@ -92,7 +99,8 @@ jobs: 
      id: branch-push

      shell: bash

      env:

        SEMANTIC_VERSION: ${{ inputs.semantic_version }}

        # TODO(GA): specify ${{ inputs.stable_semantic_version }}

        SEMANTIC_VERSION: ${{ inputs.unstable_semantic_version }}

        DRY_RUN: ${{ inputs.dry_run }}

      run: |

        set -e
@@ -125,7 +133,7 @@ jobs: 
      uses: ./smithy-rs/.github/actions/docker-build

      with:

        action: upgrade-gradle-properties

        action-arguments: ${{ inputs.semantic_version }}

        action-arguments: ${{ inputs.stable_semantic_version }} ${{ inputs.unstable_semantic_version }}

    - name: Download all artifacts

      uses: ./smithy-rs/.github/actions/download-all-artifacts

    - name: Push gradle.properties changes
@@ -133,7 +141,8 @@ jobs: 
      working-directory: upgrade-gradle-properties/smithy-rs

      shell: bash

      env:

        SEMANTIC_VERSION: ${{ inputs.semantic_version }}

        # TODO(GA): specify ${{ inputs.stable_semantic_version }}

        SEMANTIC_VERSION: ${{ inputs.unstable_semantic_version }}

        RELEASE_COMMIT_SHA: ${{ inputs.commit_sha }}

        RELEASE_BRANCH_NAME: ${{ needs.get-or-create-release-branch.outputs.release_branch }}

        DRY_RUN: ${{ inputs.dry_run }}

aws/rust-runtime/aws-config/Cargo.toml

+6 −0
Original line number Diff line number Diff line
@@ -76,3 +76,9 @@ all-features = true 
targets = ["x86_64-unknown-linux-gnu"]

rustdoc-args = ["--cfg", "docsrs"]

# End of docs.rs metadata



# make sure to keep crate stability in sync with the second element of the following tuple in

# buildSrc/src/main/kotlin/CrateSet.kt:

#  Crate("aws-config", STABLE_VERSION_PROP_NAME),

[package.metadata.smithy-rs-release-tooling]

stable = true

aws/rust-runtime/aws-runtime-api/Cargo.toml

+6 −0
Original line number Diff line number Diff line
@@ -14,3 +14,9 @@ all-features = true 
targets = ["x86_64-unknown-linux-gnu"]

rustdoc-args = ["--cfg", "docsrs"]

# End of docs.rs metadata



# make sure to keep crate stability in sync with the second element of the following tuple in

# buildSrc/src/main/kotlin/CrateSet.kt:

#  Crate("aws-runtime-api", STABLE_VERSION_PROP_NAME),

[package.metadata.smithy-rs-release-tooling]

stable = true

aws/rust-runtime/aws-types/Cargo.toml

+6 −0
Original line number Diff line number Diff line
@@ -38,3 +38,9 @@ all-features = true 
targets = ["x86_64-unknown-linux-gnu"]

rustdoc-args = ["--cfg", "docsrs"]

# End of docs.rs metadata



# make sure to keep crate stability in sync with the second element of the following tuple in

# buildSrc/src/main/kotlin/CrateSet.kt:

#  Crate("aws-types", STABLE_VERSION_PROP_NAME),

[package.metadata.smithy-rs-release-tooling]

stable = true

aws/rust-runtime/aws-types/src/build_metadata.rs

+3 −1
Original line number Diff line number Diff line
@@ -93,7 +93,9 @@ mod test { 
        let meta = &BUILD_METADATA;

        // obviously a slightly brittle test. Will be a small update for Rust 2.0 and GA :-)

        assert!(meta.rust_version.starts_with("1."));

        assert!(meta.core_pkg_version.starts_with("0."));

        // In our release process towards GA, the package version could either be 0. or 1.

        // so we need to make this `assert!` more flexible.

        assert!(meta.core_pkg_version.starts_with("0.") || meta.core_pkg_version.starts_with("1."));

        // quick sanity check that we're parsing common platforms correctly

        if cfg!(target_os = "linux") {

            assert_eq!(meta.os_family, OsFamily::Linux);