Skip to content
Unverified Commit 15c73d5d authored by Fahad Zubair's avatar Fahad Zubair Committed by GitHub
Browse files

Fix CBOR codegen for recursive types by applying Box to union variants (#4116)



# Fix CBOR recursive data structure handling with Box implementation

This PR addresses an issue in the CBOR codegen where recursive union
data structures aren't properly boxed during deserialization, resulting
in uncompilable Rust code.

## Problem

The current CBOR implementation correctly defines recursive data
structures with `Box` in type definitions, but fails to apply the
corresponding `Box::new` wrapper during deserialization. This causes
type mismatches when deserializing recursive Smithy models.

For example, with this Smithy model:
```smithy
structure RecursiveOperationInputOutputNested1 {
    variant: FooChoice,
}
union FooChoice {
    choice1: String
    choice2: RecursiveOperationInputOutputNested1
}
```

The generated Rust type correctly uses `Box`:
```rust
pub enum FooChoice {
    Choice1(::std::string::String),
    Choice2(::std::boxed::Box<crate::model::RecursiveOperationInputOutputNested1>),
}
```

But the deserializer doesn't wrap the result with `Box::new`:
```rust
// In the deserializer function:
FooChoice::Choice2(
    crate::protocol_serde::shape_recursive_operation_input_output_nested1::de_recursive_operation_input_output_nested1(decoder)
?)
```

This leads to the compiler error:
```
error[E0308]: `?` operator has incompatible types
  --> shape_foo_choice.rs:14:9
   |
14 | /         crate::protocol_serde::shape_recursive_operation_input_output_nested1::de_recursive_operation_input_output_nested1(decoder)
15 | |     ?),
   | |_____^ expected `Box<RecursiveOperationInputOutputNested1>`, found `RecursiveOperationInputOutputNested1`
```

## Solution

This PR modifies the CBOR codegen to add `.map(Box::new)` when
deserializing values for union variants that are defined with `Box`. The
fixed code will generate:

```rust
FooChoice::Choice2(
    crate::protocol_serde::shape_recursive_operation_input_output_nested1::de_recursive_operation_input_output_nested1(decoder)
        .map(Box::new)?
)
```

This ensures type compatibility between the defined structure and the
deserialized data.

## Testing

Added a protocol test in `rpcv2Cbor-extras.smithy` file.

Closes: #4115

---------

Co-authored-by: default avatarFahad Zubair <fahadzub@amazon.com>
parent ad97759d
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment