Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor and simplify logic in sdksCommon cmake script #3251

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

SergeyRyabinin
Copy link
Contributor

@SergeyRyabinin SergeyRyabinin commented Jan 15, 2025

Issue #, if available:
cmake logic depends on c2j file list in tools/code-generation/api-descriptions
This does not make much sense anymore since

  1. generated clients are all located under generated/src source tree
  2. c2j file renaming logic is extracted and duplicated within the code generation script run_code_generation.py, so additional logic of datetime extraction, c2j filename renaming and etc is not actually required in the cmake script.

i.e. cmake can rely solely on the source tree

tested by building both full main and this branch and by comparing the cmake output and resulting build/install directories, i.e.

tree ~/sdk-install-main-clean/ > mainTree.log
tree ~/sdk-install-cmake-refactor/ > refactorTree.log
diff mainTree.log refactorTree.log

Description of changes:

Check all that applies:

  • Did a review by yourself.
  • Added proper tests to cover this PR. (If tests are not applicable, explain.)
  • Checked if this PR is a breaking (APIs have been changed) change.
  • Checked if this PR will not introduce cross-platform inconsistent behavior.
  • Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

  • Linux
  • Windows
  • Android
  • MacOS
  • IOS
  • Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@SergeyRyabinin SergeyRyabinin marked this pull request as ready for review January 16, 2025 17:46
@SergeyRyabinin SergeyRyabinin changed the title [draft] Refactor and simplify logic in sdksCommon cmake script Refactor and simplify logic in sdksCommon cmake script Jan 16, 2025
get_map_element(${PROJECT_NAME} TEMP_VAR ${SDK_DEPENDENCY_LIST})
get_flatmap_value(${PROJECT_NAME} TEMP_VAR ${SDK_DEPENDENCY_LIST})
if (TEMP_VAR)
string(REPLACE "," ";" TEMP_LIST_VAR ${TEMP_VAR})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one thing I see is because get_flatmap_value returns a cmake list, it already separates by semicolon, so we do not need this line I think. It doesn't hurt but will simplify even more on removal

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw this and even tried to get rid of this replacements as well, but
while it is indeed tries to return a semicolon-separated cmake list,
thanks to some cmake deep "magic", it is returned as a comma-separated list, and then it starts to fail in other places of the project in very strange ways (like when both comma and semicolon just disappear and the whole list is concatenated into a single word).

We can address it later (if it is even possible to fix)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. Thanks for letting me know

LIST(APPEND C2J_SPECIAL_NAME_LIST "dynamodbstreams:streams.dynamodb")
set (svc "dynamodbstreams")
elseif("${svc}" MATCHES "\\.")
string(REPLACE "." ";" nameParts ${svc})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These special name mapping is also captured in https://github.com/aws/aws-sdk-cpp/blob/main/tools/code-generation/smithy/codegen/smithy2c2j_service_map.json
Please just ensure the original naming is retained otherwise smoke tests wont build because they are built using the cmake library name which mimics the folder name
example: https://github.com/aws/aws-sdk-cpp/blob/main/generated/smoke-tests/apigateway/CMakeLists.txt#L4

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please read the PR description first, I've actually documented this:

  1. c2j file renaming logic is extracted and duplicated within the code generation script run_code_generation.py, so additional logic of datetime extraction, c2j filename renaming and etc is not actually required in the cmake script.

I've run an additional test today, just in case:

# Build main branch and install
# git branch: main
rm -rf buildMainClean && cmake -B buildMainClean -S . -DCMAKE_INSTALL_PREFIX=/home/srryabin/sdk-install-main -DENABLE_SMOKE_TESTS=ON -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=toolchains/clang-c++20.cmake &> configureMain.log && cmake --build buildMainClean -j 32 --config Debug --target install &> buildMain.log

# Build this branch and install
# git branch: sr/cmakeListClients
rm -rf buildRefactor && cmake -B buildRefactor -S . -DCMAKE_INSTALL_PREFIX=/home/srryabin/sdk-install-refactor -DENABLE_SMOKE_TESTS=ON -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=toolchains/clang-c++20.cmake &> configureRefactor.log && cmake --build buildRefactor -j 32 --config Debug --target install &> buildRefactor.log

# Compare the build output, the only difference is the output from the test testing race conditions on the logging (which is pretty okay)
tree buildRefactor > cmakerefac_tree.log
tree buildMainClean > cmakeclean_tree.log
diff cmakeclean_tree.log cmakerefac_tree.log

1c1
< buildMainClean
---
> buildRefactor
24739a24740
>     │   ├── aws_sdk_2025-01-21-18.log
24847c24848,24849
<     │   └── testRaceOnLogFile2025-01-21-17.log
---
>     │   ├── testRaceOnLogFile2025-01-21-17.log
>     │   └── testRaceOnLogFile2025-01-21-18.log
25487c25489
< 4748 directories, 20736 files
---
> 4748 directories, 20738 files

63 smoke tests app are built in both cases.

Smoke test for a legacy renamed service "apigateway" also builds and runs fine on my dev machine:

~/workspace/github/aws-sdk-cpp]
my-super-duper-dev-machine $ ./buildRefactor/generated/smoke-tests/apigateway/apigateway-smoke-tests
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from APIGatewaySmokeTestSuite
[ RUN      ] APIGatewaySmokeTestSuite.GetDomainNamesSuccess
[       OK ] APIGatewaySmokeTestSuite.GetDomainNamesSuccess (61 ms)
[----------] 1 test from APIGatewaySmokeTestSuite (61 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (96 ms total)
[  PASSED  ] 1 test.

-DBUILD_ONLY logic was also tested.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for verifying.

Copy link
Contributor

@sbera87 sbera87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look fine overall. Just a minor comment and request to verify if name change affects smoke tests.

@sbera87 sbera87 self-requested a review January 21, 2025 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants