[net9.0] Merge main into net9.0 #20956
Conversation
This pull request updates the following dependencies [marker]: <> (Begin:601bc5e1-1cae-44b5-cf5f-08db9342aa2f) ## From https://github.com/dotnet/xharness - **Subscription**: 601bc5e1-1cae-44b5-cf5f-08db9342aa2f - **Build**: 20240708.1 - **Date Produced**: July 8, 2024 9:23:17 PM UTC - **Commit**: 2c5fdbed5ea74da70649c03084bc01035f35a3fe - **Branch**: refs/heads/main [DependencyUpdate]: <> (Begin) - **Updates**: - **Microsoft.DotNet.XHarness.iOS.Shared**: [from 9.0.0-prerelease.24354.1 to 9.0.0-prerelease.24358.1][1] [1]: dotnet/xharness@0302f46...2c5fdbe [DependencyUpdate]: <> (End) [marker]: <> (End:601bc5e1-1cae-44b5-cf5f-08db9342aa2f) Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Alex Soto <[email protected]>
## Description As part of the fix for: dotnet/runtime#99248 we disabled dedup optimization in partial/hybrid AOT mode (when both interpreter and AOT compiler are enabled). This change got backported to .NET 8 and with the latest servicing release regressed build times and app sizes significantly as reported in: #20848 However, it turns out that disabling dedup optimization is not required to fix dotnet/runtime#99248 but instead we should correct the Xamarin SDK integration with this optimization which this PR is doing. The following section describes the initial problem in more details. ## Overview of AOT modes and dedup optimization When the repro project from dotnet/runtime#99248 is built with dedup enabled in hybrid AOT+interpreter mode, the app crashes with: ``` 024-07-23 14:32:37.524110+0200 IvansApp[12711:20244208] debug: AOT NOT FOUND: (wrapper other) object:gsharedvt_out_sig (intptr). 2024-07-23 14:32:37.524120+0200 IvansApp[12711:20244208] error: * Assertion at /Users/ivan/repos/runtime-mono-iOS/src/mono/mono/mini/interp/interp.c:2667, condition `is_ok (error)' not met, function:init_jit_call_info, Attempting to JIT compile method '(wrapper other) void object:gsharedvt_out_sig (intptr)' while running in aot-only mode. See https://learn.microsoft.com/xamarin/ios/internals/limitations for more information. ``` To track down why these wrappers which are used to transition from interpreter to AOT code, are not generated we need to understand when they are compiled in different AOT modes with and without dedup optimization enabled: - In full AOT setup - all assemblies AOT compiled - `gsharedvt_out_sig` methods are never generated - In hybrid AOT + interpreter setup - all assemblies AOT compiled: `MtouchInterpreter=-all` - Dedup OFF: - `gsharedvt_out_sig` methods are generated in AOT images of every assembly (to enable interpreter calling into each specific assembly - here wrappers with same signatures are duplicated) - Dedup ON: - `gsharedvt_out_sig` methods are generated only in `aot-instances` AOT image - during AOT compilation of individual assemblies generation of `gsharedvt_out_sig` is skipped - during AOT compilation of `aot-instances` assembly we collect all `gsharedvt_out_sig` variants from the full program scope and generate code for them in `aot-instances` AOT image - In hybrid AOT + interpreter setup - all assemblies interpreted except a given assembly: `MtouchInterpreter=all,-MyAssembly` - Dedup OFF: - `gsharedvt_out_sig` methods are generated in AOT image of `MyAssembly` (to enable interpreter calling into it) - Dedup ON: <- $${\color{red} ISSUE }$$ - `gsharedvt_out_sig` methods *should* be generated only in `aot-instances` AOT image, but the `aot-instances` image is missing - explanation: - what happens is that generation of `gsharedvt_out_sig` is skipped during AOT compilation of `MyAssembly` (as expected). - But, the build does not mark `aot-instances` assembly as the one that should be AOT compiled. - The reason for this is that we have a global `_IsDedupEnabled` flag, but when custom linker step analysis `aot-instances.dll` it does not see it as an assembly which should not be interpreted. - To explain that better: we mark *all* assemblies as to be interpreted (via: `MtouchInterpreter=all`), but exclude only `MyAssembly` (via: `MtouchInterpreter=all,-MyAssembly`). - So when custom linker step processes `aot-instaces.dll` it treats it as an assembly to be interpreted, so it does not mark it for AOT compilation. - This further results with `aot-instances` AOT image missing, and all the methods which we skipped during AOT compilation never get generated. ## The fix To fix this and address regressions reported in: #20848 we are reenabling dedup optimization whenever AOT compilation is requested and fixing the issue where the custom linker step for generating AOT parameters always treates the dedup assembly as the one to be AOTed. Once approved this should be backported to .NET 8 as servicing releases are also affected with it. --------- Co-authored-by: GitHub Actions Autoformatter <[email protected]>
## Description This is a follow-up PR to: #20936 We should not enable dedup when targeting `maccatalyst-x64` because in case when the project file specifies `MtouchInterpreter=all,-System.Private.CoreLib`, the build will run the full AOT compiler with interpreter enabled. In this setup the runtime is configured to run in interp only mode: https://github.com/xamarin/xamarin-macios/blob/97a91cc4e3bf7cf8a4c657ca00ece620f2e28e91/tools/common/Target.cs#L812-L813 which means that during runtime, AOT images will be ignored - aot runtime will load them but mark them as unusuable since they are compiled with `full` compiler switch and the code falls back to interpreter (ref: https://github.com/dotnet/runtime/blob/efebf202a4a9bd78702bf4bf28a027f093e15d89/src/mono/mono/mini/aot-runtime.c#L2131-L2148 ) This is problematic for the `aot-instances` container image, which has a special handling at the runtime and does not accept it to be marked as unusable (we might want to revisit this in the future): https://github.com/dotnet/runtime/blob/efebf202a4a9bd78702bf4bf28a027f093e15d89/src/mono/mono/mini/aot-runtime.c#L2527-L2529 ## Changes This PR disables dedup optimization when targeting `maccatalyst-x64` and updates the required tests to match the behavior. --- Backports: - [x] #20946 - [x] Original reenabling of dedup for .NET 9 already includes these changes: #20941 --------- Co-authored-by: GitHub Actions Autoformatter <[email protected]>
This post-build pipeline should only run against net8.0 servicing branches that don't have the commit (2abbaf9) that moved maestro publishing into the regular build pipeline.
Backport of #20952 -------- ## Description Previous fix #20945 did not take into account that when we target non arm64 apple mobile platforms we are using `MONO_AOT_MODE_INTERP_ONLY` which cannot use dedup optimization as it discards AOT images during runtime. ## Changes - Enable dedup only when targeting ARM64 - Fix tests to cover builds for both ARM64 and X64 builds Finally, the change was tested against MAUI iossimulator-x64 template app --------- Co-authored-by: Ivan Povazan <[email protected]> Co-authored-by: Ivan Povazan <[email protected]>
This comment has been minimized.
This comment has been minimized.
💻 [CI Build] Tests on macOS X64 - Mac Sonoma (14) passed 💻✅ All tests on macOS X64 - Mac Sonoma (14) passed. Pipeline on Agent |
💻 [CI Build] Tests on macOS M1 - Mac Ventura (13) passed 💻✅ All tests on macOS M1 - Mac Ventura (13) passed. Pipeline on Agent |
💻 [CI Build] Tests on macOS M1 - Mac Monterey (12) passed 💻✅ All tests on macOS M1 - Mac Monterey (12) passed. Pipeline on Agent |
💻 [CI Build] Windows Integration Tests passed 💻✅ All Windows Integration Tests passed. Pipeline on Agent |
🚀 [CI Build] Test results 🚀Test results✅ All tests passed on VSTS: test results. 🎉 All 170 tests passed 🎉 Tests counts✅ cecil: All 1 tests passed. Html Report (VSDrops) Download Pipeline on Agent |
No description provided.