MSR: Create instances of NSObjects and INativeObjects without using reflection #18519

simonrozsival · 2023-06-29T14:19:09Z

This PR adds lookup tables and factory methods for INativeObjects and NSObjects. These generated methods allow us to create instances of these objects without needing reflection.

stephen-hawley

Nifty idea, well done. I have some comments and some performance nits.

tools/dotnet-linker/Steps/ManagedRegistrarLookupTablesStep.cs

stephen-hawley · 2023-06-29T19:47:47Z

tools/dotnet-linker/Steps/ManagedRegistrarLookupTablesStep.cs

-			} else {
-				type.IsPublic = true;
+			} else if (!type.IsPublic) {
+				type.IsNotPublic = true;


I don't understand this. If it's not public, why are you marking it not public?

We need the type to be at least internal. Unless it's already public, we need to change its visibility attributes. There might be a better way to set the visibility modifier but this worked for me so far.

tools/dotnet-linker/Steps/ManagedRegistrarLookupTablesStep.cs

src/Foundation/NSObject2.cs

Makefile

src/Foundation/NSObject2.cs

src/ObjCRuntime/Runtime.cs

rolfbjarne · 2023-06-30T09:31:43Z

src/ObjCRuntime/Runtime.cs

@@ -1368,6 +1455,25 @@ static void MissingCtor (IntPtr ptr, IntPtr klass, Type type, MissingCtorResolut
 			ctorArguments [1] = owns;

 			return (T?) ctor.Invoke (ctorArguments);
+
+#if NET
+			// This is a workaround for a compiler issue.


Are you going to leave me curious what this compiler issue is?

I think the comment is outdated. I was running into an issue where the compiler would crash when T.ConstructINativeObject or T.ConstructNSObject was called in the parent method but it was probably caused by something else (or it was in combination with some invalid IL that I generated during earlier stages of development). I don't observe the same crash now that I rebuilt it with "direct" calls to those methods. I'll get rid of the comment and I think I'll also remove the local function.

Update: I've been able to find the issue that led me to move the code into a separate function. Without T.ConstructINativeObject(...) being called from different function than Runtime.ConstructINativeObject<T>, for example calling CNContactFormatter.GetDescriptorForRequiredKeys (CNContactFormatterStyle.PhoneticFullName) causes a crashes with:

2023-06-30 17:11:06.854 MySingleView[90723:11572593] error: * Assertion at /Users/runner/work/1/s/src/mono/mono/mini/mini-generic-sharing.c:2283, condition `m_class_get_vtable (info->klass)' not met ================================================================= Native Crash Reporting ================================================================= Got a SIGABRT while executing native code. This usually indicates a fatal error in the mono runtime or one of the native libraries used by your application. ================================================================= ================================================================= Native stacktrace: ================================================================= 0x10362265c - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MonoBundle/libmonosgen-2.0.dylib : mono_dump_native_crash_info 0x1035e1578 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MonoBundle/libmonosgen-2.0.dylib : mono_handle_native_crash 0x1038001a0 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MonoBundle/libmonosgen-2.0.dylib : sigabrt_signal_handler.cold.1 0x103621f10 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MonoBundle/libmonosgen-2.0.dylib : mono_runtime_setup_stat_profiler 0x18cd26a24 - /usr/lib/system/libsystem_platform.dylib : _sigtramp 0x18ccf7c28 - /usr/lib/system/libsystem_pthread.dylib : pthread_kill 0x18cc05ae8 - /usr/lib/system/libsystem_c.dylib : abort 0x102e39bf0 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MonoBundle/libxamarin-dotnet.dylib : _ZL14print_callbackPKci 0x10365e4b8 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MonoBundle/libmonosgen-2.0.dylib : monoeg_g_logv_nofree 0x10365e570 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MonoBundle/libmonosgen-2.0.dylib : monoeg_assertion_message 0x10365e5b4 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MonoBundle/libmonosgen-2.0.dylib : mono_assertion_message_unreachable 0x1035eaa8c - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MonoBundle/libmonosgen-2.0.dylib : instantiate_info 0x1035e9900 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MonoBundle/libmonosgen-2.0.dylib : mini_instantiate_gshared_info 0x103595248 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MonoBundle/libmonosgen-2.0.dylib : mini_init_method_rgctx 0x1026c2acc - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MacOS/MySingleView : ObjCRuntime_Runtime_ConstructINativeObject_T_REF_intptr_bool_System_Type_ObjCRuntime_Runtime_MissingCtorResolution_intptr_System_RuntimeMethodHandle 0x1026c5788 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MacOS/MySingleView : ObjCRuntime_Runtime_GetINativeObject_T_REF_intptr_bool_System_Type_bool_intptr_System_RuntimeMethodHandle 0x1026c4860 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MacOS/MySingleView : ObjCRuntime_Runtime_GetINativeObject_T_REF_intptr_bool_System_Type_bool 0x1026c47e0 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MacOS/MySingleView : ObjCRuntime_Runtime_GetINativeObject_T_REF_intptr_bool_bool 0x1026c4768 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MacOS/MySingleView : ObjCRuntime_Runtime_GetINativeObject_T_REF_intptr_bool 0x1026b6448 - ./xamarin-macios/tests/dotnet/MySingleView/bin/Release/net7.0-maccatalyst/maccatalyst-arm64/MySingleView.app/Contents/MacOS/MySingleView : Contacts_CNContactFormatter_GetDescriptorForRequiredKeys_Contacts_CNContactFormatterStyle ...

The comment is still outdated and I'll try to change the wording of the workaround description.

Did you file an issue for this? Seems like an issue in the AOT compiler.

I haven't done that yet but I will create a follow-up compiler issue.

dotnet/runtime#90139

tools/dotnet-linker/Steps/ManagedRegistrarLookupTablesStep.cs

src/ObjCRuntime/Runtime.cs

src/ObjCRuntime/INativeObject.cs

tools/dotnet-linker/Steps/ManagedRegistrarLookupTablesStep.cs

dalexsoto

Looking good to me 👍

src/ObjCRuntime/Runtime.cs

vs-mobiletools-engineering-service2 · 2023-08-14T16:27:06Z

💻 [CI Build] Windows Integration Tests passed 💻

✅ All Windows Integration Tests passed.

Pipeline on Agent
Hash: f834c3b82fe10c492f365c4a031197c13a33646c [PR build]

vs-mobiletools-engineering-service2 · 2023-08-14T16:30:17Z

📚 [PR Build] Artifacts 📚

Packages generated

View packages

Pipeline on Agent
Hash: [PR build]

vs-mobiletools-engineering-service2 · 2023-08-14T16:41:58Z

✅ API diff for current PR / commit

Legacy Xamarin (No breaking changes)

~~iOS~~ (no change detected)
~~tvOS~~ (no change detected)
~~watchOS~~ (no change detected)
~~macOS~~ (no change detected)

NET (empty diffs)

~~iOS~~: (empty diff detected)
~~tvOS~~: (empty diff detected)
~~MacCatalyst~~: (empty diff detected)
~~macOS~~: (empty diff detected)

❗ API diff vs stable (Breaking changes)

Legacy Xamarin (:heavy_exclamation_mark: Breaking changes :heavy_exclamation_mark:)

iOS: vsdrops gist (:heavy_exclamation_mark: Breaking changes :heavy_exclamation_mark:)
tvOS: vsdrops gist (:heavy_exclamation_mark: Breaking changes :heavy_exclamation_mark:)
watchOS: vsdrops gist (:heavy_exclamation_mark: Breaking changes :heavy_exclamation_mark:)
macOS: vsdrops gist (:heavy_exclamation_mark: Breaking changes :heavy_exclamation_mark:)

.NET (:heavy_exclamation_mark: Breaking changes :heavy_exclamation_mark:)

iOS: vsdrops gist (:heavy_exclamation_mark: Breaking changes :heavy_exclamation_mark:)
tvOS: vsdrops gist (:heavy_exclamation_mark: Breaking changes :heavy_exclamation_mark:)
MacCatalyst: vsdrops gist (:heavy_exclamation_mark: Breaking changes :heavy_exclamation_mark:)
macOS: vsdrops gist (:heavy_exclamation_mark: Breaking changes :heavy_exclamation_mark:)
Microsoft.iOS vs Microsoft.MacCatalyst: vsdrops gist

Legacy Xamarin (stable) vs .NET

iOS: vsdrops gist
tvOS: vsdrops gist
macOS: vsdrops gist

ℹ️ Generator diff

Generator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes)

Pipeline on Agent
Hash: f834c3b82fe10c492f365c4a031197c13a33646c [PR build]

vs-mobiletools-engineering-service2 · 2023-08-14T17:47:33Z

💻 [PR Build] Tests on macOS M1 - Mac Ventura (13.0) passed 💻

✅ All tests on macOS M1 - Mac Ventura (13.0) passed.

Pipeline on Agent
Hash: [PR build]

vs-mobiletools-engineering-service2 · 2023-08-14T17:52:13Z

💻 [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) passed 💻

✅ All tests on macOS M1 - Mac Big Sur (11.5) passed.

Pipeline on Agent
Hash: [PR build]

vs-mobiletools-engineering-service2 · 2023-08-14T19:55:14Z

🚀 [CI Build] Test results 🚀

Test results

✅ All tests passed on VSTS: simulator tests.

🎉 All 235 tests passed 🎉

Tests counts

✅ bcl: All 69 tests passed. Html Report (VSDrops) Download
✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests: All 1 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 7 tests passed. Html Report (VSDrops) Download
✅ framework: All 8 tests passed. Html Report (VSDrops) Download
✅ generator: All 2 tests passed. Html Report (VSDrops) Download
✅ interdependent_binding_projects: All 7 tests passed. Html Report (VSDrops) Download
✅ install_source: All 1 tests passed. Html Report (VSDrops) Download
✅ introspection: All 8 tests passed. Html Report (VSDrops) Download
✅ linker: All 65 tests passed. Html Report (VSDrops) Download
✅ mac_binding_project: All 1 tests passed. Html Report (VSDrops) Download
✅ mmp: All 2 tests passed. Html Report (VSDrops) Download
✅ mononative: All 12 tests passed. Html Report (VSDrops) Download
✅ monotouch: All 35 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ mtouch: All 1 tests passed. Html Report (VSDrops) Download
✅ xammac: All 3 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 8 tests passed. Html Report (VSDrops) Download
✅ xtro: All 2 tests passed. Html Report (VSDrops) Download

Pipeline on Agent
Hash: f834c3b82fe10c492f365c4a031197c13a33646c [PR build]

simonrozsival and others added 2 commits June 29, 2023 15:40

MSR: Construct NSObjects and INativeObjects without reflection

453d701

Auto-format source code

9f12495

This comment was marked as outdated.

Sign in to view

This comment has been minimized.

Sign in to view

stephen-hawley reviewed Jun 29, 2023

View reviewed changes

This comment has been minimized.

Sign in to view

rolfbjarne reviewed Jun 30, 2023

View reviewed changes

simonrozsival and others added 7 commits June 30, 2023 15:27

Fix comments

4b3126c

Improve code based on reviews

35fe42e

Fix bug that causes problems when msr is not used

06bc85a

Bring back workaround to avoid vtable assertion crash

31b7dd2

Remove local variable

70af4e0

Add _Xamarin_ prefix

d7680e9

Auto-format source code

950a3cc

This comment was marked as outdated.

Sign in to view

This comment has been minimized.

Sign in to view

filipnavara mentioned this pull request Jul 1, 2023

NativeAOT: Asking for Array<T> MethodTable dotnet/runtime#88287

Closed

simonrozsival added 2 commits July 1, 2023 13:46

Revert unnecessary modification to Makefile

dc2f5b8

Avoid introducing new public API

1abd943

This comment has been minimized.

Sign in to view

simonrozsival mentioned this pull request Aug 8, 2023

[Mono] Calling a virtual static method on an interface in Debug mode causes a crash dotnet/runtime#90139

Closed

This comment has been minimized.

Sign in to view

dalexsoto reviewed Aug 12, 2023

View reviewed changes

dalexsoto requested a review from rolfbjarne August 12, 2023 16:20

Merge branch 'main' into msr-construct-nsobject-without-reflection

de960f0

This comment has been minimized.

Sign in to view

rolfbjarne approved these changes Aug 14, 2023

View reviewed changes

src/ObjCRuntime/Runtime.cs Outdated Show resolved Hide resolved

Remove outdated comment

f834c3b

filipnavara approved these changes Aug 14, 2023

View reviewed changes

rolfbjarne merged commit 7f67aa0 into xamarin:main Aug 15, 2023

simonrozsival mentioned this pull request Sep 11, 2023

Research using an interface with a static abstract Create method for NSObject/INativeObject to avoid reflection #18327

Closed

MSR: Create instances of NSObjects and INativeObjects without using reflection #18519

MSR: Create instances of NSObjects and INativeObjects without using reflection #18519

Conversation

simonrozsival commented Jun 29, 2023

This comment was marked as outdated.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

stephen-hawley left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment has been minimized.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as outdated.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

dalexsoto left a comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

vs-mobiletools-engineering-service2 commented Aug 14, 2023

💻 [CI Build] Windows Integration Tests passed 💻

vs-mobiletools-engineering-service2 commented Aug 14, 2023

📚 [PR Build] Artifacts 📚

Packages generated

vs-mobiletools-engineering-service2 commented Aug 14, 2023

✅ API diff for current PR / commit

❗ API diff vs stable (Breaking changes)

ℹ️ Generator diff

vs-mobiletools-engineering-service2 commented Aug 14, 2023

💻 [PR Build] Tests on macOS M1 - Mac Ventura (13.0) passed 💻

vs-mobiletools-engineering-service2 commented Aug 14, 2023

💻 [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) passed 💻

vs-mobiletools-engineering-service2 commented Aug 14, 2023

🚀 [CI Build] Test results 🚀

Test results

Tests counts