Auto merge of #119616 - rylev:wasm32-wasi-preview2, r=petrochenkov,m-…

…ou-se

Add a new `wasm32-wasi-preview2` target

This is the initial implementation of the MCP rust-lang/compiler-team#694 creating a new tier 3 target `wasm32-wasi-preview2`. That MCP has been seconded and will most likely be approved in a little over a week from now. For more information on the need for this target, please read the [MCP](rust-lang/compiler-team#694).

There is one aspect of this PR that will become insta-stable once these changes reach a stable compiler:
* A new `target_family` named `wasi` is introduced. This target family incorporates all wasi targets including `wasm32-wasi` and its derivative `wasm32-wasi-preview1-threads`. The difference between `target_family = wasi` and `target_os = wasi` will become much clearer when `wasm32-wasi` is renamed to `wasm32-wasi-preview1` and the `target_os` becomes `wasm32-wasi-preview1`. You can read about this target rename in [this MCP](rust-lang/compiler-team#695) which has also been seconded and will hopefully be officially approved soon.

Additional technical details include:
* Both `std::sys::wasi_preview2` and `std::os::wasi_preview2` have been created and mostly use `#[path]` annotations on their submodules to reach into the existing `wasi` (soon to be `wasi_preview1`) modules. Over time the differences between `wasi_preview1` and `wasi_preview2` will grow and most like all `#[path]` based module aliases will fall away.
* Building `wasi-preview2` relies on a [`wasi-sdk`](https://github.com/WebAssembly/wasi-sdk) in the same way that `wasi-preview1` does (one must include a `wasi-root` path in the `Config.toml` pointing to sysroot included in the wasi-sdk). The target should build against [wasi-sdk v21](https://github.com/WebAssembly/wasi-sdk/releases/tag/wasi-sdk-21) without modifications. However, the wasi-sdk itself is growing [preview2 support](WebAssembly/wasi-sdk#370) so this might shift rapidly. We will be following along quickly to make sure that building the target remains possible as the wasi-sdk changes.
* This requires a [patch to libc](https://github.com/rylev/rust-libc/tree/wasm32-wasi-preview2) that we'll need to land in conjunction with this change. Until that patch lands the target won't actually build.

compiler/rustc_span/src/symbol.rs

@@ -1881,6 +1881,7 @@ symbols! {
         vtable_align,
         vtable_size,
         warn,
+        wasip2,
         wasm_abi,
         wasm_import_module,
         wasm_target_feature,

compiler/rustc_target/src/spec/mod.rs

@@ -1575,6 +1575,7 @@ supported_targets! {
     ("wasm32-unknown-emscripten", wasm32_unknown_emscripten),
     ("wasm32-unknown-unknown", wasm32_unknown_unknown),
     ("wasm32-wasi", wasm32_wasi),
+    ("wasm32-wasip2", wasm32_wasip2),
     ("wasm32-wasi-preview1-threads", wasm32_wasi_preview1_threads),
     ("wasm64-unknown-unknown", wasm64_unknown_unknown),
 

compiler/rustc_target/src/spec/targets/wasm32_wasip2.rs

@@ -0,0 +1,64 @@
+//! The `wasm32-wasip2` target is the next evolution of the
+//! wasm32-wasi target. While the wasi specification is still under
+//! active development, the {review 2 iteration is considered an "island
+//! of stability" that should allow users to rely on it indefinitely.
+//!
+//! The `wasi` target is a proposal to define a standardized set of WebAssembly
+//! component imports that allow it to interoperate with the host system in a
+//! standardized way. This set of imports is intended to empower WebAssembly
+//! binaries with host capabilities such as filesystem access, network access, etc.
+//!
+//! Wasi Preview 2 relies on the WebAssembly component model which is an extension of
+//! the core WebAssembly specification which allows interoperability between WebAssembly
+//! modules (known as "components") through high-level, shared-nothing APIs instead of the
+//! low-level, shared-everything linear memory model of the core WebAssembly specification.
+//!
+//! You can see more about wasi at <https://wasi.dev> and the component model at
+//! <https://github.com/WebAssembly/component-model>.
+
+use crate::spec::crt_objects;
+use crate::spec::LinkSelfContainedDefault;
+use crate::spec::{base, Target};
+
+pub fn target() -> Target {
+    let mut options = base::wasm::options();
+
+    options.os = "wasi".into();
+    options.env = "p2".into();
+    options.linker = Some("wasm-component-ld".into());
+
+    options.pre_link_objects_self_contained = crt_objects::pre_wasi_self_contained();
+    options.post_link_objects_self_contained = crt_objects::post_wasi_self_contained();
+
+    // FIXME: Figure out cases in which WASM needs to link with a native toolchain.
+    options.link_self_contained = LinkSelfContainedDefault::True;
+
+    // Right now this is a bit of a workaround but we're currently saying that
+    // the target by default has a static crt which we're taking as a signal
+    // for "use the bundled crt". If that's turned off then the system's crt
+    // will be used, but this means that default usage of this target doesn't
+    // need an external compiler but it's still interoperable with an external
+    // compiler if configured correctly.
+    options.crt_static_default = true;
+    options.crt_static_respected = true;
+
+    // Allow `+crt-static` to create a "cdylib" output which is just a wasm file
+    // without a main function.
+    options.crt_static_allows_dylibs = true;
+
+    // WASI's `sys::args::init` function ignores its arguments; instead,
+    // `args::args()` makes the WASI API calls itself.
+    options.main_needs_argc_argv = false;
+
+    // And, WASI mangles the name of "main" to distinguish between different
+    // signatures.
+    options.entry_name = "__main_void".into();
+
+    Target {
+        llvm_target: "wasm32-unknown-unknown".into(),
+        pointer_width: 32,
+        data_layout: "e-m:e-p:32:32-p10:8:8-p20:8:8-i64:64-n32:64-S128-ni:1:10:20".into(),
+        arch: "wasm32".into(),
+        options,
+    }
+}

library/std/src/os/mod.rs

@@ -85,6 +85,9 @@ pub mod linux;
 #[cfg(any(target_os = "wasi", doc))]
 pub mod wasi;
 
+#[cfg(any(all(target_os = "wasi", target_env = "p2"), doc))]
+pub mod wasip2;
+
 // windows
 #[cfg(not(all(
     doc,

library/std/src/os/wasi/mod.rs

@@ -28,7 +28,8 @@
 //! [`OsStr`]: crate::ffi::OsStr
 //! [`OsString`]: crate::ffi::OsString
 
-#![stable(feature = "rust1", since = "1.0.0")]
+#![cfg_attr(not(target_env = "p2"), stable(feature = "rust1", since = "1.0.0"))]
+#![cfg_attr(target_env = "p2", unstable(feature = "wasip2", issue = "none"))]
 #![deny(unsafe_op_in_unsafe_fn)]
 #![doc(cfg(target_os = "wasi"))]
 

library/std/src/os/wasip2/mod.rs

@@ -0,0 +1,5 @@
+//! Platform-specific extensions to `std` for Preview 2 of the WebAssembly System Interface (WASI).
+//!
+//! This module is currently empty, but will be filled over time as wasi-libc support for WASI Preview 2 is stabilized.
+
+#![stable(feature = "raw_ext", since = "1.1.0")]

library/std/src/sys/pal/mod.rs

@@ -40,6 +40,9 @@ cfg_if::cfg_if! {
     } else if #[cfg(target_os = "wasi")] {
         mod wasi;
         pub use self::wasi::*;
+    } else if #[cfg(all(target_os = "wasi", target_env = "p2"))] {
+        mod wasip2;
+        pub use self::wasip2::*;
     } else if #[cfg(target_family = "wasm")] {
         mod wasm;
         pub use self::wasm::*;

library/std/src/sys/pal/wasi/helpers.rs

@@ -0,0 +1,123 @@
+use crate::io as std_io;
+use crate::mem;
+
+#[inline]
+pub fn is_interrupted(errno: i32) -> bool {
+    errno == wasi::ERRNO_INTR.raw().into()
+}
+
+pub fn decode_error_kind(errno: i32) -> std_io::ErrorKind {
+    use std_io::ErrorKind;
+
+    let Ok(errno) = u16::try_from(errno) else {
+        return ErrorKind::Uncategorized;
+    };
+
+    macro_rules! match_errno {
+        ($($($errno:ident)|+ => $errkind:ident),*, _ => $wildcard:ident $(,)?) => {
+            match errno {
+                $(e if $(e == ::wasi::$errno.raw())||+ => ErrorKind::$errkind),*,
+                _ => ErrorKind::$wildcard,
+            }
+        };
+    }
+
+    match_errno! {
+        ERRNO_2BIG           => ArgumentListTooLong,
+        ERRNO_ACCES          => PermissionDenied,
+        ERRNO_ADDRINUSE      => AddrInUse,
+        ERRNO_ADDRNOTAVAIL   => AddrNotAvailable,
+        ERRNO_AFNOSUPPORT    => Unsupported,
+        ERRNO_AGAIN          => WouldBlock,
+        //    ALREADY        => "connection already in progress",
+        //    BADF           => "bad file descriptor",
+        //    BADMSG         => "bad message",
+        ERRNO_BUSY           => ResourceBusy,
+        //    CANCELED       => "operation canceled",
+        //    CHILD          => "no child processes",
+        ERRNO_CONNABORTED    => ConnectionAborted,
+        ERRNO_CONNREFUSED    => ConnectionRefused,
+        ERRNO_CONNRESET      => ConnectionReset,
+        ERRNO_DEADLK         => Deadlock,
+        //    DESTADDRREQ    => "destination address required",
+        ERRNO_DOM            => InvalidInput,
+        //    DQUOT          => /* reserved */,
+        ERRNO_EXIST          => AlreadyExists,
+        //    FAULT          => "bad address",
+        ERRNO_FBIG           => FileTooLarge,
+        ERRNO_HOSTUNREACH    => HostUnreachable,
+        //    IDRM           => "identifier removed",
+        //    ILSEQ          => "illegal byte sequence",
+        //    INPROGRESS     => "operation in progress",
+        ERRNO_INTR           => Interrupted,
+        ERRNO_INVAL          => InvalidInput,
+        ERRNO_IO             => Uncategorized,
+        //    ISCONN         => "socket is connected",
+        ERRNO_ISDIR          => IsADirectory,
+        ERRNO_LOOP           => FilesystemLoop,
+        //    MFILE          => "file descriptor value too large",
+        ERRNO_MLINK          => TooManyLinks,
+        //    MSGSIZE        => "message too large",
+        //    MULTIHOP       => /* reserved */,
+        ERRNO_NAMETOOLONG    => InvalidFilename,
+        ERRNO_NETDOWN        => NetworkDown,
+        //    NETRESET       => "connection aborted by network",
+        ERRNO_NETUNREACH     => NetworkUnreachable,
+        //    NFILE          => "too many files open in system",
+        //    NOBUFS         => "no buffer space available",
+        ERRNO_NODEV          => NotFound,
+        ERRNO_NOENT          => NotFound,
+        //    NOEXEC         => "executable file format error",
+        //    NOLCK          => "no locks available",
+        //    NOLINK         => /* reserved */,
+        ERRNO_NOMEM          => OutOfMemory,
+        //    NOMSG          => "no message of the desired type",
+        //    NOPROTOOPT     => "protocol not available",
+        ERRNO_NOSPC          => StorageFull,
+        ERRNO_NOSYS          => Unsupported,
+        ERRNO_NOTCONN        => NotConnected,
+        ERRNO_NOTDIR         => NotADirectory,
+        ERRNO_NOTEMPTY       => DirectoryNotEmpty,
+        //    NOTRECOVERABLE => "state not recoverable",
+        //    NOTSOCK        => "not a socket",
+        ERRNO_NOTSUP         => Unsupported,
+        //    NOTTY          => "inappropriate I/O control operation",
+        ERRNO_NXIO           => NotFound,
+        //    OVERFLOW       => "value too large to be stored in data type",
+        //    OWNERDEAD      => "previous owner died",
+        ERRNO_PERM           => PermissionDenied,
+        ERRNO_PIPE           => BrokenPipe,
+        //    PROTO          => "protocol error",
+        ERRNO_PROTONOSUPPORT => Unsupported,
+        //    PROTOTYPE      => "protocol wrong type for socket",
+        //    RANGE          => "result too large",
+        ERRNO_ROFS           => ReadOnlyFilesystem,
+        ERRNO_SPIPE          => NotSeekable,
+        ERRNO_SRCH           => NotFound,
+        //    STALE          => /* reserved */,
+        ERRNO_TIMEDOUT       => TimedOut,
+        ERRNO_TXTBSY         => ResourceBusy,
+        ERRNO_XDEV           => CrossesDevices,
+        ERRNO_NOTCAPABLE     => PermissionDenied,
+        _                    => Uncategorized,
+    }
+}
+
+pub fn abort_internal() -> ! {
+    unsafe { libc::abort() }
+}
+
+pub fn hashmap_random_keys() -> (u64, u64) {
+    let mut ret = (0u64, 0u64);
+    unsafe {
+        let base = &mut ret as *mut (u64, u64) as *mut u8;
+        let len = mem::size_of_val(&ret);
+        wasi::random_get(base, len).expect("random_get failure");
+    }
+    return ret;
+}
+
+#[inline]
+pub(crate) fn err2io(err: wasi::Errno) -> std_io::Error {
+    std_io::Error::from_raw_os_error(err.raw().into())
+}

library/std/src/sys/pal/wasi/mod.rs

@@ -14,9 +14,6 @@
 //! compiling for wasm. That way it's a compile time error for something that's
 //! guaranteed to be a runtime error!
 
-use crate::io as std_io;
-use crate::mem;
-
 #[path = "../unix/alloc.rs"]
 pub mod alloc;
 pub mod args;
@@ -57,123 +54,12 @@ cfg_if::cfg_if! {
 mod common;
 pub use common::*;
 
-#[inline]
-pub fn is_interrupted(errno: i32) -> bool {
-    errno == wasi::ERRNO_INTR.raw().into()
-}
-
-pub fn decode_error_kind(errno: i32) -> std_io::ErrorKind {
-    use std_io::ErrorKind;
-
-    let Ok(errno) = u16::try_from(errno) else {
-        return ErrorKind::Uncategorized;
-    };
-
-    macro_rules! match_errno {
-        ($($($errno:ident)|+ => $errkind:ident),*, _ => $wildcard:ident $(,)?) => {
-            match errno {
-                $(e if $(e == ::wasi::$errno.raw())||+ => ErrorKind::$errkind),*,
-                _ => ErrorKind::$wildcard,
-            }
-        };
-    }
-
-    match_errno! {
-        ERRNO_2BIG           => ArgumentListTooLong,
-        ERRNO_ACCES          => PermissionDenied,
-        ERRNO_ADDRINUSE      => AddrInUse,
-        ERRNO_ADDRNOTAVAIL   => AddrNotAvailable,
-        ERRNO_AFNOSUPPORT    => Unsupported,
-        ERRNO_AGAIN          => WouldBlock,
-        //    ALREADY        => "connection already in progress",
-        //    BADF           => "bad file descriptor",
-        //    BADMSG         => "bad message",
-        ERRNO_BUSY           => ResourceBusy,
-        //    CANCELED       => "operation canceled",
-        //    CHILD          => "no child processes",
-        ERRNO_CONNABORTED    => ConnectionAborted,
-        ERRNO_CONNREFUSED    => ConnectionRefused,
-        ERRNO_CONNRESET      => ConnectionReset,
-        ERRNO_DEADLK         => Deadlock,
-        //    DESTADDRREQ    => "destination address required",
-        ERRNO_DOM            => InvalidInput,
-        //    DQUOT          => /* reserved */,
-        ERRNO_EXIST          => AlreadyExists,
-        //    FAULT          => "bad address",
-        ERRNO_FBIG           => FileTooLarge,
-        ERRNO_HOSTUNREACH    => HostUnreachable,
-        //    IDRM           => "identifier removed",
-        //    ILSEQ          => "illegal byte sequence",
-        //    INPROGRESS     => "operation in progress",
-        ERRNO_INTR           => Interrupted,
-        ERRNO_INVAL          => InvalidInput,
-        ERRNO_IO             => Uncategorized,
-        //    ISCONN         => "socket is connected",
-        ERRNO_ISDIR          => IsADirectory,
-        ERRNO_LOOP           => FilesystemLoop,
-        //    MFILE          => "file descriptor value too large",
-        ERRNO_MLINK          => TooManyLinks,
-        //    MSGSIZE        => "message too large",
-        //    MULTIHOP       => /* reserved */,
-        ERRNO_NAMETOOLONG    => InvalidFilename,
-        ERRNO_NETDOWN        => NetworkDown,
-        //    NETRESET       => "connection aborted by network",
-        ERRNO_NETUNREACH     => NetworkUnreachable,
-        //    NFILE          => "too many files open in system",
-        //    NOBUFS         => "no buffer space available",
-        ERRNO_NODEV          => NotFound,
-        ERRNO_NOENT          => NotFound,
-        //    NOEXEC         => "executable file format error",
-        //    NOLCK          => "no locks available",
-        //    NOLINK         => /* reserved */,
-        ERRNO_NOMEM          => OutOfMemory,
-        //    NOMSG          => "no message of the desired type",
-        //    NOPROTOOPT     => "protocol not available",
-        ERRNO_NOSPC          => StorageFull,
-        ERRNO_NOSYS          => Unsupported,
-        ERRNO_NOTCONN        => NotConnected,
-        ERRNO_NOTDIR         => NotADirectory,
-        ERRNO_NOTEMPTY       => DirectoryNotEmpty,
-        //    NOTRECOVERABLE => "state not recoverable",
-        //    NOTSOCK        => "not a socket",
-        ERRNO_NOTSUP         => Unsupported,
-        //    NOTTY          => "inappropriate I/O control operation",
-        ERRNO_NXIO           => NotFound,
-        //    OVERFLOW       => "value too large to be stored in data type",
-        //    OWNERDEAD      => "previous owner died",
-        ERRNO_PERM           => PermissionDenied,
-        ERRNO_PIPE           => BrokenPipe,
-        //    PROTO          => "protocol error",
-        ERRNO_PROTONOSUPPORT => Unsupported,
-        //    PROTOTYPE      => "protocol wrong type for socket",
-        //    RANGE          => "result too large",
-        ERRNO_ROFS           => ReadOnlyFilesystem,
-        ERRNO_SPIPE          => NotSeekable,
-        ERRNO_SRCH           => NotFound,
-        //    STALE          => /* reserved */,
-        ERRNO_TIMEDOUT       => TimedOut,
-        ERRNO_TXTBSY         => ResourceBusy,
-        ERRNO_XDEV           => CrossesDevices,
-        ERRNO_NOTCAPABLE     => PermissionDenied,
-        _                    => Uncategorized,
-    }
-}
-
-pub fn abort_internal() -> ! {
-    unsafe { libc::abort() }
-}
-
-pub fn hashmap_random_keys() -> (u64, u64) {
-    let mut ret = (0u64, 0u64);
-    unsafe {
-        let base = core::ptr::addr_of_mut!(ret) as *mut u8;
-        let len = mem::size_of_val(&ret);
-        wasi::random_get(base, len).expect("random_get failure");
-    }
-    return ret;
-}
-
-#[inline]
-fn err2io(err: wasi::Errno) -> std_io::Error {
-    std_io::Error::from_raw_os_error(err.raw().into())
-}
+mod helpers;
+// These exports are listed individually to work around Rust's glob import
+// conflict rules. If we glob export `helpers` and `common` together, then
+// the compiler complains about conflicts.
+pub use helpers::abort_internal;
+pub use helpers::decode_error_kind;
+use helpers::err2io;
+pub use helpers::hashmap_random_keys;
+pub use helpers::is_interrupted;