Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strengthen invalid_value lint to forbid uninit primitives, adjust docs to say that's UB #98919

Merged
merged 5 commits into from
Aug 30, 2022

Conversation

5225225
Copy link
Contributor

@5225225 5225225 commented Jul 5, 2022

For context: #66151 (comment)

This does not make it a FCW, but it does explicitly state in the docs that uninit integers are UB.

This also doesn't affect any runtime behavior, uninit u32's will still successfully be created through mem::uninitialized.

@rustbot rustbot added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jul 5, 2022
@rust-highfive
Copy link
Collaborator

r? @scottmcm

(rust-highfive has picked a reviewer for you, use r? to override)

@rustbot
Copy link
Collaborator

rustbot commented Jul 5, 2022

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

  • Stabilizing library features
  • Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
  • Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
  • Changing public documentation in ways that create new stability guarantees
  • Changing observable runtime behavior of library APIs

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 5, 2022
@RalfJung
Copy link
Member

RalfJung commented Jul 5, 2022

Gonna ?r

The syntax is "r?" not "?r" :)

@RalfJung RalfJung added T-lang Relevant to the language team, which will review and decide on the PR/issue. I-lang-nominated Nominated for discussion during a lang team meeting. and removed T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jul 5, 2022
@RalfJung
Copy link
Member

RalfJung commented Jul 5, 2022

Dear @rust-lang/lang, I would like to nominate this PR for discussion. I am asking that the following code is officially declared to have UB:

fn main() {
    let _val: i32 = mem::uninitialized();
}

That's just an example, of course -- this PR says that uninit memory at int, float, and raw pointer type is immediate UB, including inside tuples/arrays/structs/... (but not inside MaybeUninit, of course). This is the UCG discussion rust-lang/unsafe-code-guidelines#71.

We think this should be UB because

  • We already have a type, MaybeUninit, to handle uninitialized data. Integer types should be for integers, not arbitrary untyped data.

  • It will let us use the noundef attribute in LLVM, which can help with optimizations. Many optimizations, including ones that LLVM performs, are incorrect when inputs can be undef. LLVM is adding the noundef attribute to incrementally fix those optimizations by only performing them when inputs are know to not be undef. Clang is adding noundef to integer function arguments and return types, including char, which I do think is not justified by the C standard but anyway it makes the C/C++ equivalent of the above code have UB when compiled with clang.

  • It greatly simplifies the model of integers in an eventual Rust specification. If we allow uninit data in integers, we have to answer all sorts of new questions:

    • When we load an int where some but not all bytes are uninit, and store it back, is that preserved exactly or is now the entire thing uninit?
    • Do we allow any kind of operation on (partially) uninit integers, like arithmetic or comparison? When exactly is full initialization required?

    In contrast, the model where integers must always be fully initialized, and MaybeUninit needs to be used for arbitrary untyped data, is a lot easier to teach.

The Reference has described this code as UB for a long time. Miri considers this code UB by default for about a month now, and my plan is to soon remove the flag that can opt-out of this UB (because it will enable some significant cleanup in the interpreter engine).

This PR removes the notes from the stdlib docs which call out uninitialized integers as not being finally decided yet. It also cranks up the (warn-by-default) invalid_value lint to trigger when mem::uninitialized() or MaybeUninit::uninit().assume_init() is used to create an uninitialized integer.

@joshtriplett
Copy link
Member

Speaking only for myself, not for the lang team:

While I don't necessarily know that this pattern is the right way to specify it, I think we should have some (unsafe) mechanism to construct uninitialized or zeroed objects and freeze them. And one day when we have some of the safe-transmute infrastructure, we may have the traits or similar to be able to know when every bit pattern of a type is valid, and then we could have a safe mechanism to construct uninitialized or zeroed objects.

Would it help if MaybeUninit::assume_init() included an LLVM IR freeze instruction?

@RalfJung
Copy link
Member

RalfJung commented Jul 12, 2022

Would it help if MaybeUninit::assume_init() included an LLVM IR freeze instruction?

Help to achieve what? It would make the LLVM IR we generate for MaybeUninit::<i32>::uninit().assume_init() not UB any more even with noundef, if that's what you mean. Note however that LLVM freeeze acts on an SSA value, so we could not even easily use it for e.g. MaybeUninit<[i32; 128]>.

However, adding freeze to the language is a pretty fundamental change, that I think needs an RFC. Right now, it is impossible to observe the contents of uninitialized memory in well-defined Rust -- any attempt to do so would be UB. Once we have freeze, it will be possible for well-defined (probably even safe) Rust to observe the contents of freshly allocated memory. And as someone once said, that memory will either contain random garbage or your secret key, whichever of the two is worse for the current situation. So, I myself am fairly undecided about whether or not we should have freeze -- though I do currently expect that we will have it one day.

But I also think (a) that needs to be a deliberate decision on its own, not tied to whether any particular other operation is UB (i.e., I don't think we should entangle the freeze discussion with the discussion whether uninit integers are UB), and (b) I don't think assume_init should freeze; I think that should be done more explicitly (e.g. via a new MaybeUninit::uninit_frozen constructor, or so).

@saethlin
Copy link
Member

saethlin commented Jul 12, 2022

I am not sure freezing the return value of MaybeUnint::assume_init adds much on top of other possibilities. It is possible to synthesize a valid bit pattern for all types (I think) except references, which we can't satisfy because they get dereferenceable and freeze doesn't help there.

@RalfJung
Copy link
Member

It is possible to synthesize a valid bit pattern for all types (I think) except references,

And uninhabited types.
But that's slow, the idea with freeze is that it doesn't translate to any actual code.

@joshtriplett
Copy link
Member

While I appreciate not wanting to entangle things unnecessarily, I think this is relevant for the question of whether an uninitialized integer should in fact be UB.

I personally would like zeroed memory to be safe, and uninitialized memory to be unsafe but sound.

@RalfJung
Copy link
Member

RalfJung commented Jul 13, 2022

uninitialized memory to be unsafe but sound.

That means never using noundef attributes for integer types. Also this violates our principle that unsafe is for UB concerns only. If it's not UB to do MaybeUninit::<i32>::uninit().assume_init() then it should be fine to expose this as a safe operation.

Btw, I am not sure how exactly you use the word "uninitialized". Memory that has been frozen is initialized, as far as I am concerned. Just to a non-deterministic value. That's how it would look like in a Rust spec, anyway. After all, MaybeUninit::<i32>::uninit() is not the only way to get uninit memory, so even if we make that function freeze, we still have to be able to talk about truly uninitiailized (unfrozen) memory.

Almost all code that uses integers is safe and has them fully initiailized. IMO it would be a bad idea to pessimize optimizations in that code (i.e., not have noundef) just because some tiny amount of code wants to deal with uninit memory. The cost for that should be borne by that niche code (via using MaybeUninit explicitly), not by everyone else.

@RalfJung
Copy link
Member

RalfJung commented Jul 13, 2022

Btw, I am not sure how exactly you use the word "uninitialized".

IOW, there is a difference between saying "uninit integers should be sound" and "MaybeUninit::<i32>::uninit() should be sound".

The first statement would also make code like this sound:

use std::alloc::{alloc, dealloc, Layout};

unsafe {
    let layout = Layout::new::<u16>();
    let ptr = alloc(layout);
    assert!(!ptr.is_null());
    let val = *ptr; // Is this line UB?
    assert!(val == 0 || val > 0); // If no, is this line UB? If no, can the assertion fail?
}

Or this code:

#[repr(C)]
struct S {
  f1: u16,
  f2: u32,
}

let s = S { f1: 0, f2: 0 };
let ptr = &s as *const S;
let ptr = s.cast::<u16>().wrapping_offset(1); // now it points at the padding between the fields
let val = unsafe { *ptr }; // Is this line UB?
assert!(val == 0 || val > 0); // If no, is this line UB? If no, can the assertion fail?

Is your objection that this code should also be allowed ("uninit integers should be sound"), or specifically about MaybeUninit::<i32>::uninit().assume_init()? Also I assume you want the integer returned by MaybeUninit::<i32>::uninit().assume_init() to be "sane" in the sense that, e.g. assert!(x < 0 || x >= 0) is well-defined and the assertion passes?

I feel rather strongly that the above two snippets must be UB. IMO supporting such wild things is not sufficient justification for removing noundef benefits for everyone else. That's what we have types for, after all: to express intent and invariants. We could of course have ReallyInitialized<i32> for code that wants to benefit from noundef, but that seems just silly.

@scottmcm
Copy link
Member

As an example that I conveniently just hit, not being able to use noundef on parameters would penalize everyone using Layout -- which, via Vec/Box, is almost everyone. This optimization https://alive2.llvm.org/ce/z/XdagRK is only sound with the noundef -- remove it and Alive will point out the transformation is unsound. And that optimization is exactly what's needed to improve Layout after #95295. (I filed an LLVM issue for it in llvm/llvm-project#56562.)

I agree that uninit memory should be sound but unsafe. But to me that's called MaybeUninit<_> -- which, conveniently, is even safe to create and write to. That's a much nicer model to work with for both safe and unsafe code, since it means the soundness assertion is needed in the place that's actually capable of knowing that it's been initialized, rather than saying "yeah, yeah, I'll do that later" like mem::uninitialized::<u32>() is.

It's UB to pass uninit to essentially every standard library function, even unsafe fns. So requiring the type-level separation feels absolutely worth it.

I don't think we need freeze to accept this change. If we want freeze, then we could always add a different freeze_assume_valid or something. But a frozen &u32 doesn't really get you anything -- the frozen value could still violate the validity invariant. So really I think "no undef" being part of that validity invariant makes complete sense.

(I suppose we could have a freeze_as_bytes that gave you a [u8; sizeof(T)] that you could then validate yourself decide if it's usable. But that doesn't seem something that most unsafe code wants to do -- it'd much rather not have it get implicitly frozen, and have MIRI be able to say "hey, you didn't initialize this!" at the time it tries to use it.)

@scottmcm
Copy link
Member

scottmcm commented Aug 9, 2022

@rfcbot fcp merge

As Ralf said in #98919 (comment), I think we should do this.

For extra clarity, I don't think this says "we'll never have freeze". We can still have the possibility of a "get an unspecified but fixed bit pattern" functionality as a separate method in the future. I just don't think that MaybeUninit::assume_init (or, worse, mem::uninitialized::<i32>()) should be how that's spelled.

If we're defacto going to have to put noundef on non-MaybeUninit function parameters, then trying to carve out some sort of "well integers could be undef but not when you pass them as arguments" feels more troublesome than helpful.

@rfcbot
Copy link

rfcbot commented Aug 9, 2022

Team member @scottmcm has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

@rfcbot rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels Aug 9, 2022
@RalfJung
Copy link
Member

RalfJung commented Aug 9, 2022

We can still have the possibility of a "get an unspecified but fixed bit pattern" functionality as a separate method in the future.

Definitely. I would imagine something like MaybeUninit::frozen(), which sits half-way between MaybeUninit::uninit() and MaybeUninit::zeroed(). Then MaybeUninit::<i32>::frozen().assume_init() would be defined behavior and return an unspecified but fixed integer, but MaybeUninit::<bool>::frozen().assume_init() would be UB since the arbitrary but fixed value might not be a valid bool.

@scottmcm scottmcm removed the I-lang-nominated Nominated for discussion during a lang team meeting. label Aug 11, 2022
@5225225
Copy link
Contributor Author

5225225 commented Aug 29, 2022

Checked out the rollup that this PR failed in and realised I do still need to bless some tests here (namely, the sanitizer ones, which aren't run in CI and I didn't have enabled locally either :) )

I'll fix that as soon as I can but for the time being i think it's best to @bors r-

seeing as this will fail rollups it's included in

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 29, 2022
@rustbot
Copy link
Collaborator

rustbot commented Aug 29, 2022

Some changes occurred in src/tools/clippy

cc @rust-lang/clippy

@5225225
Copy link
Contributor Author

5225225 commented Aug 29, 2022

Okay, that's fixed now, and running the tests locally passes (sanitizer leak.rs fails locally, but I can't see how that can possibly be affected by my change, so I'm just going to ignore that)

@thomcc
Copy link
Member

thomcc commented Aug 29, 2022

I think @RalfJung used the wrong delegate syntax above.

@bors delegate+

@bors
Copy link
Contributor

bors commented Aug 29, 2022

✌️ @5225225 can now approve this pull request

@RalfJung
Copy link
Member

@bors r+ rollup=iffy

@bors
Copy link
Contributor

bors commented Aug 29, 2022

📌 Commit be32460 has been approved by RalfJung

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 29, 2022
@bors
Copy link
Contributor

bors commented Aug 30, 2022

⌛ Testing commit be32460 with merge 02654a0...

@bors
Copy link
Contributor

bors commented Aug 30, 2022

☀️ Test successful - checks-actions
Approved by: RalfJung
Pushing 02654a0 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 30, 2022
@bors bors merged commit 02654a0 into rust-lang:master Aug 30, 2022
@rustbot rustbot added this to the 1.65.0 milestone Aug 30, 2022
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (02654a0): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean1 range count2
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-1.2% [-1.2%, -1.2%] 2
All ❌✅ (primary) - - 0

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean1 range count2
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
6.0% [6.0%, 6.0%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-3.8% [-4.6%, -2.9%] 2
All ❌✅ (primary) - - 0

Footnotes

  1. the arithmetic mean of the percent change 2

  2. number of relevant changes 2

@5225225 5225225 deleted the stricter-invalid-value branch August 31, 2022 11:02
@apiraino apiraino removed the to-announce Announce this issue on triage meeting label Sep 8, 2022
flip1995 pushed a commit to flip1995/rust that referenced this pull request Sep 9, 2022
…fJung

Strengthen invalid_value lint to forbid uninit primitives, adjust docs to say that's UB

For context: rust-lang#66151 (comment)

This does not make it a FCW, but it does explicitly state in the docs that uninit integers are UB.

This also doesn't affect any runtime behavior, uninit u32's will still successfully be created through mem::uninitialized.
wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this pull request Nov 27, 2022
Pkgsrc changes:
 * We now manage to build for mipsel-unknown-netbsd, but despite the
   target spec saying cpu = "mips3", the compiler manages to emit 64-bit
   instructions which cause "illegal instruction" error.  Will need more
   work.  The mipsel-unknown-netbsd entry is commentd out since there
   is no 1.64.0 bootstrap.
 * Managed to retain the build of aarch64_be, llvm needed a patch to
   avoid use of neon instructions in the BE case (llvm doesn't support
   use of neon in BE mode).  Ref. patch to
   src/llvm-project/llvm/lib/Support/BLAKE3/blake3_impl.h.
 * The minimum gcc version is now 7.x, and that includes the cross-compiler
   for the targets.  For i386 this also needs to /usr/include/gcc-7 include
   files in the target root, because immintrin.h from gcc 5 is not
   compatible with gcc 7.x.  This applies for the targets where we build
   against a root from netbsd-8 (sparc64, powerpc, i386), and files/gcc-wrap
   gets a hack for this.
 * Pick up tweak for -latomic inclusion from
   rust-lang/rust#104220
   and
   rust-lang/rust#104572
 * Retain ability to do 32-bit NetBSD, by changing from 64 to 32 bit
   types in library/std/src/sys/unix/thread_parker/netbsd.rs.
 * I've struggled a bit to get the "openssl-src" build with -latomic
   where it's needed.  I introduce "NetBSD-generic32" system type and
   use it for the NetBSD mipsel target.  There is another attempt to
   do the same in the patch to vendor/openssl-sys/build/main.rs.
 * Bump bootstraps to 1.64.0, checksum updates.

Upstream changes:

Version 1.65.0 (2022-11-03)
==========================

Language
--------
- [Error on `as` casts of enums with `#[non_exhaustive]` variants]
  (rust-lang/rust#92744)
- [Stabilize `let else`](rust-lang/rust#93628)
- [Stabilize generic associated types (GATs)]
  (rust-lang/rust#96709)
- [Add lints `let_underscore_drop`, `let_underscore_lock`, and
  `let_underscore_must_use` from Clippy]
  (rust-lang/rust#97739)
- [Stabilize `break`ing from arbitrary labeled blocks ("label-break-value")]
  (rust-lang/rust#99332)
- [Uninitialized integers, floats, and raw pointers are now considered
  immediate UB](rust-lang/rust#98919).
  Usage of `MaybeUninit` is the correct way to work with uninitialized
  memory.
- [Stabilize raw-dylib for Windows x86_64, aarch64, and thumbv7a]
  (rust-lang/rust#99916)
- [Do not allow `Drop` impl on foreign ADTs]
  (rust-lang/rust#99576)

Compiler
--------
- [Stabilize -Csplit-debuginfo on Linux]
  (rust-lang/rust#98051)
- [Use niche-filling optimization even when multiple variants have
  data] (rust-lang/rust#94075)
- [Associated type projections are now verified to be well-formed
  prior to resolving the underlying type]
  (rust-lang/rust#99217)
- [Stringify non-shorthand visibility correctly]
  (rust-lang/rust#100350)
- [Normalize struct field types when unsizing]
  (rust-lang/rust#101831)
- [Update to LLVM 15](rust-lang/rust#99464)
- [Fix aarch64 call abi to correctly zeroext when needed]
  (rust-lang/rust#97800)
- [debuginfo: Generalize C++-like encoding for enums]
  (rust-lang/rust#98393)
- [Add `special_module_name` lint]
  (rust-lang/rust#94467)
- [Add support for generating unique profraw files by default when
  using `-C instrument-coverage`]
  (rust-lang/rust#100384)
- [Allow dynamic linking for iOS/tvOS targets]
  (rust-lang/rust#100636)

New targets:
- [Add armv4t-none-eabi as a tier 3 target]
  (rust-lang/rust#100244)
- [Add powerpc64-unknown-openbsd and riscv64-unknown-openbsd as tier 3 targets]
  (rust-lang/rust#101025)
- Refer to Rust's [platform support page][platform-support-doc] for more
  information on Rust's tiered platform support.

Libraries
---------
- [Don't generate `PartialEq::ne` in derive(PartialEq)]
  (rust-lang/rust#98655)
- [Windows RNG: Use `BCRYPT_RNG_ALG_HANDLE` by default]
  (rust-lang/rust#101325)
- [Forbid mixing `System` with direct system allocator calls]
  (rust-lang/rust#101394)
- [Document no support for writing to non-blocking stdio/stderr]
  (rust-lang/rust#101416)
- [`std::layout::Layout` size must not overflow `isize::MAX` when
  rounded up to `align`](rust-lang/rust#95295)
  This also changes the safety conditions on
  `Layout::from_size_align_unchecked`.

Stabilized APIs
---------------
- [`std::backtrace::Backtrace`]
  (https://doc.rust-lang.org/stable/std/backtrace/struct.Backtrace.html)
- [`Bound::as_ref`]
  (https://doc.rust-lang.org/stable/std/ops/enum.Bound.html#method.as_ref)
- [`std::io::read_to_string`]
  (https://doc.rust-lang.org/stable/std/io/fn.read_to_string.html)
- [`<*const T>::cast_mut`]
  (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.cast_mut)
- [`<*mut T>::cast_const`]
  (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.cast_const)

These APIs are now stable in const contexts:
- [`<*const T>::offset_from`]
  (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset_from)
- [`<*mut T>::offset_from`]
  (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset_from)

Cargo
-----
- [Apply GitHub fast path even for partial hashes]
  (rust-lang/cargo#10807)
- [Do not add home bin path to PATH if it's already there]
  (rust-lang/cargo#11023)
- [Take priority into account within the pending queue]
  (rust-lang/cargo#11032).
  This slightly optimizes job scheduling by Cargo, with typically
  small improvements on larger crate graph builds.

Compatibility Notes
-------------------
- [`std::layout::Layout` size must not overflow `isize::MAX` when
  rounded up to `align`] (rust-lang/rust#95295).
  This also changes the safety conditions on
  `Layout::from_size_align_unchecked`.
- [`PollFn` now only implements `Unpin` if the closure is `Unpin`]
  (rust-lang/rust#102737).
  This is a possible breaking change if users were relying on the
  blanket unpin implementation.  See discussion on the PR for
  details of why this change was made.
- [Drop ExactSizeIterator impl from std::char::EscapeAscii]
  (rust-lang/rust#99880)
  This is a backwards-incompatible change to the standard library's
  surface area, but is unlikely to affect real world usage.
- [Do not consider a single repeated lifetime eligible for elision
  in the return type] (rust-lang/rust#103450)
  This behavior was unintentionally changed in 1.64.0, and this
  release reverts that change by making this an error again.
- [Reenable disabled early syntax gates as future-incompatibility
  lints] (rust-lang/rust#99935)
- [Update the minimum external LLVM to 13]
  (rust-lang/rust#100460)
- [Don't duplicate file descriptors into stdio fds]
  (rust-lang/rust#101426)
- [Sunset RLS](rust-lang/rust#100863)
- [Deny usage of `#![cfg_attr(..., crate_type = ...)]` to set the
  crate type] (rust-lang/rust#99784)
  This strengthens the forward compatibility lint
  deprecated_cfg_attr_crate_type_name to deny.
- [`llvm-has-rust-patches` allows setting the build system to treat
  the LLVM as having Rust-specific patches]
  (rust-lang/rust#101072)
  This option may need to be set for distributions that are building
  Rust with a patched LLVM via `llvm-config`, not the built-in
  LLVM.

Internal Changes
----------------

These changes do not affect any public interfaces of Rust, but they represent
significant improvements to the performance or internals of rustc and related
tools.

- [Add `x.sh` and `x.ps1` shell scripts]
  (rust-lang/rust#99992)
- [compiletest: use target cfg instead of hard-coded tables]
  (rust-lang/rust#100260)
- [Use object instead of LLVM for reading bitcode from rlibs]
  (rust-lang/rust#98100)
- [Enable MIR inlining for optimized compilations]
  (rust-lang/rust#91743)
  This provides a 3-10% improvement in compiletimes for real world
  crates. See [perf results]
  (https://perf.rust-lang.org/compare.html?start=aedf78e56b2279cc869962feac5153b6ba7001ed&end=0075bb4fad68e64b6d1be06bf2db366c30bc75e1&stat=instructions:u).
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Jan 23, 2023
Pkgsrc changes:
 * pkglint cleanups, bump bootstrap kits to 1.65.0.
 * New target: mipsel-unknown-netbsd, for cpu=mips32 with soft-float.
 * Managed to retain the build of aarch64_be, llvm needed a patch to
   avoid use of neon instructions in the BE case (llvm doesn't support
   use of neon in BE mode).  Ref. patch to
   src/llvm-project/llvm/lib/Support/BLAKE3/blake3_impl.h.
   Also submitted upstream of LLVM to the BLAKE3 maintainers.
 * The minimum gcc version is now 7.x, and that includes the
   cross-compiler for the targets.  For i386 this also needs to
   /usr/include/gcc-7 include files in the target root, because
   immintrin.h from gcc 5 is not compatible with gcc 7.x.  This
   applies for the targets where we build against a root from netbsd-8
   (sparc64, powerpc, i386), and files/gcc-wrap gets a hack for this.
 * Pick up tweak for -latomic inclusion from
   rust-lang/rust#104220
   and
   rust-lang/rust#104572
 * Retain ability to do 32-bit NetBSD, by changing from 64 to 32 bit
   types in library/std/src/sys/unix/thread_parker/netbsd.rs.
 * I've tried to get the "openssl-src" build with -latomic where it's
   needed.  I've introduced the "NetBSD-generic32" system type and use
   it for the NetBSD mipsel target.  There is another attempt to do
   the same in the patch to vendor/openssl-sys/build/main.rs.


Upstream changes:

Version 1.66.1 (2023-01-10)
===========================

- Added validation of SSH host keys for git URLs in Cargo
  ([CVE-2022-46176](https://www.cve.org/CVERecord?id=CVE-2022-46176))


Version 1.66.0 (2022-12-15)
===========================

Language
--------
- [Permit specifying explicit discriminants on all `repr(Int)`
  enums](rust-lang/rust#95710)
  ```rust
  #[repr(u8)]
  enum Foo {
      A(u8) = 0,
      B(i8) = 1,
      C(bool) = 42,
  }
  ```
- [Allow transmutes between the same type differing only in
  lifetimes](rust-lang/rust#101520)
- [Change constant evaluation errors from a deny-by-default lint to a
  hard error](rust-lang/rust#102091)
- [Trigger `must_use` on `impl Trait` for
  supertraits](rust-lang/rust#102287) This
  makes `impl ExactSizeIterator` respect the existing `#[must_use]`
  annotation on `Iterator`.
- [Allow `..X` and `..=X` in
  patterns](rust-lang/rust#102275)
- [Uplift `clippy::for_loops_over_fallibles` lint into
  rustc](rust-lang/rust#99696)
- [Stabilize `sym` operands in inline
  assembly](rust-lang/rust#103168)
- [Update to Unicode 15](rust-lang/rust#101912)
- [Opaque types no longer imply lifetime
  bounds](rust-lang/rust#95474) This is a
  soundness fix which may break code that was erroneously relying on this
  behavior.

Compiler
--------
- [Add armv5te-none-eabi and thumbv5te-none-eabi tier 3
  targets](rust-lang/rust#101329)
  - Refer to Rust's [platform support page][platform-support-doc] for
    more information on Rust's tiered platform support.
- [Add support for linking against macOS universal
  libraries](rust-lang/rust#98736)

Libraries
---------
- [Fix `#[derive(Default)]` on a generic `#[default]` enum adding
  unnecessary `Default`
  bounds](rust-lang/rust#101040)
- [Update to Unicode 15](rust-lang/rust#101821)

Stabilized APIs
---------------
- [`proc_macro::Span::source_text`](https://doc.rust-lang.org/stable/proc_macro/struct.Span.html#method.source_text)
- [`uX::{checked_add_signed, overflowing_add_signed,
  saturating_add_signed,
  wrapping_add_signed}`](https://doc.rust-lang.org/stable/std/primitive.u8.html#method.checked_add_signed)
- [`iX::{checked_add_unsigned, overflowing_add_unsigned,
  saturating_add_unsigned,
  wrapping_add_unsigned}`](https://doc.rust-lang.org/stable/std/primitive.i8.html#method.checked_add_unsigned)
- [`iX::{checked_sub_unsigned, overflowing_sub_unsigned,
  saturating_sub_unsigned,
  wrapping_sub_unsigned}`](https://doc.rust-lang.org/stable/std/primitive.i8.html#method.checked_sub_unsigned)
- [`BTreeSet::{first, last, pop_first,
  pop_last}`](https://doc.rust-lang.org/stable/std/collections/struct.BTreeSet.html#method.first)
- [`BTreeMap::{first_key_value, last_key_value, first_entry, last_entry,
  pop_first,
  pop_last}`](https://doc.rust-lang.org/stable/std/collections/struct.BTreeMap.html#method.first_key_value)
- [Add `AsFd` implementations for stdio lock types on
  WASI.](rust-lang/rust#101768)
- [`impl TryFrom<Vec<T>> for Box<[T;
  N]>`](https://doc.rust-lang.org/stable/std/boxed/struct.Box.html#impl-TryFrom%3CVec%3CT%2C%20Global%3E%3E-for-Box%3C%5BT%3B%20N%5D%2C%20Global%3E)
- [`core::hint::black_box`](https://doc.rust-lang.org/stable/std/hint/fn.black_box.html)
- [`Duration::try_from_secs_{f32,f64}`](https://doc.rust-lang.org/stable/std/time/struct.Duration.html#method.try_from_secs_f32)
- [`Option::unzip`](https://doc.rust-lang.org/stable/std/option/enum.Option.html#method.unzip)
- [`std::os::fd`](https://doc.rust-lang.org/stable/std/os/fd/index.html)

Rustdoc
-------
- [Add Rustdoc warning for invalid HTML tags in the
  documentation](rust-lang/rust#101720)

Cargo
-----
- [Added `cargo remove` to remove dependencies from
  Cargo.toml](https://doc.rust-lang.org/nightly/cargo/commands/cargo-remove.html)
- [`cargo publish` now waits for the new version to be downloadable
  before exiting](rust-lang/cargo#11062)

See [detailed release notes](https://github.com/rust-lang/cargo/blob/master/CHANGELOG.md#cargo-166-2022-12-15) for more.

Compatibility Notes
-------------------
- [Only apply `ProceduralMasquerade` hack to older versions of
  `rental`](rust-lang/rust#94063)
- [Don't export `__heap_base` and `__data_end` on
  wasm32-wasi.](rust-lang/rust#102385)
- [Don't export `__wasm_init_memory` on
  WebAssembly.](rust-lang/rust#102426)
- [Only export `__tls_*` on
  wasm32-unknown-unknown.](rust-lang/rust#102440)
- [Don't link to `libresolv` in libstd on
  Darwin](rust-lang/rust#102766)
- [Update libstd's libc to 0.2.135 (to make `libstd` no longer pull in
  `libiconv.dylib` on
  Darwin)](rust-lang/rust#103277)
- [Opaque types no longer imply lifetime
  bounds](rust-lang/rust#95474)
  This is a soundness fix which may break code that was erroneously
  relying on this behavior.
- [Make `order_dependent_trait_objects` show up in future-breakage
  reports](rust-lang/rust#102635)
- [Change std::process::Command spawning to default to inheriting the
  parent's signal mask](rust-lang/rust#101077)

Internal Changes
----------------

These changes do not affect any public interfaces of Rust, but they
represent significant improvements to the performance or internals of
rustc and related tools.

- [Enable BOLT for LLVM
  compilation](rust-lang/rust#94381)
- [Enable LTO for
  rustc_driver.so](rust-lang/rust#101403)


Version 1.65.0 (2022-11-03)
==========================

Language
--------
- [Error on `as` casts of enums with `#[non_exhaustive]` variants]
  (rust-lang/rust#92744)
- [Stabilize `let else`](rust-lang/rust#93628)
- [Stabilize generic associated types (GATs)]
  (rust-lang/rust#96709)
- [Add lints `let_underscore_drop`, `let_underscore_lock`, and
  `let_underscore_must_use` from Clippy]
  (rust-lang/rust#97739)
- [Stabilize `break`ing from arbitrary labeled blocks ("label-break-value")]
  (rust-lang/rust#99332)
- [Uninitialized integers, floats, and raw pointers are now considered
  immediate UB](rust-lang/rust#98919).
  Usage of `MaybeUninit` is the correct way to work with uninitialized
  memory.
- [Stabilize raw-dylib for Windows x86_64, aarch64, and thumbv7a]
  (rust-lang/rust#99916)
- [Do not allow `Drop` impl on foreign ADTs]
  (rust-lang/rust#99576)

Compiler
--------
- [Stabilize -Csplit-debuginfo on Linux]
  (rust-lang/rust#98051)
- [Use niche-filling optimization even when multiple variants have
  data] (rust-lang/rust#94075)
- [Associated type projections are now verified to be well-formed
  prior to resolving the underlying type]
  (rust-lang/rust#99217)
- [Stringify non-shorthand visibility correctly]
  (rust-lang/rust#100350)
- [Normalize struct field types when unsizing]
  (rust-lang/rust#101831)
- [Update to LLVM 15](rust-lang/rust#99464)
- [Fix aarch64 call abi to correctly zeroext when needed]
  (rust-lang/rust#97800)
- [debuginfo: Generalize C++-like encoding for enums]
  (rust-lang/rust#98393)
- [Add `special_module_name` lint]
  (rust-lang/rust#94467)
- [Add support for generating unique profraw files by default when
  using `-C instrument-coverage`]
  (rust-lang/rust#100384)
- [Allow dynamic linking for iOS/tvOS targets]
  (rust-lang/rust#100636)

New targets:
- [Add armv4t-none-eabi as a tier 3 target]
  (rust-lang/rust#100244)
- [Add powerpc64-unknown-openbsd and riscv64-unknown-openbsd as tier 3 targets]
  (rust-lang/rust#101025)
- Refer to Rust's [platform support page][platform-support-doc] for more
  information on Rust's tiered platform support.

Libraries
---------
- [Don't generate `PartialEq::ne` in derive(PartialEq)]
  (rust-lang/rust#98655)
- [Windows RNG: Use `BCRYPT_RNG_ALG_HANDLE` by default]
  (rust-lang/rust#101325)
- [Forbid mixing `System` with direct system allocator calls]
  (rust-lang/rust#101394)
- [Document no support for writing to non-blocking stdio/stderr]
  (rust-lang/rust#101416)
- [`std::layout::Layout` size must not overflow `isize::MAX` when
  rounded up to `align`](rust-lang/rust#95295)
  This also changes the safety conditions on
  `Layout::from_size_align_unchecked`.

Stabilized APIs
---------------
- [`std::backtrace::Backtrace`]
  (https://doc.rust-lang.org/stable/std/backtrace/struct.Backtrace.html)
- [`Bound::as_ref`]
  (https://doc.rust-lang.org/stable/std/ops/enum.Bound.html#method.as_ref)
- [`std::io::read_to_string`]
  (https://doc.rust-lang.org/stable/std/io/fn.read_to_string.html)
- [`<*const T>::cast_mut`]
  (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.cast_mut)
- [`<*mut T>::cast_const`]
  (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.cast_const)

These APIs are now stable in const contexts:
- [`<*const T>::offset_from`]
  (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset_from)
- [`<*mut T>::offset_from`]
  (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset_from)

Cargo
-----
- [Apply GitHub fast path even for partial hashes]
  (rust-lang/cargo#10807)
- [Do not add home bin path to PATH if it's already there]
  (rust-lang/cargo#11023)
- [Take priority into account within the pending queue]
  (rust-lang/cargo#11032).
  This slightly optimizes job scheduling by Cargo, with typically
  small improvements on larger crate graph builds.

Compatibility Notes
-------------------
- [`std::layout::Layout` size must not overflow `isize::MAX` when
  rounded up to `align`] (rust-lang/rust#95295).
  This also changes the safety conditions on
  `Layout::from_size_align_unchecked`.
- [`PollFn` now only implements `Unpin` if the closure is `Unpin`]
  (rust-lang/rust#102737).
  This is a possible breaking change if users were relying on the
  blanket unpin implementation.  See discussion on the PR for
  details of why this change was made.
- [Drop ExactSizeIterator impl from std::char::EscapeAscii]
  (rust-lang/rust#99880)
  This is a backwards-incompatible change to the standard library's
  surface area, but is unlikely to affect real world usage.
- [Do not consider a single repeated lifetime eligible for elision
  in the return type] (rust-lang/rust#103450)
  This behavior was unintentionally changed in 1.64.0, and this
  release reverts that change by making this an error again.
- [Reenable disabled early syntax gates as future-incompatibility
  lints] (rust-lang/rust#99935)
- [Update the minimum external LLVM to 13]
  (rust-lang/rust#100460)
- [Don't duplicate file descriptors into stdio fds]
  (rust-lang/rust#101426)
- [Sunset RLS](rust-lang/rust#100863)
- [Deny usage of `#![cfg_attr(..., crate_type = ...)]` to set the
  crate type] (rust-lang/rust#99784)
  This strengthens the forward compatibility lint
  deprecated_cfg_attr_crate_type_name to deny.
- [`llvm-has-rust-patches` allows setting the build system to treat
  the LLVM as having Rust-specific patches]
  (rust-lang/rust#101072)
  This option may need to be set for distributions that are building
  Rust with a patched LLVM via `llvm-config`, not the built-in
  LLVM.

Internal Changes
----------------

These changes do not affect any public interfaces of Rust, but they represent
significant improvements to the performance or internals of rustc and related
tools.

- [Add `x.sh` and `x.ps1` shell scripts]
  (rust-lang/rust#99992)
- [compiletest: use target cfg instead of hard-coded tables]
  (rust-lang/rust#100260)
- [Use object instead of LLVM for reading bitcode from rlibs]
  (rust-lang/rust#98100)
- [Enable MIR inlining for optimized compilations]
  (rust-lang/rust#91743)
  This provides a 3-10% improvement in compiletimes for real world
  crates. See [perf results]
  (https://perf.rust-lang.org/compare.html?start=aedf78e56b2279cc869962feac5153b6ba7001ed&end=0075bb4fad68e64b6d1be06bf2db366c30bc75e1&stat=instructions:u).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. finished-final-comment-period The final comment period is finished for this PR / Issue. merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-lang Relevant to the language team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.