Pattern types MVP #107606

oli-obk · 2023-02-02T17:27:06Z

Doing #107299 the right way ™️

cc @joshtriplett for the language side

This PR adds pattern types to the parser and a minimal version to the type system. This isn't useful yet, as you can't create values of pattern types type except via transmute, but I thought it best to land this MVP and then iterate on the feature instead of producing a bigger and bigger PR that becomes harder to review with every addition.

rustbot · 2023-02-02T17:27:18Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @jackh726 (or someone else) soon.

Please see the contribution instructions for more information.

rustbot · 2023-02-02T17:27:24Z

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

rustc_error_messages was changed

cc @davidtwco, @compiler-errors, @JohnTitor, @estebank, @TaKO8Ki

Some changes occurred in src/tools/rustfmt

cc @rust-lang/rustfmt

rustdoc-json-types is a public (although nightly-only) API. If possible, consider changing src/librustdoc/json/conversions.rs; otherwise, make sure you bump the FORMAT_VERSION constant.

cc @CraftSpider, @aDotInTheVoid, @Enselic, @obi1kenobi

Some changes occurred in src/tools/clippy

cc @rust-lang/clippy

Some changes occurred in src/librustdoc/clean/types.rs

cc @camelid

Some changes occurred to the core trait solver

cc @rust-lang/initiative-trait-system-refactor

RalfJung · 2023-02-02T17:34:08Z

compiler/rustc_const_eval/src/interpret/intrinsics.rs

@@ -83,6 +83,9 @@ pub(crate) fn eval_nullary_intrinsic<'tcx>(
            ty::Alias(..) | ty::Param(_) | ty::Placeholder(_) | ty::Infer(_) => {
                throw_inval!(TooGeneric)
            }
+            ty::Pat(..) => {
+                unimplemented!("pattern types need to calculate pattern from their pattern")


My confusion about this error message remains.^^ Who's asking for a pattern to be computed from anything here? We just want a variant count! What does it even mean to compute a pattern from a pattern? Seems trivial, just use the identity function...

compiler-errors · 2023-02-02T17:35:40Z

compiler/rustc_parse/src/parser/ty.rs

@@ -342,6 +342,12 @@ impl<'a> Parser<'a> {
        let span = lo.to(self.prev_token.span);
        let mut ty = self.mk_ty(span, kind);

+        if self.eat_keyword(sym::is) {


doesn't this change the behavior of macro code like:

macro_rules! foo { ($ty:ty) => {}; (() is $pat:pat) => {}; } foo!(() is 0..1);

macro rules matches continue being the least forwards compatible thing ever :|

Maybe it's better to expose this through a builtin macro instead of surface syntax, like rustc_pattern_type!() which expands into ast::TyKind::Pat

Or builtin#Pat(…) instead of a built-in macro (with builtin# from T-compiler MCP #580).

macro rules matches continue being the least forwards compatible thing ever :|

And this is where the editions system can shine.

... I'm a little surprised/sad that foo! works at all today. Follow sets and greedy parsing were supposed to handle this -- you're not allowed to follow a :ty fragment with is -- but follow set restrictions are only applied within an arm, and not across arms. Being able to follow a fragment with some token in a simultaneously-taken arm matching the fragment's tokens essentially makes follow sets ineffectual at their job of making fragments forwards compatible. But simultaneously, applying follow set rules between arms would be very restrictive and probably not even solve the problem entirely, since you can attack non-follow-set grammar extensions similarly. I guess you'd need to ensure that the choice of which arm to take is always known to be forwards-compatibly LL(k) via some sort of "leading set," and if the choice isn't LL(k), ensure that the refining match fits within the grammar and follow set of the refined fragment.

i.e., complicated. And still overly restrictive, since we also have the rule for lexically prior arms taking priority over latter arms, so there's only a forwards compatibility concern if the refining arms come after the refined arm. (And until today I thought that macro parsing was greedier than it actually is, preventing foo! from ever matching the second arm here, since the $:ty arm is prior and matched the prefix. Despite probably relying on that not being the case.)

calebcartwright · 2023-02-02T18:41:17Z

src/tools/rustfmt/src/types.rs

+            ast::TyKind::Pat(ref ty, ref pat) => {
+                let ty = ty.rewrite(context, shape)?;
+                let pat = pat.rewrite(context, shape)?;
+                Some(format!("{ty} is {pat}"))
+            },


cc @rust-lang/style

This seems quite reasonable and noncontroversial to me, and I'd be surprised if there's any vehement objections to this formatting. However, want to make sure everyone has seen it in case we'd like a chance to discuss this as a team first.

In the event we do want to review as a team to identify and codify the formatting rules, then this can be changed to re-emit the original input contents in the interim via something like:

Suggested change

ast::TyKind::Pat(ref ty, ref pat) => {

let ty = ty.rewrite(context, shape)?;

let pat = pat.rewrite(context, shape)?;

Some(format!("{ty} is {pat}"))

},

ast::TyKind::Pat(..) => Some(context.snippet(self.span).to_owned()),

fbstj · 2023-02-02T19:13:16Z

compiler/rustc_error_messages/locales/en-US/hir_analysis.ftl

+hir_analysis_pattern_type_wild_pat = "wildcard patterns are not permitted for pattern types"
+    .label = "this type is the same as the inner type without a pattern"
+
+hir_analysis_pattern_type_non_const_range = "range patterns must have constant range start and end"


Suggested change

hir_analysis_pattern_type_wild_pat = "wildcard patterns are not permitted for pattern types"

.label = "this type is the same as the inner type without a pattern"

hir_analysis_pattern_type_non_const_range = "range patterns must have constant range start and end"

hir_analysis_pattern_type_wild_pat = wildcard patterns are not permitted for pattern types

.label = this type is the same as the inner type without a pattern

hir_analysis_pattern_type_non_const_range = range patterns must have constant range start and end

scottmcm · 2023-02-02T19:19:01Z

Not asking for it to be implemented in this PR, but is the idea that these will have subtyping or coercions to less-restrictive patterns? Is u8 the same as u8 is 0..=255?

I think I'm in favour, but it'd be nice to have a link to a post or something about a sketch of the goal here.

joboet · 2023-03-07T10:45:36Z

What if types with patterns didn't have niches on their own, but were subtypes of the unbounded type? This would solve construction and usage:

Construction is accomplished by matching the unbounded type against the pattern:

match value as i32 {
    x @ 0.. => Some(x), // x has type `i32 is 0..`
    _ => None,
}

Since i32 is 0.. is a subtype of i32, this is backwards compatible.

Only if the pattern type is explicitly used in a field does the niche get exposed:

struct NonZeroI32(i32 is ..0 | 1..);

println!("{}", size_of::<Option<NonZeroI32>>()); // Prints "4"
println!("{}", size_of::<Option<i32 is ..0 | 1..>>(); // Prints "5"

Since the field is not generic, the subtyping information is hidden, so having a niche is sound. #[repr(transparent)] would hide the niche again.

Maix0 · 2023-03-08T18:51:08Z

I am currently fiddling with rustc as a first project to implement way to make subenum (only at the declaration of the enum tho). This is kinda the same thing but with a broader range.

I was thinking of allowing a sort of "specialization" where you can override function that exists in the parent enum. it would allow for some optimization.

If I understand correctly these "pattern restricted types" are just for type checking and the layout right ?
For the rest of rustc they are the parent type.

Would it be possible to add this functionnality to this or not (I doubt it since in my approch the subenums are declared WITH the main enum, and no consumer can create new subenum, so no orphan rule applies).

tgross35 · 2023-04-20T19:36:13Z

Just some drive by prior art - this somewhat reminds me of ada pre & post conditions¹, which I've thought was a cool feature that could potentially fit in where clauses. Basically it just adds a runtime check (or compile time) for the arguments and for the function results, and panics if it's not true (for things like train/plane control systems and other high-SIL applications)

function Area(C: Circle)
   with Pre => C.X_Coord – C.Radius > 0.0,
        Post => Area'Result > 3.1 * C.Radius**2 and
                Area'Result < 3.2 * C.Radius**2;

Of course this is really more like the type constraints²³ that Ada also has, but just a bit of an interesting related note.

SoniEx2 · 2023-04-30T17:41:13Z

this does support #[non_exhaustive] yes?

tialaramex · 2023-06-15T10:36:54Z

I wrote "Where do I pick up a shovel and help?" four months ago. There has been no answer in the intervening period.

oli-obk · 2023-06-15T11:26:14Z

There's related discussion in https://rust-lang.zulipchat.com/#narrow/stream/144729-t-types/topic/Implementing.20Enum.20Variant.20Types/near/322878006 (TLDR: someone is working on reviving this PR with a placeholder macro for syntax (to eliminate the parser/macro issues this PR has).

I don't know what the status of that work is, but maybe you two can collaborate?

lcnr · 2023-06-15T11:28:27Z

please open a types team MCP before trying to merge anything in this area (even if unstable). The exact way this new type kind should work is non-trivial and may have complex interactions with the rest of the type system.

oli-obk · 2023-06-15T13:45:18Z

Apropos, keeping this PR open serves no purpose :D I'm closing it, we can continue the discussion on zulip

Don't forget that the lifetime on hir types is `'tcx` This PR just tracks the `'tcx` lifetime to wherever the original objects actually have that lifetime. This code is needed for rust-lang#107606 (now rust-lang#120131) so that `ast_ty_to_ty` can invoke `lit_to_const` on an argument passed to it. Currently the argument is `&hir::Ty<'_>`, but after this PR it is `&'tcx hir::Ty<'tcx>`.

Don't forget that the lifetime on hir types is `'tcx` This PR just tracks the `'tcx` lifetime to wherever the original objects actually have that lifetime. This code is needed for rust-lang/rust#107606 (now #120131) so that `ast_ty_to_ty` can invoke `lit_to_const` on an argument passed to it. Currently the argument is `&hir::Ty<'_>`, but after this PR it is `&'tcx hir::Ty<'tcx>`.

…iler-errors Implement minimal, internal-only pattern types in the type system rebase of rust-lang#107606 You can create pattern types with `std::pat::pattern_type!(ty is pat)`. The feature is incomplete and will panic on you if you use any pattern other than integral range patterns. The only way to create or deconstruct a pattern type is via `transmute`. This PR's implementation differs from the MCP's text. Specifically > This means you could implement different traits for different pattern types with the same base type. Thus, we just forbid implementing any traits for pattern types. is violated in this PR. The reason is that we do need impls after all in order to make them usable as fields. constants of type `std::time::Nanoseconds` struct are used in patterns, so the type must be structural-eq, which it only can be if you derive several traits on it. It doesn't need to be structural-eq recursively, so we can just manually implement the relevant traits on the pattern type and use the pattern type as a private field. Waiting on: * [x] move all unrelated commits into their own PRs. * [x] fix niche computation (see 2db07f9) * [x] add lots more tests * [x] T-types MCP rust-lang/types-team#126 to finish * [x] some commit cleanup * [x] full self-review * [x] remove 61bd325, it's not necessary anymore I think. * [ ] ~~make sure we never accidentally leak pattern types to user code (add stability checks or feature gate checks and appopriate tests)~~ we don't even do this for the new float primitives * [x] get approval that [the scope expansion to trait impls](https://rust-lang.zulipchat.com/#narrow/stream/326866-t-types.2Fnominated/topic/Pattern.20types.20types-team.23126/near/427670099) is ok r? `@BoxyUwU`

Don't forget that the lifetime on hir types is `'tcx` This PR just tracks the `'tcx` lifetime to wherever the original objects actually have that lifetime. This code is needed for rust-lang/rust#107606 (now #120131) so that `ast_ty_to_ty` can invoke `lit_to_const` on an argument passed to it. Currently the argument is `&hir::Ty<'_>`, but after this PR it is `&'tcx hir::Ty<'tcx>`.

…iler-errors Implement minimal, internal-only pattern types in the type system rebase of rust-lang#107606 You can create pattern types with `std::pat::pattern_type!(ty is pat)`. The feature is incomplete and will panic on you if you use any pattern other than integral range patterns. The only way to create or deconstruct a pattern type is via `transmute`. This PR's implementation differs from the MCP's text. Specifically > This means you could implement different traits for different pattern types with the same base type. Thus, we just forbid implementing any traits for pattern types. is violated in this PR. The reason is that we do need impls after all in order to make them usable as fields. constants of type `std::time::Nanoseconds` struct are used in patterns, so the type must be structural-eq, which it only can be if you derive several traits on it. It doesn't need to be structural-eq recursively, so we can just manually implement the relevant traits on the pattern type and use the pattern type as a private field. Waiting on: * [x] move all unrelated commits into their own PRs. * [x] fix niche computation (see 2db07f9) * [x] add lots more tests * [x] T-types MCP rust-lang/types-team#126 to finish * [x] some commit cleanup * [x] full self-review * [x] remove 61bd325, it's not necessary anymore I think. * [ ] ~~make sure we never accidentally leak pattern types to user code (add stability checks or feature gate checks and appopriate tests)~~ we don't even do this for the new float primitives * [x] get approval that [the scope expansion to trait impls](https://rust-lang.zulipchat.com/#narrow/stream/326866-t-types.2Fnominated/topic/Pattern.20types.20types-team.23126/near/427670099) is ok r? `@BoxyUwU`

Implement minimal, internal-only pattern types in the type system rebase of rust-lang/rust#107606 You can create pattern types with `std::pat::pattern_type!(ty is pat)`. The feature is incomplete and will panic on you if you use any pattern other than integral range patterns. The only way to create or deconstruct a pattern type is via `transmute`. This PR's implementation differs from the MCP's text. Specifically > This means you could implement different traits for different pattern types with the same base type. Thus, we just forbid implementing any traits for pattern types. is violated in this PR. The reason is that we do need impls after all in order to make them usable as fields. constants of type `std::time::Nanoseconds` struct are used in patterns, so the type must be structural-eq, which it only can be if you derive several traits on it. It doesn't need to be structural-eq recursively, so we can just manually implement the relevant traits on the pattern type and use the pattern type as a private field. Waiting on: * [x] move all unrelated commits into their own PRs. * [x] fix niche computation (see 2db07f94f44f078daffe5823680d07d4fded883f) * [x] add lots more tests * [x] T-types MCP rust-lang/types-team#126 to finish * [x] some commit cleanup * [x] full self-review * [x] remove 61bd325da19a918cc3e02bbbdce97281a389c648, it's not necessary anymore I think. * [ ] ~~make sure we never accidentally leak pattern types to user code (add stability checks or feature gate checks and appopriate tests)~~ we don't even do this for the new float primitives * [x] get approval that [the scope expansion to trait impls](https://rust-lang.zulipchat.com/#narrow/stream/326866-t-types.2Fnominated/topic/Pattern.20types.20types-team.23126/near/427670099) is ok r? `@BoxyUwU`

Don't forget that the lifetime on hir types is `'tcx` This PR just tracks the `'tcx` lifetime to wherever the original objects actually have that lifetime. This code is needed for rust-lang/rust#107606 (now #120131) so that `ast_ty_to_ty` can invoke `lit_to_const` on an argument passed to it. Currently the argument is `&hir::Ty<'_>`, but after this PR it is `&'tcx hir::Ty<'tcx>`.

oli-obk added 12 commits February 2, 2023 13:59

Add pattern types to ast

846b80e

Add pattern types to parser

e3dd737

Thread pattern types through the HIR

d220902

Start handling pattern types at the HIR -> Ty conversion boundary

f8c08a3

Mark the feature as incomplete

a9c203d

Test macros

b055a05

Restrict access to the private field of newtype indexes

50d50ab

use 'tcx where it is known to be (Step 1)

8f37d1f

use 'tcx where it is known to be (Step 2)

925e0b6

fixup step 1

7bdd60c

Don't forget that the lifetime on hir types is 'tcx

ff28254

Actually create ranged int types in the type system.

bd41537

rustbot assigned jackh726 Feb 2, 2023

RalfJung reviewed Feb 2, 2023

View reviewed changes

compiler-errors reviewed Feb 2, 2023

View reviewed changes

bug: nesting pattern types in enums doesn't use layout optimizations

e05cbfe

calebcartwright reviewed Feb 2, 2023

View reviewed changes

This was referenced Feb 2, 2023

Implement pattern types as the backing logic of rustc_scalar_valid_range attributes #107299

Closed

Add a Ranged wrapper struct to replace the rustc_scalar_range_valid attributes #103724

Closed

fbstj reviewed Feb 2, 2023

View reviewed changes

clarfonthey mentioned this pull request Apr 10, 2023

Proposal: add range value type and range value conversions between different ranges. rust-lang/rfcs#3412

Closed

Timmmm mentioned this pull request Apr 13, 2023

Is there a way to optimize Option<R64> ? olson-sean-k/decorum#38

Open

Aloso mentioned this pull request Apr 14, 2023

Traits for lossy conversions rust-lang/rfcs#3415

Open

CAD97 mentioned this pull request May 13, 2023

Make <[T]>::array_* methods fail to compile on 0 len arrays #99471

Closed

clarfonthey mentioned this pull request May 13, 2023

Tracking Issue for ascii::Char (ACP 179) #110998

Open

7 tasks

oli-obk closed this Jun 15, 2023

oli-obk deleted the pattern_types_syntax branch June 15, 2023 13:45

dtolnay mentioned this pull request Jul 17, 2023

ACP: NonNanFNN and FiniteFNN float wrappers rust-lang/libs-team#238

Closed

clarfonthey mentioned this pull request Nov 2, 2023

Enum variant types rust-lang/rfcs#2593

Closed

This was referenced Jan 18, 2024

Don't forget that the lifetime on hir types is 'tcx #120100

Merged

Implement minimal, internal-only pattern types in the type system #120131

Merged

compiler-errors mentioned this pull request Apr 8, 2024

patterns: wrong tracking issue? #123643

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pattern types MVP #107606

Pattern types MVP #107606

oli-obk commented Feb 2, 2023

rustbot commented Feb 2, 2023

rustbot commented Feb 2, 2023

RalfJung Feb 2, 2023

compiler-errors Feb 2, 2023 •

edited

Loading

Noratrieb Feb 2, 2023

compiler-errors Feb 2, 2023

fmease Feb 3, 2023 •

edited

Loading

estebank Feb 3, 2023

CAD97 Feb 16, 2023

calebcartwright Feb 2, 2023

fbstj Feb 2, 2023

scottmcm commented Feb 2, 2023

joboet commented Mar 7, 2023 •

edited

Loading

Maix0 commented Mar 8, 2023

tgross35 commented Apr 20, 2023

SoniEx2 commented Apr 30, 2023

tialaramex commented Jun 15, 2023

oli-obk commented Jun 15, 2023

lcnr commented Jun 15, 2023 •

edited

Loading

oli-obk commented Jun 15, 2023

Pattern types MVP #107606

Pattern types MVP #107606

Conversation

oli-obk commented Feb 2, 2023

rustbot commented Feb 2, 2023

rustbot commented Feb 2, 2023

RalfJung Feb 2, 2023

Choose a reason for hiding this comment

compiler-errors Feb 2, 2023 • edited Loading

Choose a reason for hiding this comment

Noratrieb Feb 2, 2023

Choose a reason for hiding this comment

compiler-errors Feb 2, 2023

Choose a reason for hiding this comment

fmease Feb 3, 2023 • edited Loading

Choose a reason for hiding this comment

estebank Feb 3, 2023

Choose a reason for hiding this comment

CAD97 Feb 16, 2023

Choose a reason for hiding this comment

calebcartwright Feb 2, 2023

Choose a reason for hiding this comment

fbstj Feb 2, 2023

Choose a reason for hiding this comment

scottmcm commented Feb 2, 2023

joboet commented Mar 7, 2023 • edited Loading

Maix0 commented Mar 8, 2023

tgross35 commented Apr 20, 2023

Footnotes

SoniEx2 commented Apr 30, 2023

tialaramex commented Jun 15, 2023

oli-obk commented Jun 15, 2023

lcnr commented Jun 15, 2023 • edited Loading

oli-obk commented Jun 15, 2023

compiler-errors Feb 2, 2023 •

edited

Loading

fmease Feb 3, 2023 •

edited

Loading

joboet commented Mar 7, 2023 •

edited

Loading

lcnr commented Jun 15, 2023 •

edited

Loading