Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix build-std updating the index on every build. #9393

Merged
merged 1 commit into from
Apr 23, 2021
Merged

Fix build-std updating the index on every build. #9393

merged 1 commit into from
Apr 23, 2021

Conversation

ehuss
Copy link
Contributor

@ehuss ehuss commented Apr 23, 2021

rust-lang/rust#83776 has caused a problem where build-std will update the index on every build. That PR added std_detect from the stdarch submodule as a path dependency of std. However, since stdarch has a workspace of its own, an exclusion had to be added to Cargo.toml so that it does not treat std_detect as a workspace member (because nested workspaces are not supported).

The problem is that the std Cargo.lock file is built thinking that std_detect is not a workspace member. This means that its dev-dependencies are not included. However, when cargo resolves the std workspace, it doesn't know that std_detect should be excluded, so it considers it a workspace member (because it is a path dependency). This means that it expects the dev-dependencies to be in Cargo.lock. Because they are missing, it ends up poisoning the registry and triggering an update:

poisoning registry https://github.com/rust-lang/crates.io-index because std_detect v0.1.5 (/Users/eric/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/stdarch/crates/std_detect) looks like it changed auxv

The solution here is to skip dev-dependencies if they are not actively being resolved, even if the package is a workspace member.

This has happened before (#8962), so I have updated the test to check for it.

There are some alternative solutions I considered:

  • Add support for nested workspaces. 😄
  • Use a symlink to std_detect in the rust-lang/rust repository so that it appears to cargo as-if it is "outside" of the stdarch workspace, and thus can be treated like a normal workspace member (and remove the "exclude"). That seems a little hacky.

Fixes #9390

@rust-highfive
Copy link

r? @Eh2406

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 23, 2021
@ehuss ehuss mentioned this pull request Apr 23, 2021
Closed
@alexcrichton
Copy link
Member

I really wanna get to a point where we don't call this resolve function twice, that's the source of so much pain...

@bors: r+

@bors
Copy link
Contributor

bors commented Apr 23, 2021

📌 Commit 3b7cb69 has been approved by alexcrichton

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 23, 2021
@bors
Copy link
Contributor

bors commented Apr 23, 2021

⌛ Testing commit 3b7cb69 with merge 02103f9...

@bors
Copy link
Contributor

bors commented Apr 23, 2021

☀️ Test successful - checks-actions
Approved by: alexcrichton
Pushing 02103f9 to master...

@bors bors merged commit 02103f9 into rust-lang:master Apr 23, 2021
@ehuss ehuss mentioned this pull request Apr 23, 2021
Merged
Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this pull request Apr 24, 2021
@Dylan-DPC
Rollup merge of rust-lang#84412 - ehuss:update-cargo, r=ehuss
81a0355 
Update cargo, rls

## cargo

18 commits in 65d57e6f384c2317f76626eac116f683e2b63665..0ed318d182e465cd66071b91ac3d265af63ef8a1
2021-04-04 15:07:52 +0000 to 2021-04-23 20:54:54 +0000
- Restore crates.io's `SourceId` hash value to before (rust-lang/cargo#9397)
- Fix loading `branch=master` patches in the v3 lock transition (rust-lang/cargo#9392)
- Update changelog for 1.52 beta changes. (rust-lang/cargo#9396)
- Fix build-std updating the index on every build. (rust-lang/cargo#9393)
- Fix typo in profile docs (rust-lang/cargo#9386)
- Fix disagreement about lockfile ordering on stable/nightly (rust-lang/cargo#9384)
- Don't give a hard error when the end-user specifies RUSTC_BOOTSTRAP=crate_name (rust-lang/cargo#9365)
- Fix rust-lang/cargo#9350 (cargo build -Z help is missing options) (rust-lang/cargo#9369)
- an struct -> a struct (rust-lang/cargo#9379)
- Handle man pages better on Windows. (rust-lang/cargo#9378)
- fix: better error message when dependency/workspace member missing (rust-lang/cargo#9368)
- Fix typo in book (rust-lang/cargo#9376)
- Don't re-use rustc cache when RUSTC_WRAPPER changes (rust-lang/cargo#9348)
- doc: add split-debuginfo doc in config chapter (rust-lang/cargo#9372)
- refactor: remove `CargoResultExt` (rust-lang/cargo#9367)
- Track "CARGO" in environment fingerprint. (rust-lang/cargo#9363)
- Update clippy lint allow set. (rust-lang/cargo#9356)
- Fix 'suport' typo in documentation (rust-lang/cargo#9338)

## rls

3 commits in 32c0fe006dcdc13e1ca0ca31de543e4436c1299e..74d1800c25498689c5b5120a1e8495fce0cd0d0d
2021-04-12 11:21:12 +0000 to 2021-04-22 21:29:51 +0000
- Bump default integration test message timeout to 30s (rust-lang/rls#1731)
- itertools = 0.9, fst = 0.4 (rust-lang/rls#1729)
- Update cargo (rust-lang/rls#1728)
Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this pull request Apr 24, 2021
@Dylan-DPC
Rollup merge of rust-lang#84412 - ehuss:update-cargo, r=ehuss
14bfb6b 
Update cargo, rls

## cargo

18 commits in 65d57e6f384c2317f76626eac116f683e2b63665..0ed318d182e465cd66071b91ac3d265af63ef8a1
2021-04-04 15:07:52 +0000 to 2021-04-23 20:54:54 +0000
- Restore crates.io's `SourceId` hash value to before (rust-lang/cargo#9397)
- Fix loading `branch=master` patches in the v3 lock transition (rust-lang/cargo#9392)
- Update changelog for 1.52 beta changes. (rust-lang/cargo#9396)
- Fix build-std updating the index on every build. (rust-lang/cargo#9393)
- Fix typo in profile docs (rust-lang/cargo#9386)
- Fix disagreement about lockfile ordering on stable/nightly (rust-lang/cargo#9384)
- Don't give a hard error when the end-user specifies RUSTC_BOOTSTRAP=crate_name (rust-lang/cargo#9365)
- Fix rust-lang/cargo#9350 (cargo build -Z help is missing options) (rust-lang/cargo#9369)
- an struct -> a struct (rust-lang/cargo#9379)
- Handle man pages better on Windows. (rust-lang/cargo#9378)
- fix: better error message when dependency/workspace member missing (rust-lang/cargo#9368)
- Fix typo in book (rust-lang/cargo#9376)
- Don't re-use rustc cache when RUSTC_WRAPPER changes (rust-lang/cargo#9348)
- doc: add split-debuginfo doc in config chapter (rust-lang/cargo#9372)
- refactor: remove `CargoResultExt` (rust-lang/cargo#9367)
- Track "CARGO" in environment fingerprint. (rust-lang/cargo#9363)
- Update clippy lint allow set. (rust-lang/cargo#9356)
- Fix 'suport' typo in documentation (rust-lang/cargo#9338)

## rls

3 commits in 32c0fe006dcdc13e1ca0ca31de543e4436c1299e..74d1800c25498689c5b5120a1e8495fce0cd0d0d
2021-04-12 11:21:12 +0000 to 2021-04-22 21:29:51 +0000
- Bump default integration test message timeout to 30s (rust-lang/rls#1731)
- itertools = 0.9, fst = 0.4 (rust-lang/rls#1729)
- Update cargo (rust-lang/rls#1728)
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 24, 2021
@bors
Auto merge of rust-lang#84412 - ehuss:update-cargo, r=ehuss
2b68027 
Update cargo, rls

## cargo

18 commits in 65d57e6f384c2317f76626eac116f683e2b63665..0ed318d182e465cd66071b91ac3d265af63ef8a1
2021-04-04 15:07:52 +0000 to 2021-04-23 20:54:54 +0000
- Restore crates.io's `SourceId` hash value to before (rust-lang/cargo#9397)
- Fix loading `branch=master` patches in the v3 lock transition (rust-lang/cargo#9392)
- Update changelog for 1.52 beta changes. (rust-lang/cargo#9396)
- Fix build-std updating the index on every build. (rust-lang/cargo#9393)
- Fix typo in profile docs (rust-lang/cargo#9386)
- Fix disagreement about lockfile ordering on stable/nightly (rust-lang/cargo#9384)
- Don't give a hard error when the end-user specifies RUSTC_BOOTSTRAP=crate_name (rust-lang/cargo#9365)
- Fix rust-lang/cargo#9350 (cargo build -Z help is missing options) (rust-lang/cargo#9369)
- an struct -> a struct (rust-lang/cargo#9379)
- Handle man pages better on Windows. (rust-lang/cargo#9378)
- fix: better error message when dependency/workspace member missing (rust-lang/cargo#9368)
- Fix typo in book (rust-lang/cargo#9376)
- Don't re-use rustc cache when RUSTC_WRAPPER changes (rust-lang/cargo#9348)
- doc: add split-debuginfo doc in config chapter (rust-lang/cargo#9372)
- refactor: remove `CargoResultExt` (rust-lang/cargo#9367)
- Track "CARGO" in environment fingerprint. (rust-lang/cargo#9363)
- Update clippy lint allow set. (rust-lang/cargo#9356)
- Fix 'suport' typo in documentation (rust-lang/cargo#9338)

## rls

3 commits in 32c0fe006dcdc13e1ca0ca31de543e4436c1299e..74d1800c25498689c5b5120a1e8495fce0cd0d0d
2021-04-12 11:21:12 +0000 to 2021-04-22 21:29:51 +0000
- Bump default integration test message timeout to 30s (rust-lang/rls#1731)
- itertools = 0.9, fst = 0.4 (rust-lang/rls#1729)
- Update cargo (rust-lang/rls#1728)
@ehuss ehuss added this to the 1.53.0 milestone Feb 6, 2022
@EdmundsEcho EdmundsEcho mentioned this pull request Feb 23, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@Eh2406 Eh2406

Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Milestone
1.53.0
Development

Successfully merging this pull request may close these issues.

Cargo does "updating crates.io index" on every build when build-std is involved
5 participants
@ehuss @rust-highfive @alexcrichton @bors @Eh2406