dependencies: automated OSSF Scorecard runs for Envoy deps. #14191
Conversation
This script runs https://github.com/ossf/scorecard against the runtime Envoy deps. The criteria for use_category and scorecard selection are described at https://docs.google.com/document/d/1HbREo7pv7rgeIIjQn6mNpySzQE5rx2Yv9dXm5NqR2N8/edit#heading=h.xnpvc6pk0h0v. Example output is at https://docs.google.com/spreadsheets/d/1caO4qMmG8o5i2nGoEof1qMpD5_WicfiC5WcxA_5isTY/edit#gid=0. The goal will be to evolve this script to help generate and validate metadata describing dependency conformance. Signed-off-by: Harvey Tuch <[email protected]>
| confidence = score['Confidence'] | ||
| return f'{status} ({confidence})' | ||
|
|
||
| # Releases need to be extracted from Signed-Releases. |
There was a problem hiding this comment.
@inferno-chromium FYI this is how we extract the releases signal. It's a bit hokey I think, ideally we would have a 1st class scorecard for releases as well as signed-releases. There are some bugs I've left comments on in the linked spreadsheet.
There was a problem hiding this comment.
Seperate check repeats the same code and causes more github api token to get used. Unless we add some caching, which will require some refactoring and more dedicated development resources get put to scorecard, lets keep it this way.
|
@mattklein123 that's the plan, but it runs pretty slow right now and hits GitHub rate limit API issues (ossf/scorecard#80), so we can't make it part of the normal docs build. I plan on iterating on this and eventually will have something in docs. |
I think it can now. A daily cron job updates this GCS json file https://storage.googleapis.com/ossf-scorecards/latest.json which we can pull from if the dep exists and instead use the binary to query (which also has caching support). I'm working on getting envoy's deps automatically pulled in daily rather than hard-coded in, so the only time the binary will need to query is with a new dep introduced that day. When that's done I'll refactor this script to use the GCS file and move this to docs output rather than a standalone thing. |
|
@asraa I had a quick look at writing a script to use the cron job output too but didn't get very far. I'll take a look and see if there is anything worth sharing |
This script runs https://github.com/ossf/scorecard against the runtime Envoy deps. The criteria for
use_category and scorecard selection are described at
https://docs.google.com/document/d/1HbREo7pv7rgeIIjQn6mNpySzQE5rx2Yv9dXm5NqR2N8/edit#heading=h.xnpvc6pk0h0v.
Example output is at
https://docs.google.com/spreadsheets/d/1caO4qMmG8o5i2nGoEof1qMpD5_WicfiC5WcxA_5isTY/edit#gid=0.
The goal will be to evolve this script to help generate and validate metadata describing dependency
conformance.
Part of #10471.
Signed-off-by: Harvey Tuch [email protected]