Feature Scorecard badges

[naveensrinivasan]

Is your feature request related to a problem? Please describe.
Scorecard should provide a badge for repositories to include in their README to display their compliance.

Scorecard badges

The scorecard should provide badges similar to other https://github.com/badges/shields OSS badges for compliance.

Goals

• Scorecard should provide badge results based on scorecard runs on the (cron)server to ensure compliance and validation. Similar to Codecov.
• Scorecard should provide a predictable API to fetch the badges. An example could be https://somefqdn/github/ossf/scorecard/badge
• Scorecard results calculation - TBD - The discussion of calculation should be a separate issue.

Implementaion

• Scorecard would create a separate HTTP application (within the scorecard repository) which would generate the badge.
• Scorecard would use the Results from the scheduled cron run to generate the badge. The results of the cron are stored within the GCS bucket as latest.json
• The HTTP application would be stateless.
• The application would be hosted on Google Cloud Run which will scalability with less maintenance.

In this example

• The k8s repository README.md would request the scorecard badge service for the badge
• The badge service would fetch the results from the GC bucket which has the latest.json results from the cron job
• The badge service calculates the score and returns the SVG.

[naveensrinivasan]

@inferno-chromium What are your thoughts on this?

[laurentsimon]

Is there going to be a single badge/score, or one for each check?

[naveensrinivasan]

I would think of a single badge with different levels like CII https://bestpractices.coreinfrastructure.org/en/projects/569

[laurentsimon]

sounds like a good idea. SLSA levels would be great, but I'm not 100% sure they directly map to the scorecard checks.

[naveensrinivasan]

sounds like a good idea. SLSA levels would be great, but I'm not 100% sure they directly map to the scorecard checks.

That is even better IMO. We have to just agree on what check constitutes to the levels.

[laurentsimon]

@mmaraya is driving remediation efforts and working on the definition of 'tiers', aka levels for oss. @mmaraya, feel free to provide some insights/feedback.

[inferno-chromium]

SLSA is wip, and only a set of checks will apply to SLSA not all. So, this needs more thought.

[mmaraya]

I like the idea of a single badge with different levels expressed by the number of checks passed. We'll need to list the applicable checks, some of which could come from SLSA requirements but maybe not the levels themselves.

[naveensrinivasan]

I like the idea of a single badge with different levels expressed by the number of checks passed. We'll need to list the applicable checks, some of which could come from SLSA requirements but maybe not the levels themselves.

I disagree with having checks passed as a number in the badge. It could be sending a wrong message to the consumer of any of the repositories/packages. Also, it would discourage from repositories adopting the badge.

For example, https://deps.dev/go/k8s.io%2Fkubernetes has 9/16of 16 checks only 9 passes. So if I as a user would like to consume the k8s package I would be cautious because the percentage of failure is more than 40%.

But in reality, some of the checks aren't applicable to k8s like signed-releases, SAST, Automatic-Dependency-Update.

Having a tiered approach to badges would be helpful similar to CII/SLSA.

Thoughts?

[laurentsimon]

I also prefer qualitative information. In addition to what Naveen said, it empowers consumers of the dependency to make informed decisions about the risks they are comfortable taking. Some users may weight different checks differently so it's hard for us (scorecard) to make that decision on behalf of everyone.

Let's discuss this more in an upcoming meeting.

[mmaraya]

Agree that this needs more discussion. We'll need to find a balance across badge adoption, check applicability, and giving package users easily-digestible information to make a risk-based decision. Note that I wasn't proposing a percentage as not all checks will be applicable to all packages, so the denominator could vary from package to package.

[azeemshaikh38]

This is now on our roadmap. @laurentsimon @asraa could one of you update this issue with a high-level overview of the design?

[azeemshaikh38]

@rohankh532 fyi.

[azeemshaikh38]

ossf/scorecard-action#133 fixes this. Closing the issue.