Some naive people may still think they’re not using open-source software. They’re wrong. Everyone does. According to the Synopsys Cybersecurity Research Center (CyRC) 2021 “Open Source Security and Risk Analysis” (OSSRA) report, 95% of all commercial programs contain open-source software. By CyRC’s count, the vast majority of that code contains outdated or insecure code. But how can you tell which libraries and other components are safe without doing a deep code dive? Google and the Open Source Security Foundation (OSSF) have a quick and easy answer: The OpenSSF Security Scorecards.
These Scorecards are based on a set of automated pass/fail checks to provide a quick review of many open-source software projects. The Scorecards project is an automated security tool that produces a “risk score” for open-source programs.
That’s important because only some organizations have systems and processes in place to check new open-source dependencies for security problems. Even at Google, though, with all its resources, this process is often tedious, manual, and error-prone. Worse still, many of these projects and developers are resource-constrained. The result? Security often ends up a low priority on the task list. This leads to critical projects not following good security best practices and becoming vulnerable to exploits.
The Scorecards project hopes to make security checks easier to make security easier to achieve with the release of Scorecards v2. This includes new security checks, scaled up the number of projects being scored, and made this data easily accessible for analysis.
For developers, Scorecards help reduce the toil and manual effort required to continually evaluate changing packages when maintaining a project’s supply chain. Consumers can automatically access the risks to make informed decisions about accepting the program, look for an alternative solution, or work with the maintainers to make improvements.
Here’s what new:
Identifying Risks: Since last fall, Scorecards’ coverage has grown; the project has added several new checks, following Google’s Know, Prevent, Fix framework.
Spotting malicious contributors: Contributors with malicious intent or compromised accounts can introduce potential backdoors into code. Code reviews help mitigate such attacks. With the new Branch-Protection check, developers can verify that the project enforces mandatory code review from another developer before code is committed. Currently, this check can only be run by a repository admin due to GitHub API limitations. For a third-party repository, use the less informative Code-Review check instead.
Vulnerable Code: Even with developers and peer review’s best efforts, bad code can still enter a codebase and remain undetected. That’s why it’s important to enable continuous fuzzing and static code testing to catch bugs early in the development lifecycle. The project now checks to see if a project uses fuzzing and SAST tools as part of its continuous integration/continuous deployment (CI/CD) pipeline.
Build system compromise: A common CI/CD solution used by GitHub projects is GitHub Actions. A danger with these action workflows is that they may handle untrusted user input. Meaning, an attacker can craft a malicious pull request to gain access to the privileged GitHub token, and with it the ability to push malicious code to the repo without review. To mitigate this risk, Scorecard’s Token-Permissions prevention check now verifies that the GitHub workflows follow the principle of least privilege by making GitHub tokens read-only by default.
Bad dependencies: A program is only as secure as its weakest dependency. This may sound obvious, but the first step to knowing our dependencies is simply to declare them… and have your dependencies declare them too. Armed with this provenance information, you can assess the risks to your programs and mitigate those risks.
That’s the good news. The bad news is there are several widely used anti-patterns that break this provenance principle. The first of these anti-patterns are checked-in binaries — as there’s no way to easily verify or check the contents of the binary in the project. Thanks in particular to the continued use of proprietary drivers, this may be an unavoidable evil. Still, Scorecards provides a Binary-Artifacts check for testing this.
Another anti-pattern is the use of curl or bash in scripts, which dynamically pulls dependencies. Cryptographic hashes let us pin our dependencies to a known value. If this value ever changes, the build system detects it and refuses to build. Pinning dependencies is useful everywhere we have dependencies: Not just during compilation, but also in Dockerfiles, CI/CD workflows, etc. Scorecards checks for these anti-patterns with the Frozen-Deps check. This check is helpful for mitigating against malicious dependency attacks such as the recent CodeCov attack.
Even with hash-pinning, hashes need to be updated once in a while when dependencies patch vulnerabilities. Tools like dependabot or renovatebot can review and update the hashes. The Scorecards Automated-Dependency-Update check verifies that developers rely on such tools to update their dependencies.
It is important to know vulnerabilities in a project before using it as a dependency. Scorecards can provide this information via the new Vulnerabilities check, without subscribing to a vulnerability alert system.
That’s what new. Here is what the Scorecards project has done so far.
It now has evaluated security for over 50,000 open source projects. To scale this project, its architecture has been massively redesigned. It now uses a Pub/Sub model. This gives it improved horizontal scalability and higher throughput. This fully automated tool periodically evaluates critical open source projects and exposes the Scorecards check information through weekly updated public BigQuery dataset
To access this data, you can use the bq command-line tool. The following example shows how to export data for the Kubernetes project. For your purposes, substitute the Kubernetes repo url with the one for the program you need to check:
$ bq query –nouse_legacy_sql ‘SELECT Repo, Date, Checks FROM openssf.scorecardcron.scorecard_latest WHERE Repo=”github.com/kubernetes/kubernetes“‘
You can also see the latest data on all Scorecards analyzed projects. This data is also available in the new Google Open Source Insights project and the OpenSSF Security Metrics project. The raw data can also be examined via data analysis and visualization tools such as Google Data Studio. With the data in CSV format, you can examine it with whatever your favorite data analysis and visualization tool may be.
One thing is clear from all this data. There’s a lot of security gaps still to fill even in widely used packages such as Kubernetes. For example, many projects are not continuously fuzzed, don’t define a security policy for reporting vulnerabilities, and don’t pin dependencies. According to Google, and frankly, anyone who cares about security: “We all need to come together as an industry to drive awareness of these widespread security risks, and to make improvements that will benefit everyone.”
As helpful as Scorecards v2 is, much more work remains to be done. The project now has 23 developers, more would be welcomed. If you would like to join the fun, check out these good first-timer issues. These are all accessible via GitHub.
If you would like us to help you run Scorecards on specific projects, please submit a GitHub pull request to add them. Last but not least, Google’s developers said, “We have a lot of ideas and many more checks we’d like to add, but we want to hear from you. Tell us which checks you would like to see in the next version of Scorecards.”
Looking ahead, the team plans to add:
If I were you, I’d start using Scorecards immediately. This project can already make your work much safer and it promises to do even more to improve not only security for your programs but the programs it covers.
