A few months ago, if you’d asked someone what their biggest concern was about IT security, you would have received lots of different answers. Then Solarwinds catastrophically failed to secure its software supply chain, leading to what’s been called IT’s Pearl Harbor. So it is today that locking down your software supply chain has become job number one for all CSO and CISOs who take their jobs seriously. To answer this call for open source, the Linux Foundation, along with Red Hat, Google, and Purdue University have created the sigstore project.
The just-announced sigstore aims to improve the security of the software supply chain by enabling the easy adoption of cryptographic software signing backed by transparency log technologies. It will do this by empowering developers to securely sign software artifacts such as release files, container images, and binaries. These signing records will then be kept in a tamper-proof public log. This service will be free for all developers and software providers to use. The sigstore code and operation tooling that will be used to make this work is still being developed by the sigstore community.
With this, as David A Wheeler, the Linux Foundation’s director of Open Source Supply Chain Security, observed earlier, we’ll be on our way to creating verified reproducible builds.
Wheeler explained, “A reproducible build is one “that always produces the same outputs given the same inputs so that the build results can be verified. A verified reproducible build is a process where independent organizations produce a build from source code and verify that the built results come from the claimed source code.”
This, in turn, could be used to create a software bill of materials (SBOM). With an SBOM you’ll know exactly what code you’re using in any given project. This is another argument for open source. Orion, Solarwinds hacked program, for example, like all proprietary software, is a black box. No one except its builders knows what’s in it. And as we now know, Solarwinds didn’t know what was inside it until outside companies spotted its corruption.
Sigstore will avoid this, Luke Hinds, Red Hat’s Security Engineering lead in the office of the CTO, explained as it will enable “all open-source communities to sign their software and combine provenance, integrity, and discoverability to create a transparent and auditable software supply chain.”
This isn’t easy. While there are some open-source digital signing tools available today, few developers use them. Many programmers, even now, don’t see the point of taking the extra steps needed to “sign” their software.
Besides, as Matt Sicker, Apache Software Foundation member and CloudBees’ senior security engineer, said, “Applications commonly used for signing software typically have confusing UIs and require learning basic cryptography concepts in order to properly use them. Without some sort of code signing policy in place for a larger open source project, many developers are simply unaware of the benefits of signing their software.”
Because of that, what tools there are for confirming the origin and authenticity of software relies on an often disparate set of approaches and data formats. The solutions that do exist, often rely on digests that are stored on insecure systems that are susceptible to tampering.
Newer, better signing tools are on their way. For example, Tidelift-managed catalogs track well known-good, proactively maintained components that cover common language frameworks such as JavaScript, Python, Java, Ruby, PHP, .NET, and Rust.
Even so, very few open-source projects currently cryptographically sign their software releases. That’s largely because of the challenges software maintainers face on secure key management, key compromise/revocation. and the distribution of public keys and artifact digests. Users are all too often left to fend for themselves to find out which keys to trust and how to validate signing. That is not a job for ordinary IT people.
But, wait, there’s more. The ways we currently distribute digests and public keys is, in a word, bad. All too often they’re stored on hackable websites or a README file on a public git repository. That’s just asking to be hacked. Sigstore seeks to solve these issues by utilization of short-lived ephemeral keys with a trust root leveraged from an open and auditable public transparency log.
In other words, as Alex Karasulu, also an ASF member and OptDyn CEO, observed, “The problem isn’t that open-source developers are lazy or reluctant. It is that a standard mechanism for two-factor authentication (2FA) specifically around code signing does not exist. Some techniques exist to achieve this: Git revisions can be signed and the process loosely protected with mandated 2FA accounts at GitHub, or GPG code signing keys can be stored on devices requiring a second factor to digitally sign anything including code and release checksums. There are many ways to skin this cat — but there is no standard making the process consistent. It’s essentially discretionary.”
Without standardization, securing the software supply chain will be almost impossible. It’s sigstore backers’ hope that they can fix these issues. The goal is worth the effort. As Josh Aas, executive director of the Internet Security Research Group (ISRG) and Let’s Encrypt, said “Securing a software deployment ought to start with making sure we’re running the software we think we are. Sigstore represents a great opportunity to bring more confidence and transparency to the open-source software supply chain.”
There is, after all, as Santiago Torres-Arias, Purdue assistant professor of Electrical and Computer Engineering and project founder, pointed out, “The software ecosystem is in dire need of something like it to report the state of the supply chain. I envision that, with sigstore answering all the questions about software sources and ownership, we can start asking the questions regarding software destinations, consumers, compliance (legal and otherwise), to identify criminal networks and secure critical software infrastructure.”
We really need sigstore. Even now, we still haven’t really grasped how bad the Solarwinds disaster was. Without a truly secure open-source supply chain, we can be certain we’ll see even worse disasters.
