Semantic Versioning Will Not Save You

The widely used Python package cryptography changed their build system to use Rust for low-level code, which caused an emotional GitHub thread. Enthusiasts of 32-bit hardware from the 1990s aside, a vocal faction stipulated adherence to Semantic Versioning from the maintainers, claiming it would’ve prevented all grief. I will show you not only why this is wrong but also how relying on Semantic Versioning hurts you – the user.

Dedicated to Alex and Paul who are willing to take the heat for the rest of us.

[This article has been translated to Russian: Семантическое версионирование вас не спасет]

Version Numbers

Let’s set the stage by laying down the ultimate task of version numbers: being able to tell which version of an entity is newer than another. This can apply to anything, but we’ll focus on software packages here.

The software community has settled on interpreting version numbers as tuples of integers, separated by periods, with a precedence from left to right. Therefore 2.0 is newer than 1.10.0, which is newer than 1.9.42. The Python community has PEP 440 to formalize that.

And that’s all there is as far as this article is concerned: version numbers are unique, orderable identifiers of software releases.

Semantic Versioning

Over the years, well-intentioned people experimented with adding meaning to those numbers. The arguably most popular take is Semantic Versioning (SemVer). You have MAJOR.MINOR.MICRO and the promise is that as long MAJOR doesn’t change (aka a major bump), nothing will break and you can update your dependencies without prejudice. Unless MAJOR is a zero, which means YOLO time for the maintainer: anything goes.

This article is not about discouraging maintainers from using SemVer for their projects if they like it. There’s nothing wrong about encoding the intent of a release into the version number as a service to its users.

It’s entirely about unrealistic expectations of users and the consequences thereof.

Hyrum’s Law

Let’s start with broken promises and of course there’s an xkcd about it:

It channels one of the fundamental laws of software development:

With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.

— Hyrum Wright, Hyrum’s Law

What this means in practice is: Even if the maintainer is pure of heart, extremely diligent, and super conservative with what constitutes a breaking change¹, it is impossible to predict the ways a change can affect your users.

You want to claim that version 3.2 is compatible with version 3.1 somehow, but how do you know that? You know the software basically “works” because of your unit tests, but surely you changed the tests between 3.1 and 3.2 if there were any intentional changes in behavior. How can you be sure that you didn’t remove or change any functions that someone might be calling? That bug that you just fixed – what happens to people relying on the old behavior or working around it?

There’s obviously some nuance to this claim. I’m writing this from the perspective of a Python and Go programmer. And Python packages – through Python’s dynamic nature – are much more likely to suffer from breakage caused by unintentional side-effects. On the other hand, I’m currently dealing with the fallout of unintended incompatibilities between two minor releases of a C library.

In essence, relying on updates not breaking if the maintainer doesn’t intend it, means relying on software being bug-free.

This does not mean that SemVer is bad or without merit. Knowing the intentions of a maintainer can be valuable – especially when things break. Because that’s all SemVer is: a TL;DR of the changelog.

What it does mean though, is that you can’t rely on the semantic meaning of SemVer and you must treat every update as potentially breaking. If a bump of the micro version never broke your production app, you just have to wait a bit longer.

On the bright side, I’ve seen major bumps come and go without affecting me at all. A major bump can only tell you about the existence of an intentional breaking change – but nothing about the impact, because it lacks the granularity.

Taking Responsibility

But some things have to be done. It’s better to do them, than to live with the fear of them.

— Joe Abercrombie, The Blade Itself

The only person who is responsible for the health of your application is you. Your customers aren’t going to be understanding if you tell them that they can’t access their data because some teenager on a different continent broke your workflow by not adhering to SemVer as strictly as you’d like them to.

Pinning the major version number does not avoid your breakage. The best you can hope for is a temporary postponement by slowing down updates. Postponing problems is generally a horrible idea because most problems only get worse the longer you neglect them.

In practice that means that you need to be pro-active, regardless of the version schemes of your dependencies:

1. Have tests with good test coverage.

2. Pin your dependencies and their transitive dependencies to their exact versions.

   Go’s modules, Rust’s Cargo, and JavaScript’s npm do that by default. In Python, I use pip-tools and PDM, but a plain pip freeze is better than nothing. You must separate your requirements that say Flask from your pin files that say Flask==1.1.2, along with Flask’s dependencies, and ideally their hashes. Otherwise, every build is a lottery.

3. Regularly try to update your dependencies to their latest versions. There are tools that help you with that.

4. If your tests pass, pin the new versions. If they don’t:

   • Fix them, then pin & commit the new versions.

   • If a single version of a package is broken due to a mishap and the maintainer intends to fix it in the next release, block the specific version from being considered for updates (e.g. Flask!=1.1.2, but not Flask<1.1.2).

   • If a package has intentionally made significant backward-incompatible changes and incremented their major version, block that major version (e.g. Flask<2), but – unless you have a contract about long-term support of the old major version – start working on adapting to the new major version immediately. Or look for alternatives.

     This is the only acceptable situation to pin the major version and it’s strictly temporary.

5. GOTO 3

Depending on the amount of churn, you can use the same process for open source packages. Tools like Dependabot will help you.

And that’s it. This is what you have to do to prevent third-party packages from breaking your project or even your business. Most of the people that were angry at the cryptography maintainers about breaking their builds didn’t properly execute step 2.

There is nothing a version scheme can do to make it easier. It can only help you determining whether the breakage is on purpose or not.

This is also true for ecosystems like Rust or Go, that have SemVer baked into their packaging toolchain². And most of the unintended consequences that I’ll enumerate next apply to them just as well.

Unintended Consequence: ZeroVer

Maintainers often feel like there’s an obligation to do SemVer. Some of that is self-imposed by not knowing about alternatives or thinking that “it’s the way it’s done”. Some of that is the result of demands by entitled users. And while SemVer promises freedom (technically you can break compatibility with every release, as long as you increment your major version!), in reality it delivers additional pressure and work.

The observable result of that pressure is what’s tongue-in-cheek called 0-based Versioning.

I’ve mentioned that in SemVer the maintainer can do whatever they want, as long as the major version is zero. That leads to many maintainers sticking to their beloved zero forever.

The SemVer standard clearly states that a package fit for production should be a 1.0:

How do I know when to release 1.0.0?

If your software is being used in production, it should probably already be 1.0.0. If you have a stable API on which users have come to depend, you should be 1.0.0. If you’re worrying a lot about backward compatibility, you should probably already be 1.0.0.

— Semantic Versioning, FAQ

But unfortunately, incrementing the major version is culturally frowned upon³. So people stick with 0ver and the version number means absolutely nothing while claiming it’s semantic.

Thus, a package that has a 0ver version and at the same time claims to be production-ready is a paradox.

I don’t write that to throw shade on projects. I write that to demonstrate that SemVer is not a good fit for most projects and adds to maintainer burnout.

Unintended Consequence: Lack of Security Updates

The reason so many people were angry at the cryptography maintainers is that they are convinced that if only cryptography adhered to SemVer, they could just pin on the major version and nothing would ever break.⁴

As I’ve shown above, the “not breaking part” is nothing but a false sense of security and wishful thinking.

The pinning part is even worse, though: Most open source projects don’t have the capacity to maintain multiple major branches.

Therefore the moment you pin the major version of a package, it usually means that you won’t get any updates whatsoever once the package bumps its major version. In the case of security-sensitive projects – like cryptography but also web frameworks and their dependencies – this has potentially catastrophic consequences⁵.

Unlike npm, Python mainstream packaging tools have no concept of vulnerable versions. You need extra tools or services to achieve that. That means that if you pin the major versions of your dependencies, your application will eventually be full of CVEs and you’ll never learn about it⁶.

But Wait – It Gets Worse!

If you maintain a public package and pin the major version of a dependency of yours, you transitively do this to the applications of your users.

Imagine an application depends on the wonderful urllib3 and your package does too. Now if you pin urllib3 to <2, the user of your package doesn’t have it in their power to ever receive an update from urllib3 again, once urllib3 bumps its major version to 2 and beyond.⁷ They may not even realize how far back they are.

On the other hand, if a new major version of a package surprisingly breaks your package, they can always add a pin themselves (see step 4 above) until you fix your package. But there’s no practical way for them to remove your pin.

Some Python packaging tools have adopted npm’s major-version pinning (^) by default, despite the lack of npm’s security features and despite Python’s flat package space. Make sure to unpin them by hand if possible.

Unintended Consequence: Version Conflicts

This is a related problem to the last one. In languages with a flat package space that only allows one version of a package to be installed at a time⁸, speculative pinning of the major version of packages that you don’t control will inevitably cause unnecessary version conflicts to your users that they can’t fix themselves.

At least your users will know what’s going on. But they won’t be happy about it and it may bring some negative vibes to your tropical vacation, financed by the millions you’ve made from maintaining a FOSS project.

Of course, this risk of version conflicts raises the stakes to bump major versions, which in turn leads to more ZeroVer and more arguments about why a breaking change isn’t actually breaking.

Summary

If you’re a maintainer and you like SemVer as an extra service to your users: go wild! I’m not here to tell you how to spend your time. There is value to adding semantic meaning to versions.

I am however here to tell you that applying SemVer to your project is entirely optional and if it stresses you out, or you’re stuck in 0ver land forever (meaning it does stress you out, but you don’t notice or acknowledge it), consider some of the alternatives.

As a user, I hope I have shown you that relying on SemVer:

• … can’t prevent breakage. At best, it can postpone it. Which is worse.
• … leads to version conflicts that will make your users unhappy.
• … leads to security problems that will make your boss and your customers unhappy.
• … adds burden on maintainers that will make the maintainers unhappy.

There’s also plenty of high profile projects that look like SemVer but aren’t: Linux, Python, Django, glibc… it’s fine!

Further Reading

• Version numbers: how to use them? by Bernát Gábor.
• Should You Use Upper Bound Version Constraints?⁹ by Henry Schreiner III.