The xz Backdoor: A Multi-Year Con Caught by Half a Second of Latency

On Friday, March 29, a PostgreSQL developer at Microsoft named Andres Freund posted to the oss-security mailing list with a subject line nobody wants to read: a backdoor in upstream xz/liblzma leading to ssh server compromise. It's April 1 as we publish this, and to be clear: this is not a joke post.

xz is a compression tool, and liblzma is its library. It's one of those packages installed on practically every Linux system, rarely thought about, and maintained by a very small number of people. Someone spent a long time earning the trust needed to ship malicious code in it. They very nearly got away with it.

Analysis is moving fast and some details will change, so read this as the state of things over the weekend, not the final word.

What we know so far

• Affected versions: xz-utils 5.6.0 (released in February) and 5.6.1 (March). It's tracked as CVE-2024-3094. Earlier versions are not known to be affected, and distributions are rolling back to 5.4.x.
• Where it landed: mostly in fast-moving and pre-release distributions. Debian testing and unstable, Fedora Rawhide and the Fedora 40 pre-release, openSUSE Tumbleweed, Kali and Arch all shipped an affected version at some point. Stable releases such as Debian stable, RHEL and Ubuntu's current LTS are reported as unaffected. Check your distribution's advisory rather than trusting a summary, including this one.
• How it was hidden: the malicious code wasn't in the project's git history in readable form. It was assembled at build time from "test files" in the repository and a modified build script that appeared only in the release tarballs. Anyone reviewing the source on GitHub would have seen nothing obviously wrong.
• What it targets: the code only activates in specific conditions: x86-64 Linux, built as a Debian or RPM package, and loaded by OpenSSH's sshd. Upstream OpenSSH doesn't use liblzma, but several distributions patch sshd to support systemd notification, which pulls in libsystemd, which pulls in liblzma. That chain is the way in.
• What it does: early analysis says it hooks into the RSA key verification path in sshd. Researchers are still working out the details. The current reading is that it lets someone holding a specific private key execute commands on the affected server before authentication, which is about as bad as it gets for an internet-facing SSH daemon.
• Who did it: the changes came from an account using the name "Jia Tan," which had been contributing to xz for roughly two years and had become a co-maintainer. Volunteers are piecing together a timeline that appears to show other accounts pressuring the original, overstretched maintainer to hand over more control. Who is behind it is unknown.

GitHub has disabled the xz repository while this is investigated, and CISA has advised downgrading to an uncompromised version.

How it was found: a slow login

This is the part we keep coming back to. Freund wasn't auditing xz. He was doing performance work on a Debian unstable machine and noticed that SSH logins were using more CPU than they should, and taking around half a second longer. Valgrind had also been throwing odd errors. He dug in, rather than shrugging it off as noise, and found the backdoor.

A multi-year operation, designed to survive source review, was undone because someone noticed a latency change that most people wouldn't have. We've argued before that slow is the new down for user experience; here it's also a security signal. An unexplained change in how long something takes, how much CPU it uses, or what it connects to is worth investigating, even when everything still "works."

That only helps if you know what normal looks like. If you don't have baselines for response times and resource use, you can't notice when they shift.

Check your systems

Start with the version:

xz --version

Anything reporting 5.6.0 or 5.6.1 needs attention. Check the package manager too, since the library matters more than the CLI:

# Debian / Ubuntu
dpkg -l xz-utils liblzma5

# Fedora / RHEL / openSUSE
rpm -q xz xz-libs

# Arch
pacman -Q xz

And see whether your sshd loads liblzma at all:

ldd "$(command -v sshd)" | grep -i lzma

If you find an affected version on a machine exposed to the internet, update to your distribution's fixed package (which in most cases reverts to 5.4.x). Given what the backdoor appears to do, treating the machine as potentially compromised is the cautious choice, especially if SSH was reachable from the internet. Don't forget container images and CI runners built from rolling or testing distributions over the last two months.

The uncomfortable lessons

Critical infrastructure is maintained by volunteers. xz is a dependency of nearly everything and was, for most of its life, largely maintained by one person in their spare time. The attack exploited exactly that: a tired maintainer, and a helpful newcomer offering to share the load. We wrote about left-pad in 2016, when a single tiny package broke builds everywhere. This is the same structural problem, used deliberately.

Reviewing source isn't the same as reviewing what you run. The payload lived in release tarballs, not in the code people browse. Reproducible builds, and checking that released artefacts match the repository, would have made this much harder to hide.

Transitive dependencies are the attack surface. Nobody decided to link their SSH daemon against a compression library. It happened through a chain of reasonable distribution patches. After Log4Shell and last year's MOVEit campaign, "do you know what you're actually running?" keeps being the question that matters.

Luck isn't a control. This was caught early, before it reached most stable distributions, because one engineer was curious about half a second. It's fair to be grateful and still uncomfortable: there's no reason to think it's the only attempt of its kind.

Watch for things that don't add up

Most teams will never need to reverse-engineer a backdoor. But everyone can keep a record of how their systems normally behave, so that a change stands out. Response time history for your public endpoints, from CompleteStatus or any other monitoring you trust, is one small part of that. The xz story shows that "it's a bit slower than usual" is sometimes the most important alert you'll get.