All issues

Hot Path #1

Alpine's sshd let in any password long enough to break crypt()

Today's issue: the asterisk that let long passwords into Alpine's sshd, a mutex that still loses commits, and a database result that beats the time it takes to read the data.

Welcome to #1. I started Hot Path because the backend, systems and AI stories worth knowing are more useful with the mechanism attached. It comes twice a week, on Tuesdays and Fridays. If it isn't for you, one click at the bottom unsubscribes you from Hot Path and nothing else.

A password over 256 bytes opened key-only accounts on Alpine

A long password going into a hash function and a single asterisk coming out, matching the asterisk stored for an account

On November 5, OpenSSH shipped 10.6p2 to close CVE-2026-123158. On Linux with musl libc and no PAM, sshd let a password of 257 to 1,024 bytes into any account whose stored password field is *, the usual value for key-only accounts. Password logins are on by default, and the only other requirement was one account on the machine with a $5$ or $6$ hash.

Without PAM, sshd hashes your attempt with crypt(), using the stored hash as the salt, and compares the two strings. A * is too short to be a salt. Since a 2016 fix for a timing leak (CVE-2016-6210), sshd then borrows another account's salt, so the check takes as long as a real one.

With a $5$ or $6$ salt, musl runs SHA-256 or SHA-512. In August 2012 musl stopped returning NULL from a failed crypt(), because many callers never checked for NULL, and returned an unmatchable string instead. Twenty days later its new SHA code arrived with a 256-byte limit on the key. Past the limit, crypt() fails and returns that unmatchable string: "*". sshd compares it with the stored "*" and lets you in.

The failure value and the "no password" marker are the same character.

Alpine's root is root:*, but the default PermitRootLogin prohibit-password keeps password logins away from it. The exposed accounts are other key-only users, directory users that come through musl-nscd, and the alpine user in the official tiny-cloud images from 3.18.11 to 3.24.3, which has passwordless doas.

The reporter, Özgün Kültekin of Wakefield, says agents sent about 17,600 unusual logins to sshd builds, and only musl without PAM let the wrong ones in. OpenSSH's release notes say many recent reports come from AI tools and that fixes will ship in more frequent releases.

What I'd take from this one: an error value that looks like real data will eventually be compared with real data. Here it took ten years.

What changed:

  • sshd refuses short stored hashes. In 10.6p2, a stored field under 4 characters never passes, whatever crypt() returns.
  • Alpine, built on musl, patched 3.16 to edge the same day. 3.24 gets openssh 10.3_p1-r2 and edge gets 10.6_p2-r0. The running sshd keeps the old code until it restarts.
  • musl may stop returning "*". Rich Felker said on November 9 that crypt() will possibly return NULL on failure from the next release cycle, which is what it did before 2012. He advises against backporting the change, since callers may break.
  • The logs show past use. Alpine says to search /var/log/messages for Accepted password for an account that has no password.

Bottom line: on any musl machine running sshd, check sshd -T | grep -i passwordauthentication, and if it says yes, upgrade openssh and restart sshd, or set PasswordAuthentication no.

From the house

One LLM feature, from the first call to real users

My newest course, Building with LLM APIs, follows one feature: the "Check my writing" button in Inkstep, a writing-practice app. Its 18 lessons cover versioned prompts kept in code, structured output, streaming, retries, token budgets, tests that don't call the model, and moving to a new model. The "Shape is not truth" lesson fits today's story: every check returns typed corrections or a named failure, never an error that looks like an answer. Start the course.

Spot the race

This moves a branch like git push without --force: it refuses if someone moved the branch first. Every map access holds the mutex, and go test -race passes. How can two pushes from the same starting commit both succeed?

type Store struct {
	mu   sync.Mutex
	refs map[string]string
}

func (s *Store) Get(ref string) string {
	s.mu.Lock()
	defer s.mu.Unlock()
	return s.refs[ref]
}

func (s *Store) Push(ref, old, next string) error {
	if s.Get(ref) != old {
		return errors.New("stale: fetch first")
	}
	s.mu.Lock()
	s.refs[ref] = next
	s.mu.Unlock()
	return nil
}

The race, explained

Push takes the lock twice, once inside Get and once to write. In between, the other goroutine runs its own Get. Both read the old commit and pass the check, then both write. The second write replaces the first, so one pusher's commit is gone though its Push returned nil. Every map access is under the lock, so the race detector stays quiet.

The fix is one lock around the compare and the set: lock at the top of Push, read s.refs[ref] directly, then return the error or write next. That is a compare-and-swap, which GitHub's Ref-CAS does per ref. Two pushers over 2,000 rounds were both accepted 13 to 34 times per run and never in 20,000 with the fix, so a concurrent test that asserts one winner catches it.

[01] Share

Know someone who would like this one? Send it their way.

[02] Subscribe

Hot Path

Get the next issue by email

Two issues a week, Tuesday and Friday mornings: one story from backend, systems or AI told with its real numbers, a bug to spot, and the links worth reading.

Every issue has a one-click unsubscribe link.

Unsubscribed before? Use ‘Manage preferences’ in any Hot Path email to come back.