If you’ve used computers long enough to leave a project for a long while and come back to it, you may have had the “fun” of resuming, only to find that, despite your having changed nothing, it no longer works. We computer geeks call that “software rot” – as if the bytes on the storage drive had decomposed over time.
What happened is that you didn’t change anything, but someone else did. The operating system got an update. Or, on a shared system, somebody updated software that you had previously used – or, indirectly, something you didn’t know about that, behind your back, was being used by the software you did know you used.
This started happening to me last week, and I am delighted that today, after about seven hours of work over that time, I got things working again.
What Happened To Me
For the Linux software geeks (if any) among you readers, this is what happened.
Back in 1972, in grad school, I had an idea for a book about the fundamental ideas in Computer Science. Finally in 2020, after many hard times including 15 years of disability leave, I got a serious stab at writing it. I was on a 6-month stay-at-home sabbatical. For Reasons, I was using LaTeX on Solaris, with the 2016 version of TeXLive. I added to the book bit by bit over the next few years, but completely stopped in the latter half of 2023, at the end of which I had to retire (again, for Reasons). I finally tried to pick it up earlier this year, but software rot had set in. The Solaris system had been updated, and LaTeX (or at least some of the LaTeX packages I needed) no longer worked. Almost nobody was still using that software, so the technical staff suggested I move it to a Linux release (Ubuntu under WSL) on my office Windows computer.
So last week I started that: set up WSL, create a user account under Linux, transfer files from the Solaris machine, discover seven things I needed to update or install on Linux (three of which didn’t completely finish, but seemed to work anyway), and finally try to run LaTeX to recreate my document from scratch.
Compilation failed with status code 2.
TeXLive was updated in June, and Things Are Different Now.
I spend about four and a half hours this week tracking down the bugs, partly by experimenting, partly by sifting through (mostly) out-of-date Reddit and StackOverflow suggestions (finally finding a GitHub fork of a critical package that fixed some issues in what I had been using), partly by just rewriting the bits that failed. Finally, today, the old document compiled and looked great.
Then I spent a glorious half hour adding new prose. It was the best session of “scholar / writer / computer geek” (all parts together) in four years.
What Happens to Others
Something like this might have happened to you the last time your operating system forced an update on you.
Operating systems used to allow you to delay (or outright refuse) to update, but historically that led to Russian bot networks hacking your machine because of security flaws that had been patched months or years ago that you never installed. So now, Windows (at least, and likely others) makes it really, really hard to avoid updating.
And that’s a problem for any big company with a public-facing service. It is completely impossible for even a very careful operating system company to test a new release under every circumstance that users might create. So there is a nontrivial chance that something will go wrong the instant your company’s public-facing servers update.
You can’t afford to let that happen.
I have a writer acquaintance who is also in the computer industry, and his company faces exactly this situation. Every time Windows updates, they have to stop the public-facing servers from updating, and try the new version in a “sandbox” not available to the public. They then spend a frantic time finding and fixing whatever broke before updating the public servers.
This whole thing is part of configuration management: the discipline of keeping track of all your bits and pieces, and deciding when an update should happen. I worked for a startup for about 13 months during 1983-84, and the configuration manager had the perfect “passive aggressive” personality required by the job. He spent a lot of his time telling programmers “You don’t want to upload that fix to the shared version. Yes, it fixes a bug. But five of your colleagues are tracking down other bugs, and they’ll have to start over if your fix changes anything at all that affects their problems. And you don’t know that your fix doesn’t change anything that matters to them.”
He was probably the most critical employee we had. It’s a job where you get a lot of pushback and pressure, but it’s essential to your production timeline.
