What my AI company did this week
Week three of the weekly recap: what my AI employees shipped, what broke, and what got fixed.
The company stopped checking on things that can't move
A lot of runs this week ended with the same line: 'swept the board, nothing workable'. The scheduler counted that as progress, so it kept waking the same employee every few minutes to re-read tickets that were waiting on traffic, a date or me.
That got fixed on Tuesday. A run that only says 'I looked, nothing changed' no longer counts as progress, so the scheduler backs off. A ticket that sits still for three runs in a row now parks itself for three hours. And every blocked ticket has to say what it is waiting on: me, another employee, a date or something outside the company.
On Friday the CEO employee ran a retro over the last two weeks of company data, 6,207 runs and 466 questions to me, and put a number on the problem. About one run in nine had ended with nothing to do. The worst offender was my own portfolio's A/B test, re-checked 455 times while it waited for visitors.
Also new this week: the employees can now ship changes to HQ's own code themselves. They submit a patch, automatic checks run, I tap once to approve, and a backup, a health check and an automatic rollback are built in.
The A/B test is over
The portfolio ran a 50/50 test of two versions of its homepage hero for two weeks. Result: zero conversions on either side. Digging in, almost half of the recorded visitors were bots, crawlers and HQ's own screenshot tool. Those are filtered out now.
Even with clean numbers, the site gets around five real visits a day, and the thing being measured is one click on the 'Email me' button. No test reaches a verdict on that in any sensible time. The mechanism stays switched on in case traffic grows.
New on the portfolio
The contact card now has a proper contact form next to the email link. It is spam-protected and messages land directly in HQ's own contact list.
What broke
A server ran out of memory. On Saturday morning one of my servers filled its memory and took a batch of preview sites and tools offline for several hours. The employees caught it early, but the safety rule that only allows restarts during the night maintenance window also blocked the emergency restart, so the fix took longer than it should have. It came back with more memory.
A review on Tuesday then looked at every 'this server is down' alert from the past two weeks. Five of six were one monitor pointed at the wrong address. The sixth was a real configuration bug that had already been hardened. That server now checks its own configuration every hour, so the bug can't come back unnoticed.
One leftover line of code. A feature was removed from the HQ wall screen, but one line that still called it stayed behind and threw an error on every live update. The automated checks caught it overnight and kept filing incident tickets, six of them for the same line, and the open incidents kept the wall from dimming at night. One-line fix the next morning, checked on every wall tab.
Sleeping apps look like dead apps. Two of my apps now sleep overnight to save power. The monitoring didn't know about it, so a few late-evening alerts turned out to be apps sleeping on schedule, and one employee woke them up again before checking. The employees now look for a planned shutdown before calling anything an outage.
Security, the quiet kind
- The nightly patch run found a curl security update on 14 servers, and an SSH package on another server that was two security releases behind. All applied and checked.
- Two more servers now install security updates automatically.
- Three internal services that any device on my home network could reach directly are now closed off. They only answer through the proper front door.
- My main domain now publishes which certificate authorities are allowed to issue certificates for it.
- A new pilot called Paper Trail keeps a tamper-evident log of what the AI employees do. Each line carries a fingerprint of the line before it, so editing an old entry breaks the chain. It runs for one employee as a test, and a deliberate edit to an old line was caught.
That's week three. More next Friday.