Notebook · Post-mortem
Lessons paid for twice
A post mortem from a product in the workshop, unnamed until it’s ready. Three lessons I learned the expensive way, twice.
Draft for review
Before the current version of one product in my workshop, there were two earlier prototypes. Both taught me the same three things. I only listened the second time, so these are lessons paid for twice.
- Use real tool calls, not pattern matching. The first prototypes guessed intent from text with rules. They worked on the examples and broke on people. The current version lets the model call real, typed tools, and checks every call.
- Stubs must fail loudly. A placeholder that quietly returns success hides the gap until a person finds it. Every stub now throws, with a message that says what is missing.
- Never report a false success. If the system isn’t sure something happened, it says so. “Every claim here is one the product can keep today” became the copy rule for the whole product.