A system that learns from your frustration
Build logEntry 01By Jonathan Uhlemann, Matthias Bigl and Ferdinand Rubenbauer
Under every Piloti answer sits a button we hope you never need. Thumbs down. Bad answer. In most products, that click files a report and that is the end of it. Maybe it felt good for a moment. Nothing changed.
That bothered us. Our internal working rules contain a sentence we keep quoting at each other: when a human has to step in, the machine has failed. If that is true, the correction should be worth more than a bar in a dashboard nobody opens on Fridays.
So we built the loop. A failure that has been reported once should not happen a second time.
↺ EVERY ANSWER CARRIES IT → 01
What happens after you click
You pick a category, write a sentence about it if you like, and send. From there, everything runs without you.
First the report is scrubbed by machine. Names, addresses, project numbers, email addresses, anything that could identify you or your project is removed before anything else happens to it.
Then a model reads the scrubbed report and looks for the general mistake behind the specific complaint. Not “the answer about project X was wrong” but something like: “When a question touches two federal states, their building codes must not be merged.” Short, general, boring on purpose. A good lesson contains nothing of the conversation it came from.
A second, independent model audits that draft. It never gets to see your original report, only the candidate, and it rejects anything that still smells like a single case.
Finally the lesson is compared against every lesson that already exists. Fifty offices tripping over the same behavior become one lesson with a counter at fifty. Not fifty lessons.
Every answer knows it
The active lessons ride along with every answer Piloti writes, for every organization on the platform. Your annoyance in Vienna improves an answer in Vorarlberg. Nobody sees your report. Everybody benefits from it.
We think that is only consistent. The same mistake, made twice, is no longer an accident. It is a decision.
Why we call it a bandage
Now the honest part. A lesson tells the system “be careful here”. It does not fix the underlying cause, which may sit in a knowledge gap, a missing document, or a weakness of the model. In our platform view, every lesson therefore visibly carries the label of a symptomatic treatment, with its own field recording whether the cause has been found and fixed. The bandage spares the next user the mistake. It does not close the file.
And we measure whether lessons work instead of believing it. A small share of conversations deliberately runs without them, as a control group. Counters record whether answers with a lesson attached collect fewer thumbs-downs than answers without. A lesson that improves nothing shows up that way.
Two more automatisms keep the loop honest. A lesson that keeps collecting new reports while it is active gets flagged as not holding, because a bandage that does not stop the bleeding needs a doctor, not more gauze. And once the root cause behind a lesson is fixed and two weeks pass without a single recurrence, the system retires the lesson on its own. The bandage comes off when the wound stays closed, and both steps land in the audit trail like everything else here.
What never leaves your project
Your report is scrubbed before any model distills it. The auditing model only sees the anonymous draft. The stored lesson contains no names, no projects, no quotes. And every lesson has a complete history: when it was created, how often it was confirmed, when it was retired. All of it is inspectable, none of it happens in the dark.
How the answers this loop works on are produced in the first place is covered in How Piloti works.
So the next time an answer annoys you: click the button, write the sentence. It costs ten seconds and is the most direct way to make the product better. Frustration is valuable raw material. Now there is a machine that refines it.