Edo Sanjaya PerkasaSearch ranking & decision supportBandung, ID
I build one scoring model, and I name where it stops working.
Scored rankings that come with a leak-free split, a baseline the model has to beat, metrics that reproduce from a clean clone, and failure modes named with a fix beside each one.
found in my own build
3,388 pages could not fall, and the model was scoring them anyway.
Mine ranked 30,000 pages by risk of losing search traffic. 3,388 had zero impressions the month before. A page on zero cannot fall — those rows were answered by arithmetic, not by the model.
I found it myself. I had already published a number that rested on it.
what the fix cost
Removing the rows took the result apart. The record stays public.
Measured after the zero-impression rows were removed. A coin flip is 0.500.
Re-measured at 0.600 against a base rate of 0.611 — below the base rate. Withdrawn rather than restated.
The split was measured, not assumed. Holding out whole clients cost 0.007 — now I know the number instead of claiming it.
what I do about it
Four rules, applied to every build.
- Hold out whole clients, then measure what it bought.
- Quote only numbers that reproduce from a clean clone.
- Name the limits in the repo, with a fix beside each one.
- Label every figure with the slice it came from.
read the work
Where the evidence is.
the code itself