Second Avenue
A data product that warns a consignment retailer when contracts are falling behind target, while there's still time to act. I pre-registered the test, and it said my first bet didn't hold.
The flag was judged on Ops cost as well as accuracy. About 23 flags a week at 10 minutes each was three hours of review time, mostly false alarms.
| False alarms, first bet | 80% |
| False alarms, after the fix | 36% |
| Warning before contract ends | 30 days, vs 0 today |
| Question layer, Sonnet | 30/30 at $1.08 |
| Question layer, Haiku | 29/30 at $0.05 |
Fictional company, simulated data of 1,968 contracts.