A business with nothing to advertise now makes its own campaigns, from its own shelves, overnight, for about ten cents a product.
Context
My father's lighting business in Willemstad. Family, not a client, and worth saying out loud. About eight hundred products on shelves, no catalogue of any kind, and everything anyone knew about them living in their heads. A hundred and eighty-seven are live in the system today. I designed it and made the calls.
The problem
Four problems in a row, each one caused by the limits of the last. Find the products inside ordinary shop photos. Turn those crops into images good enough to sell with. Describe each product the way a shopper would. Then build a flow that never asks a question the catalogue can't answer well.
Design goals
Use each model only where it beats the alternative. Nothing in the system does more than it can justify. And one hard rule at the front: a customer must never hit a dead end. Six imperfect results beat zero good ones.
Key decisions
A language model sits in front of the object detector and writes the list of things to look for in each photo. A fixed list does badly in a shop that has bulb pegboards, chandelier displays and outdoor floodlight racks in one session. Detection runs on the compressed image; the crop is cut from the original. The detector gains nothing from the extra quality and the crops gain everything, because they feed every step after. The backdrop is grey, not white, because the tool that cuts products out later cannot tell white frosted glass from a white background. A problem at the end decided a choice at the start. And two cutting-out tools run together rather than one tuned tool: one catches the whole outline but leaks through clear glass, the other understands what the object is but loses its dark parts. Together they work.
The data model, designed backwards from the interaction
Every lighting catalogue online is sorted by specification: wattage, lumens, colour temperature, bulb fitting. That is how an electrician shops. It is not how someone whose hallway feels like a hospital shops. So each product gets scores rather than tags. A pendant is great over a kitchen island and merely fine in a dining room, and a yes/no tag turns that into a lie. One rule follows from it: never filter on those scores, sort by them. A filter can return nothing. A sort always returns something. The specifications are still there and still correct. They live on the product page, not in the first four taps.
What didn't work
I tried to make a light appear switched on, using reference photos. It failed, and I can say exactly why. Image-editing models copy the shape of the reference along with its lighting. A diamond-shaped reference put forty-five-degree diagonal cones of light on round fixtures. And the only reference that worked well was almost identical to the product it was applied to, which defeats the whole point of a small reference set covering many products. Parked, with the next experiment written down. Separately, finding products automatically on dense display boards failed across three different models. So I built a tool to draw the boxes by hand and timed it: about five minutes per thirty-product board. Picking the human step over a broken model was right. Knowing what that step costs is what makes it a decision.
Where the data overruled me
I had picked period style as one of the first questions to ask a shopper. Then I counted: ninety-one percent of the products fell into two of its ten options. Asking it would have told us almost nothing, so it was cut. The question order got rebuilt from what the real stock could actually distinguish, rather than from what felt like a sensible funnel. I was wrong and the catalogue said so.
What this demonstrates
Four different skills in one system: finding things in images, generating images, pulling structure out of them, and designing something a shopper can use. Each used where it is strongest. The costs are measured, not guessed. And what failed is written down with the reason, which is the part you can actually check me on.
Python · YOLO-World · BiRefNet · Grounded-SAM · Generative image models · Structured LLM extraction · Next.js · Supabase · TypeScript