How to judge an AI web design tool in ten minutes

If you search for the best AI for web design you get lists, and the lists agree on very little except their tone. That is because the things being compared — accuracy, speed, "the latest model" — are claims rather than mechanisms, and claims are free.

Six questions separate these tools properly. All six are answerable in a ten-minute trial, none of them appear on a feature page, and I will answer them for our own tool as I go so this is checkable rather than merely opinionated.

The six questions

1. Will it invent a fact?

The test: give it a brief with no prices in it, and ask for a pricing section. A tool with no rule about this will cheerfully produce three tiers at $29, $79 and $199, and they will look completely finished.

That is the most expensive failure in the category, because a made-up price does not look like a bug. Ask what stops it. "Our AI is very accurate" is not an answer; a rule is.

Ours: a price, number, name, quote, address or opening hour is kept only if it appears in the material you handed over, and a figure nobody wrote is refused anywhere on the page. Text too long for its slot is marked rather than silently cut.

2. Does it look at what it made?

Most do not. The model emits a page, the page is shown to you, and nothing in between has any idea what it looks like.

The test: ask whether there is a step that renders the page and inspects it. Then ask the better follow-up — what does it do when it dislikes the result?

Ours renders the finished page and has a job whose only purpose is to look at it. That job needs a model that accepts images, and it is advisory: when no such model is available it is skipped rather than faked.

3. Can it say no?

A tool that answers every request answers some of them badly. Refusal is a feature, and it is rare enough that asking about it is a good filter.

The test: ask for something it should not do — invent a testimonial, copy a competitor's page wholesale, state a certification you have not got. A good tool declines and says why.

Ours reads a competitor's URL as a brief only: the palette and the shape come across, the copy and the facts never do. Your own site is different, and that distinction is the whole subject of a separate article.

4. Does the model change with the job?

This is the question that predicts both your bill and your blandness.

One model for everything has two failure modes and you get to pick: a cheap model deciding your layout, or an expensive model writing your button labels. Neither is what you want.

The test: ask whether the model is chosen per request or per account. Ours is chosen per call, by the job — seven jobs in a page build, and only one of them asks for the strongest model available. The whole routing table is published.

5. Can it work on the site I already have?

Generating a new site and editing an existing one are genuinely different problems, and a surprising number of tools only solve the first. If you have a site already, this is the question that decides whether the tool is any use to you at all.

6. What do I have at the end?

A file you can host anywhere, or a page that has to keep running on their servers. Both are legitimate products; only one of them survives you cancelling.

Which builders give you what is worth ten minutes before you commit, not after.

The answers that should worry you

The pattern is consistent: a warning sign is a claim about the tool's quality, and a real answer is a description of a mechanism you can test.

"Unlimited AI" deserves its own note. Somebody is paying per call to a model provider, so unlimited means either a quiet cap you will discover at the worst moment, or a cheap model doing work you thought was going to the good one. A number is better news than the word unlimited, every time.

Mechanisms you can test, not adjectives. The fact firewall, per-job model routing, and a page it looks at before you do. Browse the Market.

What these questions deliberately leave out

They say nothing about how good the output looks, and that is not an oversight. Every one of these tools can produce one attractive page in a demo. What varies is the second page, the one you build on a Tuesday for a client who has their own colours and a price list — and the six questions above are exactly the ones that predict how that page goes.

They also leave out which underlying model a tool uses. That turns out to matter much less than how the tool uses it, which is why the model-by-model comparison is a shorter article than people expect.

The useful thing to take from this

Pick the two questions that match your actual risk and ask only those. If you are building for clients, ask 1 and 6 — invented facts and ownership are what get you fired. If you are rebuilding your own site, ask 5 and 3. If you are worried about the bill, ask 4.

And whatever a tool tells you, the test is the same and it takes one attempt: ask it for a pricing section without giving it any prices, and see what it does.