AI writes most of the code we ship. We say that plainly because it is true, and because the interesting question is not whether a model wrote a line but who is answerable for it once it runs on a store taking real orders. Our answer is simple: a named senior engineer, every time, on every merge.
The team behind StoreSmith has shipped on Shopify Plus for years. The process below is what we kept after trying the alternatives.
the process, end to end
Every change goes through the same five steps. Small changes go through them quickly. None of them skip a step.
1. a written spec
Nothing starts from a chat message. A change starts as a short written spec: what the store needs, what must not change, the data involved, and how we will know it works. For a webhook handler, that means naming the topic, the fields we read, what happens on a duplicate delivery and what happens when the downstream system is down.
The spec is the most valuable thing in the process. A model given a vague request produces confident, plausible code that solves a slightly different problem. A model given a precise spec produces a reviewable first draft. Writing the spec is engineering work, and it is done by an engineer.
2. an AI draft
The model writes the first draft against the spec, in the repository, with the existing code and tests in context. It is good at this. It is fast at boilerplate, consistent with patterns it can see, and tireless about writing tests when asked.
It is also wrong in ways that look right. It will call a field that existed two API versions ago. It will handle the happy path of a webhook and forget that Shopify can deliver the same event more than once. It will write a retry loop with no ceiling. None of that is a reason not to use it. It is the reason the next step exists.
3. a line-by-line review of the diff
A senior engineer reads the full diff. Not a summary of it, and not the tests alone. Every line.
We review against the spec and against a short list of things that go wrong on live stores:
- Idempotency. Can this run twice without double-charging, double-tagging or double-syncing? Webhooks are delivered at least once, so the answer has to be yes.
- Data boundaries. Does it read only what it needs? Does it log customer data it should mask?
- Failure paths. What happens on a timeout, a rate limit, a malformed payload, a partial write?
- API surface. Is every field real in the API version we pin? Is anything deprecated?
- Blast radius. If this is wrong, what breaks, for whom, and how fast would we notice?
A typical review comment looks less like style feedback and more like this:
// before: processes every delivery, including duplicates
await syncOrder(payload);
// after: dedupe on the webhook id before doing any work
if (await seen(webhookId)) return ok();
await markSeen(webhookId);
await syncOrder(payload);If the engineer cannot explain why every line is there, it does not merge. Often the fastest fix is to throw the draft away, tighten the spec and generate again. That is fine. Drafts are cheap. Production incidents are not.
4. a signed merge
The engineer who reviewed the change merges it under their role handle: lead.apps, lead.checkout, lead.themes or lead.data. The signature is not decoration. It means one person read it, understood it and is answerable for it. If something breaks, we know who to ask and they know the code.
We use role handles rather than personal names on the website for now. Inside the work, every merge maps to one accountable engineer.
5. a guarantee behind it
Work we ship is covered by a 90-day guarantee: if something we built breaks because of how we built it, we fix it. A signed merge is what makes that guarantee something we can offer without crossing our fingers.
what we refuse to automate
Some decisions stay with people, regardless of how good the tooling gets.
Deciding what to build. Models are poor judges of whether a feature should exist. Scoping is a conversation with the merchant, and we often talk people out of builds. If a native Shopify feature or a maintained app does the job, we say so.
Review. We do not use a model to approve another model’s code. Automated checks run, and they catch real things: type errors, lint, failing tests. But approval is a human reading the diff.
Anything that touches money without a human in the loop. Refunds, price changes, discount logic, payment customisation. The code can be drafted by a model. The rules and the review are ours, and we test them against real carts.
Production access. No automated agent holds credentials to a live store. Deploys go through the same pipeline as every other change, triggered by an engineer.
Talking to the merchant. When something goes wrong, a person explains what happened, what we changed and what we are doing so it does not happen again.
where does AI genuinely help?
We are not sceptics. The model saves time on the parts of the job that used to eat it: scaffolding an app, writing GraphQL queries against a schema it can read, generating test fixtures from real webhook payloads, migrating code across API versions, writing the first pass of documentation. It is also a decent second reader. Asking it to find edge cases in a spec regularly surfaces one we missed.
What it does not do is understand a particular store. It does not know whether a merchant’s warehouse team relies on a particular order tag, or which courier feed sends malformed status updates. That context lives in people, and it is what turns a plausible change into a safe one.
why this matters to a plus merchant
Large Plus stores are full of code nobody remembers writing: integrations, scripts, apps from a previous agency. When something breaks, the first question is always who knows how this works. On our work, the answer is on the merge.
If that is how you would like custom work on your store to be built, talk to an engineer.