AI Engineering vs. Vibe Coding

Sebastian Mertens · 20 September 2026

“Build me a Vercel, Supabase and Resend clone. Here are four screenshots. Build this now.”

That prompt can produce an impressive starting point. A dashboard appears. There are projects, deployment cards, database settings and a button for sending email. The result gives me something tangible to inspect, change and discuss.

But the request has left almost every consequential product decision to inference. Who is the customer? What is expensive about their existing workflow? Which capabilities must be reproduced? Which should be purchased from another provider? What happens when a deployment partially succeeds? How will I know whether the new product improves the situation that made me want to build it?

I approach AI engineering by making those questions part of the work. I start with the problem, investigate the alternatives, state the constraints and develop a plan I can understand. Then I use agents to implement that plan in small increments, with evidence that each increment behaves as intended.

The concrete example in this essay is hosting. I want to build and run professional applications without accumulating a substantial fixed bill before I know whether anyone will use them. I want PostgreSQL, serverless application execution and transactional email. I want a managed environment that lets me avoid administering virtual machines. I also want agents to operate the product through explicit interfaces. OhMyHost, the product I have been building, is the practical reference for this problem.

These requirements lead to a different project from reproducing the appearance of three existing products. They also expose the central distinction I want to make: engineering with AI means retaining responsibility for the problem, the decisions and the evidence that the result works. The amount of implementation delegated to a model can be very large.

Here, “AI engineering” means software engineering with AI assistance. The term also has an established use for developing applications with foundation models, as in Chip Huyen's book of that name. My subject is the development process, including products whose own functionality may contain no language model at all. Huyen, 2025

1 What the distinction actually describes

Andrej Karpathy introduced “vibe coding” in February 2025 through an account of casual development in which he accepted generated changes, returned errors to the model and allowed the implementation to grow beyond his immediate understanding. Simon Willison subsequently distinguished this from AI-assisted programming that includes review, testing and an ability to explain the result. Both accounts also recognize the value of quickly creating things that would otherwise never be built. Karpathy, 2025; Willison, 2025

For this essay, I use vibe coding to describe an exploratory workflow in which generated behavior is accepted mainly because it appears to work, while systematic verification and implementation understanding remain limited. I use AI-assisted engineering for a workflow that makes requirements, tradeoffs, verification and operational ownership explicit. These are working definitions for the argument. They are not a standardized classification of developers.

Actual practice occupies a spectrum. Chou and colleagues' FSE 2026 study examined twenty public videos, including seven coding sessions containing 254 prompts. They observed different levels of delegation, inspection and adaptation, with requirements sometimes emerging during the interaction. The study provides a qualitative account of practice; its selected public videos cannot establish how often a workflow succeeds in commercial development. Chou et al., 2026

Figure 1. Two possible feedback structures. An appearance-driven prototype can support exploration. A product engineering process also connects implementation to the problem, evidence and behavioral verification. These are conceptual workflows, not measured performance comparisons.

Sarkar and Drosos similarly found expertise being applied to context selection, evaluation, diagnosis and intervention in five selected videos totaling more than eight hours. All the practitioners they studied had programming expertise. The research therefore illuminates how expertise can change form, without establishing what novices can or cannot accomplish. Sarkar and Drosos, 2025

I would evaluate two dimensions separately: how much work is delegated, and how well the resulting system is understood and verified. High delegation can coexist with strong engineering. A person can also type every line of a poorly specified, inadequately tested product.

Even the criterion of personally reading every generated line needs care. In May 2026, Willison described increasingly delegating implementation without that level of inspection, while continuing to care about production quality and accountability. His account is an experienced practitioner's reflection, not an experiment, but it makes an important distinction visible: assurance can operate at the level of interfaces, properties and independently checked behavior. It still has to be earned. Willison, 2026

A screenshot-driven prototype belongs comfortably inside an engineering process when its purpose is clear. It might test navigation or help someone explain a workflow. The mistake would be treating success on that visual task as evidence about isolation, billing, recovery or customer demand. Each of those questions needs its own evidence.

2 Start with the bill and the job the customer needs done

My starting frustration is specific. I can make several individually reasonable infrastructure choices and end up paying meaningful monthly charges for an application whose demand remains unproven. Development and production environments can make that accumulation more visible.

The arithmetic needs to be honest. Supabase lists a Pro organization at $25 per month and includes $10 in monthly compute credits. A Micro project's compute is approximately $10 for a full month. Under those assumptions, one continuously running Micro project is about $25 in total, while two are about $35. Two projects can represent development and production; they do not necessarily mean two separate customer-facing applications. Compute is metered, so the exact monthly amount depends on runtime. Supabase pricing; Supabase compute billing

Vercel's Pro plan lists a $20 monthly starting price, including one deploying seat and a $20 usage credit. Resend lists a $20 monthly Pro tier for transactional email, while also offering a free tier with a monthly allowance and a daily limit. Those are different products with different entitlements. Their plan fees should not be described as pure waste simply because an application is young. Vercel Pro documentation; Resend pricing

For a deliberately selected paid stack, the following comparison makes the concern concrete. These are illustrative monthly US-dollar amounts for one deploying developer, with the database projects in one Pro organization, before taxes, additional seats, add-ons and usage beyond included allowances. Live pricing was checked on 24 September 2026.

The last row matters. Paid email is optional when the free tier satisfies the workload. Other free tiers, local development environments or alternative providers may also fit. “A professional application costs at least $75 before its first user” would be an unsupported conclusion. The useful claim is that a particular set of managed paid plans can create a $65–75 monthly commitment, and some builders would prefer a different tradeoff.

That observation becomes a product hypothesis: a segment of developers may value a lower initial commitment and one coherent way to deploy an application, attach a database and send email. The hypothesis contains several separable needs. Some customers may care primarily about cost. Others may care about fewer accounts, simpler setup, predictable metering or a workflow their coding agent can complete.

I would investigate which of those needs is actually painful. A developer who already has spare capacity in an existing organization may face a different incremental cost from a new customer. A company with strict recovery requirements may willingly pay more for an established service. Someone testing an idea may be well served by existing free plans. These differences define the possible customer segment.

The first research question is therefore about the job and its constraints: what must a builder do to move from a repository to a working application, what does that process cost at low usage, and which parts do they want to avoid managing?

3 Use research to challenge the product hypothesis

An agent can produce a long market report quickly. I would still want to know which statements came from pricing pages, which came from actual users and which are interpretations. A generated report becomes useful when its claims can change a decision.

For this hosting problem, I would collect evidence in three practical forms. Official documentation establishes the capabilities and limits of candidate services. A small, reproducible cost model exposes the assumptions behind the price comparison. Conversations and observed onboarding attempts reveal whether the proposed simplification matters to the intended users.

The third form cannot be replaced by asking an LLM to imagine a customer interview. A model can suggest questions, summarize authorized notes or identify inconsistencies. It cannot turn invented responses into evidence of demand. Until real observations exist, willingness to pay and preference for a unified workflow remain hypotheses.

There is empirical support for treating entrepreneurial decisions this way. Camuffo and colleagues' 2024 replication combined four randomized trials involving 759 firms. Training in a scientific approach affected idea termination and produced a more focused pattern of strategic changes. The intervention concerned entrepreneurship rather than AI coding, so it does not validate a particular software workflow. It does support the value of explicitly testing assumptions and being willing to stop pursuing an idea. Camuffo et al., 2024

In my case, useful disconfirming evidence could be quite ordinary. Target users may prefer the existing free tiers. The supposed setup problem may disappear after the first project. A consolidated offering may cost more once support and upstream usage are included. A provider may lack the provisioning interface that the proposed user experience requires.

I would ask the research agent to look for these outcomes deliberately. A recommendation should say what evidence supports it, what assumptions it depends on and what would cause me to reject it. I would also distinguish an unavailable capability from one that exists but has not yet been tested in the intended environment.

The architecture research needs the same discipline. Suppose an agent recommends a cheap virtual machine because it can run an application, PostgreSQL and an email-related service at a low advertised price. That recommendation may be sensible for someone willing to administer a server. It conflicts with my stated requirement to avoid that operational responsibility.

I would make the constraint explicit: the initial product should use serverless application execution and managed data services. By serverless execution, I mean a provider-managed runtime in which I do not provision or administer the underlying application servers. Avoiding an operator-managed fleet of virtual machines is part of the choice, as is avoiding that burden for customers. I still own the application configuration, integrations and operational response. This does not imply that every dependency scales to zero or has no idle cost. Those are separate properties to verify.

Research has done its job when I can explain why one option fits the problem and another does not. A list of fashionable technologies, however well formatted, leaves that work unfinished.

4 Write the second prompt around the problem

The first prompt asks the model to infer a product from familiar names and images. A stronger starting prompt exposes the reasoning that the implementation will have to satisfy.

Prompt A — The clone request

Build me a Vercel, Supabase and Resend clone.
Use these four screenshots as references.
Make the dashboard look polished and build it now.

Prompt B — The engineering request

I want to investigate a managed hosting product for small applications.
The problem is the fixed commitment and setup work created by combining
separate application hosting, PostgreSQL and transactional email services
before demand is known.

Research existing alternatives using primary sources. Verify prices,
included allowances, development versus production costs, provisioning
APIs and operational limits. Identify cases where the existing products
already solve the problem. Separate facts from hypotheses.

My constraints are serverless application execution, managed PostgreSQL,
transactional email and an agent-usable interface. Customers should not
need to administer virtual machines, and the initial platform should not
require me to operate a VM fleet. Do not assume every dependency scales
to zero. Begin with individual users owning deployments.

First propose at most twenty concise requirements and differentiators.
Then create plan.md with the architecture, verified API contracts,
alternatives, costs, ownership model, failure handling, meaningful tests,
implementation slices and unresolved decisions.

Challenge unnecessary organization layers, duplicate systems and features
that do not serve the first use case. Explain technical choices so I can
review them. Begin implementation after the plan has been reviewed.
For each slice, define observable acceptance criteria and a bounded
build-test-review loop. Use test-driven development in small increments:
start with the first happy path, then add consequential failure cases
before release. Report evidence for completion.

The second prompt takes longer to read because it communicates decisions the first prompt omits. Its value will still depend on the quality of the research, review and implementation that follow. A long prompt can contain poor assumptions. A short prompt can be sufficient when a repository already contains excellent context.

I would also separate the initial feature brief from the technical plan. A compact brief establishes the intended product before an agent starts expanding every possibility into a subsystem. For this case, an illustrative brief could contain seventeen points:

• Serve individual developers running small applications whose demand is uncertain.
• Reduce the initial financial commitment where a metered alternative is viable.
• Make costs and the effects of starting or stopping resources understandable.
• Deploy a deliberately limited set of application types.
• Run application code on a serverless runtime without operating a VM fleet.
• Provide managed PostgreSQL for application data.
• Provide transactional email through a supported delivery service.
• Keep virtual-machine administration outside the customer's workflow.
• Make deployment and inspection available through an API and a CLI.
• Expose selected operations to agents through MCP where useful.
• Provide separate development and production targets when requested.
• Make the data-isolation choice and its cost explicit.
• Start with individual ownership of deployments and resources.
• Show the actual state of provisioning, including incomplete operations.
• Support understandable recovery from failed deployment attempts.
• Preserve a defined route for data export and resource removal.
• Defer team hierarchies and broader platform features until a user need justifies them.

This is a proposed brief derived from the problem, not a claim that each item describes a verified feature of the current product. The list is short enough to review as a whole. Every addition has to explain which user need it serves and what new work it creates.

5 What OhMyHost makes concrete

OhMyHost gives this argument a real product reference. Its public description presents an agent-oriented hosting workflow that combines application hosting, databases, transactional email and metered credits, with CLI and MCP access. It also includes a portal. The meaningful design direction is that programmatic operation is central to the product; describing it as having no interface of any kind would overstate the case. OhMyHost product description

A consolidated experience can create value through the relationship between existing capabilities. The customer wants the application to run, the database connection to be available, the email workflow to function and the account of usage to be understandable. Reproducing the internals of every upstream service is a different undertaking, with different costs and responsibilities.

I would assess that product direction through the quality of the integrated workflow. Can the intended user complete it? Can the agent discover what operation comes next? Does the response reveal an actionable state? Can a failed step be diagnosed without guessing? Does the price remain attractive for the actual workload?

Public product claims alone do not establish those outcomes. OhMyHost's pricing documentation also makes qualifications visible: independently isolated development and production databases are separately metered, custom-domain connection is distinct from domain registration, and the page checked for this essay stated that production purchases were not yet enabled. Those details limit what can be claimed about current availability and comparison with established paid plans. OhMyHost pricing

The product is therefore an example through which to explain engineering decisions, rather than an experiment proving the superiority of one development method. The architecture examples below are proposed designs. They do not purport to reconstruct a private implementation or a development history that has not been documented here.

6 Make plan.md a record of decisions

Once the brief is coherent, I would develop plan.md. Its purpose is to connect the user problem to decisions that implementation can follow and verification can challenge.

The plan may become technically detailed. It should explain the important components, the resources they own, the interfaces between them, the failure conditions and the evidence needed for release. I need to understand that material before allowing an agent to turn assumptions into a large codebase.

Reading the plan is active work. If the agent proposes an organization table, I ask which initial user journey requires it. If it proposes a queue, I ask which operation needs durable asynchronous execution. If it proposes two accounts, I ask whether they represent separate identities, supplier requirements or accidental duplication. If it proposes a custom abstraction, I ask which concrete variation that abstraction must support now.

I can ask the agent to explain unfamiliar concepts. I can ask it to walk through a deployment, identify the authoritative owner of each record or show what happens after a timeout. Understanding becomes visible when I can predict the consequences of a design choice and recognize a contradiction in the explanation.

A plan I would accept contains the following kinds of material, at a level appropriate to the first implementation slice.

The plan should identify uncertain statements instead of smoothing them into confident prose. “The provider supports an API” is weaker than identifying the operation, version, authentication method, permissions, asynchronous behavior and limits. A documentation link supports investigation. A successful probe shows that the specific operation worked under the conditions tested.

I would keep the primary plan navigable. Detailed API notes, data schemas and test specifications can live in linked files. The top-level document should expose the decisions and their dependencies. Thousands of lines may sometimes be justified, but line count is a poor measure of understanding. Repetition increases reading cost without resolving uncertainty.

Design-science research provides a useful intellectual foundation here. Hevner and colleagues connect the construction of an artifact to a relevant problem and to evaluation of its usefulness. Their framework is methodological, rather than a controlled test of Markdown planning. The applicable principle is that building and evaluation inform each other as the problem and solution become better understood. Hevner et al., 2004

That principle also prevents the plan from becoming a frozen specification. An integration test may disprove an assumption. A user trial may expose a missing requirement. The appropriate response is a reviewed revision with a reason, followed by corresponding changes to implementation and acceptance criteria.

Figure 2. A proposed chain from user need to constraints, alternatives and verification. The simple ownership model is an initial design choice for the example. Deferring an organization hierarchy does not remove the need to enforce ownership.

7 Remove complexity with a reason

Generative systems can elaborate a plausible architecture faster than a person can assess it. That creates a practical risk: breadth can look like completeness. Organizations, workspaces, teams, projects, environments, subscriptions and policies all sound reasonable in isolation. Together they can create a product model far beyond the first use case.

For an initial hosting service aimed at individuals, a user owning a deployment may be sufficient. Related resources can belong to that deployment or to the user under explicitly defined rules. This arrangement still needs stable identifiers and authorization checks. It does not automatically need invitations, team membership, inherited roles or organization-level billing.

The distinction is between a required invariant and a speculative feature. A user must be prevented from reading another user's resources from the beginning. Team administration can wait if there is no team workflow. A required supplier account may be necessary for provisioning. Asking the end user to manage redundant accounts may conflict with the product's purpose.

I would record the condition that would justify revisiting each deferred decision. Shared ownership by several people could trigger the need for a team model. A measured backlog or a dependency's asynchronous API could justify a durable job mechanism. A demonstrated availability requirement could justify additional redundancy.

This is more precise than a blanket instruction to avoid complexity. A persistent operation record can look like extra work until the first timeout leaves the system uncertain whether it created a billable resource. That record directly addresses a failure inherent in the proposed workflow. A generalized organization-permission language may have no comparable initial justification.

The same reasoning applies to future flexibility. I would preserve resource identifiers and clear interfaces so the design can evolve. I would avoid implementing every possible future consumer of those interfaces. The plan should make the expected cost of deferral visible, including migrations that may be needed if the customer model changes.

Research can also overrun its purpose. Once a consequential decision has enough evidence for a bounded experiment, I would build that experiment. Additional reports should answer a remaining question. Otherwise the process can accumulate documents while postponing contact with the system it is supposed to explain.

8 Follow one deployment through the architecture

To review the plan, I would walk through one ordinary request: an authenticated user asks the service to deploy an application with a database. The request is simple from the user's perspective. It may involve several operations with different durations and failure modes.

First, the service determines who is asking and whether that identity owns the target resource. It validates the requested configuration and records the operation. It then invokes the required managed services and observes their results. Finally, it reports a state that reflects what actually happened.

This walkthrough raises questions a dashboard screenshot cannot answer. Does a successful API response mean that work was accepted, or that the application is already healthy? Where is the authoritative deployment state? If the database was created but the application build failed, what remains billable? Can the user inspect and remove those resources? Which information is safe to return to a coding agent?

Now introduce a realistic failure. The service requests a database. The provider creates it, but the response is lost. The client sees a timeout and retries. If the implementation treats each retry as a new intention, one attempted deployment can create two resources.

I would give the operation a stable identity and define its retry behavior. The service would inspect its record and, where supported, reconcile that record with the provider's state. It would reuse the provider's idempotency mechanism when one exists. If the outcome remains unknown, the system should expose that uncertainty and recover deliberately, rather than immediately creating another resource.

This does not require a claim of universal exactly-once execution. It requires an explicit answer to duplication and partial completion. The same principle applies to billing: a repeated callback or client request should not be mistaken for a second independent chargeable event. The implementation mechanism depends on the provider contracts and the product's accounting model.

The plan should also describe what “ready” means. For the first slice, it could mean that the selected application artifact is deployed, a health check succeeds and the configured database connection works. If email is in scope, acceptance can include a test message to a controlled recipient through a verified sender. A build completing successfully is evidence about the build. It does not establish the whole user journey.

Figure 3. Proposed deployment semantics. A stable operation identity allows inspection and reconciliation after a lost response. The diagram illustrates a requirement to prevent unintended duplication; it does not assert that all providers offer identical guarantees.

This example also makes headless product design concrete. An agent needs operation names, input schemas, understandable errors, resource identifiers and status information. It benefits when the API, CLI and MCP interface expose consistent meanings. A decorative label such as “agent-ready” cannot substitute for those contracts.

MCP standardizes the exposure and invocation of tools, including input schemas and optional structured output schemas. Its specification distinguishes protocol errors from tool-execution errors. These facilities help an agent interact with a system, while the application still has to implement its own business rules and resource permissions. MCP tools specification, 2025-11-25

A minimal ownership model is sufficient only if it is enforced. The implementation must derive identity from authenticated context and check resource access on the server. A client-supplied user identifier cannot establish permission by itself. Sensitive provider credentials should remain in the service boundary that needs them. These are requirements of this proposed product, even if its first customer is a single developer.

The resulting architecture may be compact. It can still contain deliberate answers about ownership, state, secrets and recovery. That is the technical depth I want from the plan: enough precision to expose the mistakes that would matter in ordinary use.

9 Use TDD to shorten feedback cycles

I would ask the implementation agent to use test-driven development in bounded increments. Begin with one behavior, express its expected result, implement the smallest coherent change and run the test. Then inspect the result and improve the design where the evidence calls for it.

For the first deployment slice, the initial happy path can be modest. An authorized user submits a supported application, the service creates an operation, the application becomes available and the reported state agrees with the observed result. That test gives the agent a concrete target and gives me a way to distinguish actual progress from additional scaffolding.

The happy path is a useful starting sequence, not a sufficient release standard. Before the product handles other people's data or billable resources, the relevant ownership and duplication cases belong in scope. Other boundary cases can be added as the supported behavior expands. Prioritization should follow consequences and likely failure modes.

The empirical case for TDD is more nuanced than a universal instruction to write tests first. Fucci and colleagues analyzed eighty-two observations from thirty-nine professional developers. Their selected statistical models associated stronger outcomes with shorter, steadier development cycles, without identifying test-first ordering as a contributing factor. This was observational process analysis, not proof that sequence never matters. Fucci et al., 2017

Tosun and colleagues' experiment with twenty-four professionals found no statistically significant quality difference between TDD and incremental test-last development. Productivity varied with task characteristics. These studies predate current coding agents, but they support a practical emphasis on useful feedback and manageable increments, without promising that one ritual always improves speed. Tosun et al., 2017

I would therefore define tests by the mistakes they can detect. A unit test can establish a narrow rule such as rejecting an invalid transition. A contract test can show whether a provider response is handled as expected. An integration test can check that real components cooperate in a controlled environment. An end-to-end test can establish that the intended user journey completes.

Mocks have a legitimate role in fast feedback, but a mocked provider cannot establish that a real credential has permission or that the provider behaves as imagined. The plan should state which uncertainties require a live integration probe and which can be covered locally. A small number of well-chosen probes can answer more than a large suite that simply repeats the implementation's assumptions.

EvalPlus supplies a concrete example of why test quality matters. For the same GPT-4-generated solutions to short programming problems, stronger tests changed greedy pass-at-one performance from 88.4% to 76.2%. The programs had not deteriorated; the evaluation had become better at exposing failures. The paper also addresses ambiguous input contracts and reducing test suites while retaining effectiveness. Its lesson is discriminating evaluation, rather than test-count maximization. Liu et al., 2023

Twenty thousand generated tests would not impress me without an account of their purpose. I would ask which requirements they cover, whether their expected results are independently justified and whether failures are easy to diagnose. Tests that mirror internal code structure can make refactoring expensive while missing the behavior the customer depends on.

A reasonable stopping condition for a slice is that its agreed behavior has been demonstrated, its consequential failure cases have been addressed and no concrete unresolved defect prevents the next step. Test expansion should respond to a remaining question or a required release condition.

10 Engineer the loop around observations

After reviewing the plan, I would execute it through a repeated cycle: select a bounded task, provide the relevant context, make the change, observe the result and decide what follows. Each repetition should produce evidence or a clearer diagnosis.

This resembles the reason-and-act structure studied in ReAct, which interleaves model reasoning with actions and observations from an environment. Its original evaluations concern question answering and interactive tasks, rather than complete software delivery. The relevant architectural idea is that new observations can revise the next action. Yao et al., 2023

For a coding agent, the observation might be a compiler error, a failing acceptance test, an API response or the behavior of a deployed development artifact. The next step should follow from that observation. If the agent repeatedly emits new code without resolving what failed, the loop is consuming resources without demonstrating learning.

SWE-agent provides more direct evidence about the importance of the agent's surrounding tools. With the same GPT-4 Turbo model, its tailored interface resolved 18% of SWE-bench Lite issues, compared with 11% for a shell-only agent given a demonstration. The interface incorporated file navigation, editing and execution feedback. These are historical 2024 results on repository issue repair, not a ceiling on current agents, but they show that workflow design affects performance. Yang et al., 2024

I would make the development loop's state explicit. It needs the current task, the version of the plan it follows, the relevant repository revision, the latest observations, unresolved questions and the condition for completion. The model's narrative of its progress is one input. The repository and observed test results are stronger evidence of what exists.

Useful feedback should also identify the expected behavior. “It failed” is less informative than “the repeated request created a second database even though it used the same operation identifier.” The second statement narrows the defect and supplies a criterion for a repair.

Olausson and colleagues studied this distinction in code self-repair. In a small diagnostic experiment on forty failing programs, human feedback increased the fraction of passing GPT-4 repair candidates from 33.3% to 52.6%, relative to model-generated feedback. That result concerns selected programs and does not include the full cost of human diagnosis. It nevertheless shows why repeating a request is not equivalent to supplying useful corrective information. Olausson et al., 2024

I would put a budget on repetition. After a defined number of unsuccessful attempts, or when the same failure returns without new evidence, the agent should report the diagnosis and propose a different experiment or a revision to the plan. The appropriate budget depends on task size and cost; there is no scientifically established universal number of retries.

This bounded loop supports autonomy. An agent can carry out substantial work without constant intervention when the task, evidence and stopping conditions are clear. The person responsible can concentrate on decisions that change scope, architecture, cost or consequences.

11 Give context and agent graphs a concrete purpose

The plan becomes useful to an agent only when the relevant parts are available at the right time. Sending every research note into every model call can hide the current decision among irrelevant material. Sending only the task title can omit the constraint that makes the task meaningful.

For the deployment slice, I would supply the current objective, the accepted ownership model, the relevant service contracts, the operation-state rules and the acceptance criteria. I would also include the latest verified state of the code and any failure that prompted the task. Background research can remain available by reference.

This is the connection to context engineering. Durable files preserve decisions and observations across sessions. The working context selects what the next decision requires. When a constraint changes, the plan and the dependent tasks need to change together. Otherwise an agent can implement an obsolete decision correctly.

Anthropic's account of long-running agent harnesses describes a similar need for incremental work, explicit feature state, progress records and verification across context windows. Its examples come from internal web-application development experiments rather than a controlled industry trial. They supply practical implementation ideas, without establishing that one harness or a large multi-agent system is universally best. Young, 2025

A graph can make dependencies and return paths explicit. One node may investigate a provider capability. Another may implement a contract. A verification node may produce test evidence. A review decision may integrate the change, return it for correction or revise the plan.

The nodes describe responsibilities. They need not each be a separate agent. One agent can perform several roles sequentially, while independent investigations can run in parallel when their outputs do not conflict. I would add agents when they resolve a specific bottleneck or provide useful independent scrutiny.

Parallelism needs boundaries. Two agents can independently inspect separate provider APIs. Two agents editing the same schema without coordination can invalidate each other's assumptions. A reviewer also needs the current implementation and acceptance criteria; a summary from the implementer is not enough to establish that the work is correct.

The graph should have a defined path for disagreement. If a test exposes a missing requirement, the answer may be to revise the requirement through review. If the implementation violates an accepted requirement, the answer is usually to repair the implementation. Quietly weakening a test so that the run becomes green erases the evidence that prompted the decision.

Figure 4. Proposed development graph with durable task state and explicit feedback. Nodes denote responsibilities and may be performed by one agent or several. Integration depends on observed results; repeated failure returns to diagnosis or planning.

I would retain a small record at each completed transition: what changed, which decision it implements, what was checked and what remains uncertain. That record makes the next session easier to start and gives another person a way to assess the result without reconstructing an entire conversation.

The purpose of loop and graph engineering is to control work and preserve useful feedback. A complex diagram without meaningful state transitions is another form of overengineering.

12 Measure the outcome the study actually measured

The research on AI-assisted development does not support one universal productivity number. Studies investigate different people, tools, tasks and outcomes. That variation matters to this essay because a hosting product includes work that a code-completion or issue-repair benchmark does not measure.

Cui and colleagues conducted three company experiments involving 4,867 developers. Their preferred pooled estimate associates induced adoption of GitHub Copilot with 26.08% more completed tasks or pull requests per developer-week. The estimate uses an instrumental-variable approach and has a standard error of 10.3 percentage points. It concerns output under existing organizational practices, rather than a 26% reduction in the time needed to build a successful business. Cui et al., 2026

METR's 2025 randomized study reached a different result in a narrower setting. Sixteen experienced open-source maintainers completed 246 tasks in familiar repositories. Access to early-2025 AI tools increased completion time by 19%, even though participants believed the tools made them faster. The result demonstrates the importance of measuring the complete activity, including inspection and correction, in the setting where the tool will be used. Becker et al., 2025

That finding should not be presented as a timeless verdict. METR's February 2026 update reported newer estimates leaning toward speedup, while explaining serious recruitment, task-selection and measurement problems. The researchers treated those data as weak evidence about the magnitude of newer gains. The later update changes how cautiously the earlier result should be generalized. METR, 2026

Maintainability also deserves direct evidence. Borg and colleagues' two-phase study found no statistically significant overall downstream maintainability penalty for the AI-assisted solutions in its bounded Java application setting. Absence of a significant difference does not establish equivalence or long-term production reliability. It does challenge a blanket assumption that AI authorship necessarily makes software harder to maintain. Borg et al., 2026

I would carry that balance into evaluating the hosting project. Useful measures include elapsed time to a verified deployment, the developer's active work, failed attempts, cost of agent execution, defects discovered after handoff and the effort required for a later change. Product usefulness requires additional evidence about whether the intended user wants the result.

These measures can move in different directions. An agent may reduce active typing while increasing elapsed waiting time. It may produce an acceptable first version quickly while leaving expensive follow-up work. It may also improve both speed and quality in a well-structured task. The evaluation should make those possibilities distinguishable.

Counting generated lines, screenshots or tests would miss the question. I care about whether the system fulfills its requirements at an acceptable total cost and can be changed when the requirements evolve.

13 Test the method rather than declaring it proven

The workflow I propose is informed by research, but the complete method has not been experimentally compared with the clone prompt in this essay. No cited study establishes that a particular feature count, one plan.md file or a specific agent graph is optimal.

A useful evaluation could compare two development procedures on the same hosting task. One begins with the sparse clone request and screenshot references. The other begins with the problem statement, constraints and reviewed plan. The comparison would need to include the time and cost spent producing the research and plan. Otherwise the planned approach would receive uncounted preparation.

Both procedures should use the same allowed model versions, tools, dependency environment and overall resource limits. Repeated runs are necessary because one successful generation can be luck. Human expertise and intervention time should be recorded. Evaluation should use criteria established before the implementations are inspected.

The evaluators could assess a shared set of requirements: completing the intended deployment, enforcing ownership, reporting partial failure, preventing unintended duplication, preserving data through the supported update path and making usage understandable. A later modification task would test whether another developer can extend the result. Acceptance tests should be held apart from the generated implementation where practical, so the comparison does not reward a system for defining its own success.

That design would estimate the effect of two complete workflows, including their different information content. It would not isolate the value of a Markdown file. To investigate mechanisms, further comparisons could provide the same verified requirements with or without an explicit plan, or compare bounded execution with an unrestricted loop under a matched budget.

The product hypothesis requires its own evaluation. Intended users could attempt deployment with the consolidated workflow and with the relevant existing alternative. Their task, application and familiarity would need to be comparable. Observing setup time, intervention, confusion and recurring cost would be more informative than asking whether the new dashboard looks appealing.

A successful technical experiment and a useful customer product are related achievements. Neither automatically supplies evidence for the other. Keeping those evaluations distinct allows the team to learn whether it should improve the implementation, change the product hypothesis or stop pursuing the current direction.

14 Finish the product work beyond code generation

For a hosting service, release changes the nature of the work. The product now has to report failures, handle provider changes, account for usage and give users a way to recover or leave. These responsibilities remain even when managed services perform much of the underlying operation.

I would make the first release conditions explicit in the plan. They should fit the promised scope. A service that stores customer data needs a defined recovery approach and a tested export path. If automatic backups are not included, the product should say so. If a balance limit can interrupt service, the user should understand when and how that happens.

The economics also need to include more than upstream compute. A consolidated product may incur costs for support, resource coordination, failed operations, retained storage and its own development. A lower starting bill for one workload does not establish lower total cost for every workload. I would publish the relevant usage assumptions and compare like-for-like capabilities.

Security practice belongs throughout this process. NIST's Secure Software Development Framework organizes practices for preparing development, protecting artifacts, producing secure software and responding to vulnerabilities. It is a framework, not empirical proof that a checklist prevents every defect. For this example, its practical relevance is to connect requirements and review to the continuing response to problems found after release. Souppaya et al., 2022

Human confidence is not a substitute for that evidence. Perry and colleagues' CCS 2023 study found that participants using a historical AI assistant could produce insecure solutions while expressing greater confidence in their security. The small study used security-focused tasks and an older model, so it cannot establish the security of current generated code. It does provide a reason to judge concrete properties rather than reassuring explanations. Perry et al., 2023

This continuing responsibility also makes technical understanding useful after the initial plan. When a provider changes an API or a customer reports unexpected behavior, someone needs to determine which assumption no longer holds. A documented decision, a reproducible test and an observable operation history make that investigation possible.

15 What I expect from engineering with AI

I want to use AI for substantial parts of the work: researching alternatives, explaining unfamiliar systems, drafting architecture, implementing changes, generating useful tests and investigating failures. The ability to delegate those activities increases the amount I can attempt.

My responsibility is to make the objective coherent and to assess the evidence on consequential decisions. That includes rejecting a cheap server when serverless execution is a requirement, removing a team hierarchy that has no initial user, questioning a cost comparison that ignores free tiers and refusing to treat a successful build as proof of a working deployment.

The two prompts at the start of this essay express different starting conditions. The clone request can produce a useful prototype. The engineering request initiates an investigation whose output includes both software and reasons to trust its behavior. A project can move from the first mode into the second as its purpose and consequences become clearer.

For the hosting case, I would begin with the recurring cost and setup problem. I would test whether that problem matters to a specific group of developers. I would reduce the proposed product to a reviewable brief, work through the architecture in plan.md, and implement one complete path before expanding. I would use bounded loops and relevant context so agents can make progress that survives the next session.

The finished work should let another person answer practical questions. What need does this serve? Why were these dependencies selected? Who owns a resource? What does success mean? What happens after a failed request? Which claims have been checked, and which remain assumptions?

When those answers are available and supported by the system's behavior, I have something I can operate, improve and explain. That is what I am trying to build with AI.

Appendix A A compact plan structure

The following outline is an illustrative working artifact for the case. It is deliberately smaller than a complete technical specification. Each section can link to detailed evidence when the first implementation requires it.

# plan.md

## Problem and intended user
Individual developers need a managed path from repository to a running
application with PostgreSQL and transactional email. Validate whether
lower initial commitment and simpler operation matter to this segment.

## Constraints and first scope
Serverless application execution and managed data services; no VM fleet
for me to operate and no customer-managed virtual machine.
Begin with one supported application path and individual ownership.
Treat scale-to-zero, retained-resource cost and data isolation separately.

## Evidence and decisions
Link dated pricing and official API documents.
Record alternatives, assumptions, integration probes and rejected options.
Identify the evidence that would reverse a decision.

## Ownership and state
Define User, Deployment, Resource and Operation identifiers.
Define server-enforced ownership and authenticated service boundaries.
Specify accepted, provisioning, ready and failed states.
Specify unknown outcomes, retry behavior and reconciliation.

## First implementation slice
An authorized user deploys one supported application with a database.
Verify the deployed artifact and connection in a controlled environment.
Record actual state and expose useful inspection commands.

## Verification
Write one behavioral acceptance test before implementing the slice.
Add ownership, duplicate-request and partial-failure checks where relevant.
Distinguish mocked checks from real integration evidence.
Record commands, observations and remaining uncertainty.

## Execution loop
Select task -> load relevant context -> implement -> verify -> review.
Integrate when acceptance criteria pass and identified defects are resolved.
After repeated failure without new evidence, stop and revise the diagnosis.
Persist task state and the plan revision at each handoff.

## Deferred work and triggers
Add team membership only when shared ownership is a supported requirement.
Add other infrastructure only when a demonstrated need justifies it.
Keep release conditions, resource cost and recovery obligations explicit.

Appendix B What useful acceptance evidence looks like

These examples describe a proposed initial test scope. The production scope should follow the actual supported operations and their consequences.

Research scope and references

This essay is a critical synthesis of primary research, official documentation and identified practitioner accounts. It includes a proposed development method and a concrete product illustration. It reports no new experiment and is not a systematic review or meta-analysis. Research publications cited here were available before the essay's 20 September 2026 issue date; live product and pricing pages were verified on 24 September 2026. The OhMyHost descriptions are public-source observations, while the example prompts, brief, architecture and evaluation design are the author's proposed method.

References

1. Huyen, C. (2025). AI Engineering: Building Applications with Foundation Models. O'Reilly. Author's book description.
2. Karpathy, A. (2025, February). Original account introducing “vibe coding.” Original post. Practitioner account; text also reproduced in Willison's March essay.
3. Willison, S. (2025, March 19). Not all AI-assisted programming is vibe coding (but vibe coding rocks). Practitioner essay.
4. Chou, Y.-H., Jiang, B., Chen, Y. W., Weng, M., Jackson, V., Zimmermann, T., & Jones, J. A. (2026). Building Software by Rolling the Dice: A Qualitative Study of Vibe Coding. Proceedings of the ACM on Software Engineering, 3, FSE, Article FSE077. Paper.
5. Sarkar, A., & Drosos, I. (2025). Vibe coding: programming through conversation with artificial intelligence. Psychology of Programming Interest Group Conference. Author manuscript.
6. Willison, S. (2026, May 6). Vibe coding and agentic engineering are getting closer than I'd like. Practitioner reflection.
7. Supabase. Pricing. Official plan information. Live page checked 24 September 2026.
8. Supabase. Compute usage and billing. Official documentation. Live page checked 24 September 2026.
9. Vercel. Pro plan. Official documentation. Live page checked 24 September 2026.
10. Resend. Pricing. Official plan information. Live page checked 24 September 2026.
11. Camuffo, A., Gambardella, A., Messinese, D., Novelli, E., Paolucci, E., & Spina, C. (2024). A scientific approach to entrepreneurial decision-making: Large-scale replication and extension. Strategic Management Journal, 45(6), 1209–1237. Paper.
12. OhMyHost. About. Public product description, linked in Section 5. Live page checked 24 September 2026.
13. OhMyHost. Pricing. Public product documentation, linked in Section 5. Live page checked 24 September 2026.
14. Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design Science in Information Systems Research. MIS Quarterly, 28(1), 75–105. Paper.
15. Model Context Protocol. (2025). Tools. Specification revision 2025-11-25. Official specification.
16. Fucci, D., Erdogmus, H., Turhan, B., Oivo, M., & Juristo, N. (2017). A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last? IEEE Transactions on Software Engineering, 43(7), 597–614. Accepted manuscript.
17. Tosun, A., et al. (2017). An industry experiment on the effects of test-driven development on external quality and productivity. Empirical Software Engineering, 22, 2763–2805. Paper.
18. Liu, J., Xia, C. S., Wang, Y., & Zhang, L. (2023). Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation. NeurIPS. Paper.
19. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. ICLR. Paper.
20. Yang, J., Jimenez, C. E., Wettig, A., Lieret, K., Yao, S., Narasimhan, K., & Press, O. (2024). SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. NeurIPS. Proceedings paper.
21. Olausson, T. X., Inala, J. P., Wang, C., Gao, J., & Solar-Lezama, A. (2024). Is Self-Repair a Silver Bullet for Code Generation? ICLR. Paper.
22. Young, J. (2025, November 26). Effective harnesses for long-running agents. Anthropic. Engineering account.
23. Cui, K. Z., Demirer, M., Jaffe, S., Musolff, L., Peng, S., & Salz, T. (2026). The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers. Management Science, published online 27 February. Paper.
24. Becker, J., Rush, N., Barnes, E., & Rein, D. (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. arXiv preprint 2507.09089. Paper.
25. Becker, J., Rush, N., Cunningham, T., Rein, D., & Mahamud, K. (2026, February 24). We are Changing our Developer Productivity Experiment Design. METR. Primary research update.
26. Borg, M., et al. (2026). Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability. Empirical Software Engineering, 31, Article 161. Paper.
27. Souppaya, M., Scarfone, K., & Dodson, D. (2022). Secure Software Development Framework Version 1.1: Recommendations for Mitigating the Risk of Software Vulnerabilities. NIST SP 800-218. Official publication.
28. Perry, N., Srivastava, M., Kumar, D., & Boneh, D. (2023). Do Users Write More Insecure Code with AI Assistants? ACM Conference on Computer and Communications Security. Paper.