# How I put AI into a delivery system Source: https://manaienko.com/how-i-work/ai-in-the-delivery-system/ Author: Artem Manaienko The useful unit is a workspace, not a prompt. Curated context, task-specific workflows, and a human gate everywhere a wrong change would actually cost something. ## My principles 08 operating rules 1. [01 The expensive part is reconstructing the system](#practice-ai-in-the-delivery-system-principle-1) 2. [02 The unit is a workspace](#practice-ai-in-the-delivery-system-principle-2) 3. [03 Every output meets a human gate](#practice-ai-in-the-delivery-system-principle-3) 4. [04 Encode the boundaries into the tooling](#practice-ai-in-the-delivery-system-principle-4) 5. [05 Adoption is earned per person](#practice-ai-in-the-delivery-system-principle-5) 6. [06 Evaluate before you pay](#practice-ai-in-the-delivery-system-principle-6) 7. [07 Price per token is not the cost](#practice-ai-in-the-delivery-system-principle-7) 8. [08 Route by task class, not by a house model](#practice-ai-in-the-delivery-system-principle-8) ### The expensive part is reconstructing the system Writing the code is rarely the expensive part. Before anyone can write anything, several people have to reconstruct what the system already does. That is where a model earns its cost. ### The unit is a workspace Repository context, a structured knowledge base of architecture and constraints, and task-specific workflows for each stage. A good prompt improves one interaction. A workspace improves the workflow. ### Every output meets a human gate Machine review is an extra gate, never a replacement for one. Normal pull-request review stayed mandatory on every production change, and that is what made the output usable rather than just fast. ### Encode the boundaries into the tooling Access and confidentiality rules belong inside the workspace, not in a policy document sitting next to it. Platform engineers had no access to protected health data, and the workspace rules said so. ### Adoption is earned per person Prove it on your own delivery first. Then two pilot engineers, then roughly ninety minutes of pairing each, then the team. Announcing a tool to a team that has already been burned by one does not work. ### Evaluate before you pay Run deterministic filters before any model call, sample on a bounded set, and put the real cost on screen next to the result. Then a prompt change is a measured decision instead of an impression that the output looked better. ### Price per token is not the cost A cheaper tier makes more calls and burns more tokens, so neither the price per token nor the token count decides anything on its own. What decides is price multiplied by the total work the model needs to finish the task, and that second factor only exists after the run. Which is why every attempt records its route, usage and attributable cost. ### Route by task class, not by a house model Planning and independent review stay on the stronger profile, because a wrong plan is expensive to catch late and the review is what gates acceptance. Bounded implementation goes to the cheaper tier, where deterministic gates catch what it gets wrong. An extra half hour of wall clock is nearly free on an unattended run and not free at all in a pairing loop, so who is waiting is part of the routing decision. ## What that looked like 04 cases ### An AI workspace for cross-platform component delivery Teladoc Health · mid-2025 onward **Situation:** Our design system had roughly 52 newer components and 50+ legacy components across the core product, several business-unit products, and around eight product teams. A component could arrive without useful technical documentation. Before anyone wrote code, several engineers compared the specification with the released Android, iOS, React, and Ruby implementations and agreed one contract. Discovery took two hours or more with several people. Integration then took two to four engineering days. The team had already had a poor experience with an AI coding assistant, so I also had to earn back trust. **What I did:** - I assembled one workspace with repository, GitHub, and Figma context, plus a linked Markdown knowledge base for architecture, code style, platform constraints, integration gates, and developer checks. - I split the delivery chain into bounded workflows. Specification analysis, shared contract, implementation, validation, and release each had their own job. - Before the discovery meeting, the model read the specification, inspected the released implementations, found contradictions, and proposed a cross-platform contract for the engineers to review. - I kept normal pull-request review on every production change. When the model failed, I changed the workflow or prompt and tried again on reviewed work. - I used it on my own delivery first, then with two pilot engineers. Paired onboarding took roughly 1.5 hours per person before I rolled it out to the team. **Outcome:** The shared discovery meeting fell from two-plus hours to no more than thirty minutes. A component could be analysed and agreed in around four hours. Roughly 12 engineers used the workspace daily and about 20 in a typical month. My estimate for the whole integration workflow was approximately 3x productivity and about 120 developer-hours saved per month. The 3x and 120-hour figures are my own confirmed estimates. They were not independently audited, and I did not preserve the calculation model. I would instrument this before rollout if I built it again. The delivery chain the workspace covered 1. 01 ### Specification analysis Read the product and design specification against released per-platform behaviour; surface gaps and contradictions before anyone meets. 2. 02 ### Shared contract Propose one cross-platform property and interface model, preserving the platform-specific constraints that cannot be standardised away. Human review 3. 03 ### Backend representation Express the contract in the Ruby API core, where the integration actually starts. 4. 04 ### Per-platform implementation Android, iOS, and React implemented separately against the same agreed contract, plus Storybook and demo-app integration. 5. 05 ### Validation and triage Compare delivered behaviour against the specification; route incoming defects to the owning layer with a probable fault area attached. Human review 6. 06 ### Release and PR governance Test whether the proposed release classification matches the change; enforce required PR content and commit scope. Mandatory PR review The workspace followed six bounded workflows, with human gates where a wrong change would have been expensive. ### The design-to-code experiment that did not work Teladoc Health · 2025 **Situation:** Screen generation looked like the next useful extension. I built a workflow that passed Figma context into Cursor, then tried Builder.io and Figma Make against our screens and component system. **What I did:** - I generated screens with each tool and compared the output with the components we actually shipped. - I checked how the generated code used runtime components and fitted into navigation, backend state, validation, and existing flows. - I separated the screen rendering that worked from the integration work that still had to be done by hand. **Outcome:** Parts of screen generation worked. The generated code carried styles and tokens, but did not reliably use the runtime components underneath. Then the integration work started: navigation, backend state, validation, and legacy flows. That was most of the work. I would put future effort there. Further work stopped after reorganisation and headcount reductions. I do not have enough evidence to say how much better tooling would have changed the result. ### Making a prompt change a measured decision Independent product · 2026-present **Situation:** In the AI product I am building now, a prompt change can move both output quality and the running cost of the whole information pipeline. Looking at one nicer output told me almost nothing. I needed the same input, the same sample, and the cost beside the result. **What I did:** - I built prompt evaluation into the product. A candidate prompt set and hard-filter configuration run against a bounded temporary sample from real sources. - I applied deterministic filtering before any paid model call, so obvious noise did not enter the evaluation. - I put token use and real model cost beside what the pipeline kept and dropped. - I kept evaluation runs isolated from product data and the saved configuration until an explicit save. **Outcome:** I can now compare a candidate prompt with the current behaviour on the same sample and see the cost before I save it. So far, this gives me a repeatable decision point. I do not have enough runs yet to claim better output or lower cost. ### Routing the workflow by measured cost, not by price per token Independent product · 2026 **Situation:** The implementation step of my delivery workflow is a coding agent working inside its own worktree, and I held two plausible positions about which model tier it should run on. The cheaper tier costs roughly a tenth per token, so use it wherever it can do the job. Or a weaker model wanders, burns the saving on extra calls, and ends up more expensive. Both are guesses about the same unknown, which is how much more work the weaker model needs to finish the same task. **What I did:** - I ran the same specification at the same reasoning effort on both tiers, changed nothing but the model, and kept the counters. - I compared tool calls, tokens, wall clock and estimated bill together, instead of arguing from price per token. - I routed by task class afterwards. Planning and independent review on the stronger profile, bounded implementation on the cheaper one. - I made every attempt record its route, usage and attributable cost, because the second factor in the cost only exists after the run. **Outcome:** The cheaper tier made 2.64x the tool calls, used 3.35x the tokens, and took about twice the wall-clock time. Its estimated token bill still came to roughly a third, because the price gap was wider than the efficiency gap. So the cheaper model did far more work and was still the cheaper choice for this class of task. One run, one task, one specification. Tool calls, tokens and minutes are measured; the bill is an estimate from list prices at the time, not an invoice. I did not count review rounds or rework, so the quality side is bounded by the gates both runs had to pass and not measured beyond that. I do not yet have a run that shows where the break-even sits. One implementation run, same spec and reasoning effort The stronger tier ### Fewer steps, higher price per token - 123 tool calls. - 13.97M tokens. - About 30 minutes wall clock. - Roughly ten times the price per token. The cheaper tier ### More steps, and still the cheaper bill - 325 tool calls, 2.64x as many. - 46.74M tokens, 3.35x as many. - About 62 minutes wall clock. - Estimated token bill about a third of the stronger tier. **~3x** cheaper token bill, estimated from list prices The cheaper model did far more work. The per-token gap absorbed all of it and most of a second run besides. What I would do differently Build the evaluation set first. Representative specifications, bugs, and review comments, scored for precision, false positives, and rework, versioned as product artifacts next to the prompts. The workspace worked. The evidence that it worked got reconstructed afterwards, which is a weaker position than it needed to be.