Skip to main content
Published in Engineering Insights

How to Run a Claude Code Pilot With One Team

How to Run a Claude Code Pilot With One Team
Table of contents
  1. What should a Claude Code pilot prove?
  2. How many engineers should be in a Claude Code pilot?
  3. Which team and which workflow should go first?
  4. What goes in the pilot charter?
  5. What has to be set up before week one?
    1. Provider and seats
    2. Settings and limits
    3. Usage visibility
    4. Data handling
    5. Project instructions
  6. What happens in each of the four weeks?
  7. Which numbers should decide the pilot?
  8. What did a four-week Claude Code cohort look like at GSC Enterprises?
  9. What should the CTO decide when the four weeks end?
  10. When does outside help make sense for a pilot?

Run a Claude Code pilot with one engineering team for four weeks, on one workflow the team already does in its own code. Before anyone signs in, write down the number that decides the pilot and the rule that turns it into a buying decision.

A group of early adopters drawn from several teams mixes codebases, review habits and release schedules in one set of numbers. One team that ships together gives you one baseline and one result. That result is something a CTO can take to the CFO and defend.

The Gnar Company is a Claude Select Services Partner. Its AI training for engineering teams puts the first step in one line: "Start with a four-week Immerse cohort on your own codebase".

This post is for the CTO who wants proof before buying Claude Code for every engineer. The plan below is general engineering advice.

What should a Claude Code pilot prove?#

A Claude Code pilot should prove whether one team's work in its own code gets measurably better. It answers whether to buy seats for teams like the pilot team. Where usage is billed per token to your organization, it also gives you your own spend figure.

Anthropic's Claude Code costs page says: "To estimate spend for your own team, start with a small pilot group and use the tracking tools below to establish a baseline before wider rollout". The same page gives Anthropic's averages across enterprise deployments. It puts the average at around $13 per developer per active day and $150 to $250 per developer per month, with 90% of users below $30 per active day.

How you pay for the pilot decides what the spend figure covers. Anthropic's costs page says: "On Teams and Enterprise plans, usage draws from each member's seat allowance. On the Console and on cloud providers, usage is billed per token to your organization". Pick the route you would use for the whole company, so the pilot's bill looks like the real one.

A one-team pilot does not prove how a company-wide rollout will go. Other teams have other code, other reviewers and other data rules. What it can show is whether this team, on this code, with this setup, gets a result worth paying for.

How many engineers should be in a Claude Code pilot?#

Put one whole team in the pilot, whatever its size, instead of a hand-picked group from several teams. Anthropic's rollout guide starts with a larger group.

Anthropic's guide, Scaling Agentic Coding Across Your Organization, says to "Begin with a pilot group" of 20 to 50 developers "who are already comfortable with AI-assisted tools". The guide notes that it "reflects current insights as of August 2025". Its closing chapter says: "Start small with a pilot group".

A mid-market company may not have 20 developers to put in a pilot. For a company like that, the pilot team is the engineering team.

There is a measurement reason to keep the pilot to one team as well. DORA's guide to software delivery performance metrics says the metrics "are best suited for measuring one application or service at a time". It also warns that the urge to "blend metrics across multiple teams" runs into differences in context, and that "doing so can be problematic". A pilot drawn from five teams starts with five baselines.

Take the whole team, including the engineer who doubts the tool. A rollout buys seats for everyone on the team.

Which team and which workflow should go first?#

Pick a team that passes four tests: it ships one codebase, its changes go through one review process, it has a recurring workflow Claude Code can work on, and its delivery history is already recorded. The workflow should be work the team does every week, chosen before the pilot starts.

  • One codebase. The team's own repository, with the stack it runs in production. A sample project tells you nothing about your code.
  • One review process. The same reviewers, the same rules and the same automated checks for every change, before and during the pilot.
  • A recurring workflow. Anthropic's guide lists the kind of work its pilot users should look for, including "test suite creation, documentation updates, code refactoring tasks, API client generation, dependency updates, and routine bug fixes".
  • Recorded history. The ticket tracker and source control already hold the before numbers, so nobody has to rebuild them from memory.

Here is an illustrative example that passes all four. A team fixes a steady stream of billing bugs in its invoicing code. Each bug is a ticket, and each fix merges through a pull request that the same two people review. That workflow runs every week, and its history is already in the tracker.

What goes in the pilot charter?#

The pilot charter is one page with eight fields: the team, the workflow, the deciding number, its baseline, the threshold, the limits, the owner, and the end date with its decision rule. Write it before week one. Have the CTO and the team lead sign it, and date it.

Field What to write Example (illustration)
Team One team that ships together The engineers on the billing service
Workflow One recurring job in its own code Billing bugs from the ticket queue
Deciding number One work number from delivery history Median days from ticket opened to fix merged
Baseline That number for a stretch of ordinary work The eight weeks before week one
Threshold The result that counts as a pass, as a number A specific percentage the CTO and lead write down before week one, with reopened bugs not rising
Limits Repositories in scope, permission rules, data rules The billing service repository, managed settings on
Owner One person who answers questions and keeps the page The team lead
End date and rule The date, and what each result triggers Last day of week four, and the three decisions below

Write the threshold as a number before the pilot starts. A threshold written after the data arrives can be fitted to the data. Pair the deciding number with a quality number, such as reopened bugs, so a better deciding number cannot hide worse work.

Name the limits in terms the tool enforces. Anthropic's permissions page says: "Permission rules are enforced by Claude Code, not by the model. Instructions in your prompt or CLAUDE.md shape what Claude tries to do, but they don't change what Claude Code allows". Put any rule the pilot depends on into settings.

What has to be set up before week one?#

Anthropic's admin setup page lists five deployment decisions: choose your API provider, decide how settings reach devices, decide what to enforce, set up usage visibility, and review data handling. Make all five for the pilot team before the first session, so the pilot runs on the setup the rest of the company would get.

Provider and seats#

The admin setup page calls Claude for Teams or Enterprise "the default recommendation" for teams that want "Claude Code and claude.ai under one per-seat subscription with no infrastructure to run". It points API-first teams, or those that "want pay-as-you-go billing", to the Claude Console. It points teams that want to "inherit existing AWS compliance controls and billing" to Amazon Bedrock.

Settings and limits#

The same page says "Claude Code enforces organization policy through managed settings that take precedence over local developer configuration". Put the charter's limits there. Then the pilot tests the policy the next team will get.

Usage visibility#

Turn on analytics before the first session. Anthropic's analytics page says contribution metrics need setup: "You need the Owner role to configure analytics settings. A GitHub admin must install the GitHub app". It adds that "Data typically appears within 24 hours after enabling, with daily updates".

These scope notes from that page matter for a pilot. Contribution metrics "are in public beta and available on Claude for Teams and Claude for Enterprise plans". They also "cover repositories hosted on github.com". The page says they "only cover users within your claude.ai organization" and that usage through the Claude Console API or third-party integrations "is not included". It says they are "not available for organizations with Zero Data Retention enabled", and the dashboard "will show usage metrics only".

Data handling#

Run the pilot on company accounts. Anthropic's data usage page says it "does not train generative models using code or prompts sent to Claude Code under commercial terms, unless the customer has chosen to provide their data to us for model improvement". The page lists commercial users as "Team and Enterprise plans, API, 3rd-party platforms, and Claude Gov".

Personal accounts follow a different rule. For Free, Pro, Max and Ultra accounts, the same page says Anthropic will "train new models using data" from them "when this setting is on".

Project instructions#

Give the pilot repository a CLAUDE.md file. Anthropic's memory page says a project CLAUDE.md holds instructions that "are shared with your team through version control". It also says "Run /init to generate a starting CLAUDE.md automatically".

What happens in each of the four weeks?#

Week one sets up the team and lands a first change. Weeks two and three are the workflow itself, with a check at the midpoint. Week four reads the result against the charter and writes down the decision.

Week Focus What the week produces
1 Setup and a first task Settings in place, a CLAUDE.md in the repository, one merged change
2 The workflow, every day Notes on prompts and settings that worked and ones that failed
3 The workflow, with a midpoint check A midpoint read of the deciding number, and settings fixed where they blocked work
4 Result and decision The deciding number against the baseline and the decision on the charter

Keep the workflow fixed for all four weeks, so the end number compares like with like. Keep the pilot inside normal delivery too. At GSC, the case study says the cohort "fit inside a normal workweek, roughly five to six hours per engineer, so delivery commitments held".

Give the team one place to ask questions and one person who answers them. Anthropic's rollout guide suggests "a dedicated Slack or Teams channel" for the pilot group to share what works and troubleshoot problems.

Gnar's AI training page describes how an Immerse cohort runs across its four weeks. Each engineer "is assessed at kickoff on how they specify work, delegate it, check the output and handle safety and cost". Then "A senior Gnar mentor pairs with each engineer on their own project, with a weekly cohort check-in and a Slack channel between sessions". The page adds: "Engineers are retaken at midpoint and wrap-up, so each one sees how far they moved".

Which numbers should decide the pilot?#

One work number from the team's delivery history should decide the pilot, with Claude Code's usage numbers read beside it. Usage numbers show whether the team used the tool. The work number shows whether the work changed.

Prefer a deciding number that a system records without the team's input, such as the ticket tracker or an automated reviewer. A number the team reports about itself is easier to argue about later.

Number Where it comes from What it tells you
Deciding work number Ticket tracker or source control Whether the workflow changed
Quality number Reopened bugs or change fail rate Whether the change cost quality
Daily active users and sessions Anthropic's analytics dashboard Whether the team kept using it
PRs with Claude Code Contribution metrics, with the GitHub app How much merged work it touched
Spend Teams and Enterprise spend report, once usage credits are turned on; Cost page on the Console; your cloud billing console What the team cost to run

DORA defines change fail rate as "The ratio of deployments that require immediate intervention following a deployment". Keep the quality number beside the deciding number, so a gain on one cannot hide a loss on the other.

A pilot on a cloud provider reads its usage elsewhere. Anthropic's costs page says "Claude Code does not send metrics from your cloud back to Anthropic, so the analytics dashboards and the Claude Code Analytics API do not cover this usage".

Read the pull requests and lines of code that Anthropic credits to Claude Code with its caveat. Its analytics page says "These metrics are deliberately conservative", and that they "represent an underestimate". "Only lines and PRs where there is high confidence in Claude Code's involvement are counted". The page suggests using them "alongside DORA metrics, sprint velocity, or other engineering KPIs".

Know what the spend figure covers on your plan. On Teams and Enterprise, Anthropic's costs page says the spend report "covers usage-credit spend" and that "Usage inside the seat allowance isn't metered in dollars".

What did a four-week Claude Code cohort look like at GSC Enterprises?#

GSC Enterprises, the wholesale distributor behind Grocery Supply Company, ran a four-week Gnar Immerse cohort for five engineers on their own production code. GSC was not testing whether to buy Claude Code. The case study says GSC "had already put Claude Code in the hands of its engineering team".

The case study names the gap: "What it did not have was a shared way of working with it". It also says: "Each of them had adopted Claude Code, and each used it differently". The team supports a custom-built ERP on Node.js and AWS, integrations with LANSA on IBM i and DB2, and analytics in Snowflake.

A single senior Gnar mentor, Alex, ran the cohort. The case study lists "two 45-minute 1:1s per engineer per week, each pairing on that engineer's live project", a weekly cohort check-in, live-coding sessions and open office hours, and a Slack channel between sessions.

Each engineer was assessed at intake and again at close against the Immerse Four Ds: Description, Delegation, Discernment and Diligence. Each dimension is placed in one of four bands: Craft, Direct, Systematize, or Orchestrate. By the end, all five self-assessed at Systematize or Orchestrate on every dimension.

The case study's line on the external signal is: "The clearest external signal came from a tool nobody touched". GSC runs an independent code review agent against every pull request. "PRs with review-agent comments went from roughly 100% to less than 10% over the four weeks". Gnar credits the drop to "better up-front specification and tighter context discipline".

Each engineer left with "a merged change in their own codebase, a maintained context file, at least one reusable skill, and a workflow they can describe". The case study says "Claude Code is now part of the team's daily production workflow". After the cohort, the participants "ran an internal AI training for 70 GSC leaders".

The case covers one team of five engineers over four weeks, and the training the engineers then ran for 70 leaders. Read it as evidence of what one team's four weeks can measure.

What should the CTO decide when the four weeks end?#

Read the deciding number against the threshold, check whether the team still used Claude Code in week four, and apply the rule written before week one. The rule should map each result to one of three decisions: buy seats for the next team, run the pilot again with one change, or stop and find out why.

Result Decision What happens next
Threshold met, team still using it Buy seats for the next team Copy the settings, the CLAUDE.md approach and the charter
Threshold missed, team still using it Run again with one change Change the workflow, the setup or the training, and keep the rest
Team stopped using it Stop and find the cause Talk to the team before any more seats are bought

Change only one thing in a second run. If the workflow, the setup and the training all change at once, the second result cannot tell you which change mattered.

Where usage is billed per token through the Console or a cloud provider, replace Anthropic's averages with the pilot's own spend figure before you budget the next team. Gnar's post on what Claude Code costs per developer sets a re-forecast date at the end of a pilot.

When does outside help make sense for a pilot?#

Outside help makes sense for a pilot team that already has the tools but not the practice, the team Gnar's AI training page says its cohort is built for. Gnar says: "If your own team has the capacity and the direction, you do not need us".

When the team does want help, Gnar's AI training page describes Immerse as "Built for a small team that already has the tools but not the practice". It also says "Immerse isn't a course we wrote for clients. We built it to train our own engineering team into Context-Driven Development".

For leaders who are not ready for a cohort, the same page's answer is: "Take the free AI Readiness Assessment, or join a free Claude workshop".

Frequently asked questions

What is a Claude Code pilot?

A Claude Code pilot is a time-boxed trial in which one engineering team uses Claude Code on its own code, so the company can decide whether to buy seats for more engineers. Run it for four weeks on one recurring workflow, with the deciding number and the decision rule written down before it starts.

How long should a Claude Code pilot run?

Plan on four weeks for one team. Use week one to set up and land a first change, weeks two and three on the workflow with a midpoint check, and week four to read the result against the threshold written before the start. Keep the same workflow from start to finish.

How many engineers should be in a Claude Code pilot?

Put one whole team in it, the engineers who ship one codebase through one review process. Anthropic's rollout guide, which reflects its insights as of August 2025, suggests a pilot group of 20 to 50 developers who are already comfortable with AI-assisted tools. A company with one engineering team can pilot with that team.

Does Anthropic train on code sent to Claude Code on a Team or Enterprise plan?

No, unless the customer has chosen to provide its data for model improvement. Anthropic's Claude Code data usage page says it does not train generative models using code or prompts sent to Claude Code under commercial terms. It lists Team and Enterprise plans, the API, third-party platforms and Claude Gov as commercial.

What has to be turned on before the analytics dashboard shows pull requests?

A GitHub admin installs the Claude GitHub app, an Owner turns on Claude Code analytics and GitHub analytics, and the setup then asks you to complete the GitHub authentication flow and select which GitHub organizations to include. Anthropic says data typically appears within 24 hours. Contribution metrics are in public beta on Claude for Teams and Claude for Enterprise, and they cover repositories hosted on github.com. For organizations with Zero Data Retention enabled, the dashboard shows usage metrics only.

Can we run a Claude Code pilot with our own engineers?

Yes. Gnar says "If your own engineers have the capacity and the direction, do it internally and we will say so on the call." The Gnar Company is a Claude Select Services Partner. Its Immerse cohort is built for a small team that already has the tools but not the practice, with a senior Gnar mentor pairing on each engineer's own project.

Stay In The Loop

Never miss an update. Get the latest blogs, webinars, and expert insights straight to your inbox from The Gnar.