Brief №029 Audit Packet

The Brief argues that when AI takes over starter work, employers should decide which learning action stays with the employee. Its proposal is to keep the employee's attempt and reasoning ahead of the answer, then test whether that practice earns its cost. The proposal is a design to pilot, not a finding.

Draft assessment: the Brief's source-supported claims are stated within what the four sources show. The strongest evidence is short-run. It covers immediate comprehension and near transfer in one coding task, two classroom settings and one customer-support rollout. The employer sequence, the selection rule and the 12-month forecast are Autonoma analysis. No cited study tested them. This assessment is provisional and is not an editorial or Auditor judgment.

Draft verdict: the Brief's empirical sentences stay within what the sources report. Its recommendation is a pilot proposal, not an empirical finding.

Types: fact means source-supported finding. Association means nonrandomized pattern. Synthesis, forecast and dissent are Autonoma reasoning, not verified empirical findings.

• [1] Shen and Tamkin, "How AI Impacts Skill Formation." Preprint, arXiv 2601.20245, with author research summary dated 29 January 2026. https://arxiv.org/abs/2601.20245 Author summary: https://www.anthropic.com/research/AI-assistance-coding-skills
• [2] Bastani and colleagues, "Generative AI without guardrails can harm learning: Evidence from high school mathematics." Journal article, PNAS, published online 25 June 2025. https://www.pnas.org/doi/10.1073/pnas.2422633122
• [3] Brynjolfsson, Li and Raymond, "Generative AI at Work." Journal article, Quarterly Journal of Economics, published online 4 February 2025. https://doi.org/10.1093/qje/qjae044 Public author PDF: https://danielle.li/assets/docs/GenerativeAIatWork.pdf
• [4] Kestin and colleagues, "AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting." Journal article, Scientific Reports, 3 June 2025. https://www.nature.com/articles/s41598-025-97652-6
• Designs: randomized experiment, classroom randomized trial, staggered-rollout quasi-experiment, randomized crossover, in that order. Source [1] is vendor-affiliated.

• Mathematics only. Only the Turkish study compares ordinary chat with designed tutoring. The 17% figure is a relative deficit, not 17 percentage points. No other cited study makes that comparison.
• Speed. A result that is not significant is not a zero effect. The coding study leaves room for some speed effect.
• No demonstrated advantage. The designed math tutor was not shown to beat control. That does not prove equivalence or rule out benefit.
• Physics caveat. Home self-paced use differed from in-person teaching in the physics trial only. The result is an immediate post-test on a bundle, not a test of answer policy.
• Outages. The workplace evidence is noisy duration data from rare outages. It is suggestive. It does not show maintained service quality or lasting skill.
• Immediate learning is not retention. Immediate comprehension, near transfer, delayed retention and far transfer are separate outcomes. The sources chiefly address the first two.
• Untested combination. Our employer sequence was not tested by any cited study. Both classroom treatments changed several features at once.

The strongest dissent is that unaided practice may test the wrong skill. If employees will always work with AI, a quiz or exam that bars AI measures something the job never asks for. The coding quiz and the math exam both barred it. 12

Cost sharpens the objection. Expert-checked feedback uses the senior time automation was meant to free. The physics tutor took work to build. 4 Support agents appeared to gain from ordinary use. 3 For many tasks, automating and moving on may be the sound answer. That cost argument is our reasoning, and no cited study measured it.

Limit: the dissent is not refuted by the evidence. We accept part of it. The later check should match real work, with assistance stated. Where a pilot shows no difference worth its cost, drop the practice.

These results would weaken the employer proposal:

• Attempt-first groups show no better result than ordinary assisted work on a different case with assistance stated.
• A later retention check shows the gain fades to the level of plain assistance.
• Expert feedback cost exceeds the value of any gain the business can measure.
• Trials that separate attempt, reasoning and feedback show the sequence adds nothing beyond ordinary explanation-seeking.
• No role has a judgment that stays needed and stays hidden in ordinary work, leaving the selection rule empty.
• Delayed retests show that immediate gains in the classroom trials do not persist.

Analysis, not a finding. The Brief forecasts that over the next 12 months, employer claims about AI learning will rest mostly on assisted output and immediate quizzes. It expects the first credible workplace evidence to come from small pilots comparing attempt-first designs with ordinary assistance. This is Autonoma's analytical forecast about employer claims and small pilot evidence. No source supports it, and it gives no adoption numbers.

This Audit checks the approved Brief against saved source excerpts and a source ledger covering a preprint with an author summary, two classroom trials and a published workplace study. Claims are labeled as fact, association, synthesis, dissent, limit or forecast, and each row states its limitation. This Audit draft was prepared with an AI assistant and awaits editorial review and approval.