Research design
How each question will be answered
For each question the study asks: the data that answers it, how it will be analysed, and what kind of result would answer it. It’s set out before the results are in, so the questions can’t be bent to fit them.
Version 2, 13 November 2026 (written during the pilot, before the study opens to everyone; version 2 adds time saved, work that gets used, employer support and experience)
What “real results” means here
No single measure is perfect: an AI can misjudge, and people can be generous about themselves. The study uses three, and is more confident where they agree.
Level
Your AI’s assessment of your level, from novice to expert, based on the evidence it can see.
- Report summaryYour level: novice, experimenter, practitioner or expert
proficiency_level
Impact
Whether AI has made your work better or worse, in your own view.
- Your answersWhether AI has made your work better or worse (question 5)
Time saved
About how much work time AI saves you in a typical week, in your own estimate.
- Your answersTime AI saves you in a typical week (question 5)
Work that gets used
Whether there’s evidence that work done with AI was actually used: sent, adopted or run, not just produced.
- Report summaryEvidence that work done with AI was actually used (sent, adopted or run): none, some, clear or unknown
results_evidence
Question 1 of 3
What habits do people who get real results from AI have in common?
1 The data
How you brief AI
- Report summaryHow long your messages usually are, in characters
median_prompt_length_chars - Report summaryWhether you ask for a plan before it starts: often, sometimes, rarely or never
asks_for_plan_first - Report, Part 2How I brief you
How you check its work
- Report summaryRoughly how often you check its work against reality, for example “~20%”
verification_rate - Report, Part 2How I check your work
How you push back and keep going
- Report summaryRoughly how often you question or correct it
pushback_rate - Report summaryHow many back-and-forths a task usually takes
typical_rounds_per_task
How you keep what you learn
- Report summarySaved instructions it follows: none, basic or extensive
persistent_instructions - Report summaryHow many saved rules came from fixing its mistakes
rules_from_corrections
How often you use it
- Your answersHow often you use AI (question 4)
To rule out experience alone
- Your answersHow long you’ve used AI for work (question 4)
2 The analysis
Compare each habit between people with stronger and weaker results, among people with similar tools and similar jobs.
3 What would answer it
- If a habit is consistently more common among people with stronger results, across different tools and jobs, it’s a real candidate.
- If no habit stands out, results may depend more on the tools or the work itself, which is question 2.
Question 2 of 3
How much comes down to the person, the tools they have, or the work itself?
1 The data
The person
- Report, Part 2The habits from question 1
The tools
- Your answersWhich AI tools you use (question 2)
- Your answersWhat your AI can do (question 2)
- Your answersWhether your employer provides your AI tool (question 2)
- Report summaryWhat your AI could see: your files and history, your history only, or just this chat
access_level - Report summaryWhether your AI can run code: yes or no
env_can_run_code - Report summaryWhether it can reach real files or data: yes or no
env_can_access_real_data - Report summaryWhether it remembers past chats: yes or no
env_has_memory - Report summaryWhether you use several AIs, or have one review another’s work: yes or no
uses_multiple_agents_or_reviews
The work
- Your answersYour role (question 1)
- Your answersYour industry (question 1)
- Your answersOrganization size (question 1)
- Report summaryYour kind of work in a word or two, for example “operations” or “marketing”
primary_work_domain - Report summaryWhether you write software for a living: yes, no or unknown
user_is_professional_developer
2 The analysis
Compare people with the same tools but different habits, and people with similar habits but different tools or jobs.
3 What would answer it
- If people with the same tools get very different results, habits matter more.
- If results rise and fall with what the tools can do, or with the kind of work, those matter more. Most likely it’s a mix; the aim is to estimate how much of each.
Question 3 of 3
How can organizations find and support the people who do this well?
1 The data
What people build and share
- Report summaryWhether you’ve built anything reusable with AI: yes or no
builds_tools - Report summaryHow many tools, templates or automations you’ve built that get used more than once
things_built_count - Report summaryWhether other people reuse anything you’ve built: yes, no or unknown
outputs_used_by_others
What helps
- Your answersTraining, guidance, time and help your workplace gives you (question 3)
- Your answersWhether your employer provides your AI tool (question 2)
What gets in the way
- Report summaryLimits your workplace puts on AI, like blocked tools or no access to real data: none seen, some, severe or unknown
workplace_limits - Report, Part 2What hurts quality or wastes time
- Report summaryWhether it can try things somewhere safe first: yes or no
env_has_safe_test_environment
What would break if they stopped
- Report, Part 2Fragility (history version)
2 The analysis
Look for signs an organization could see from the outside, like tools people build and others reuse, and check whether they line up with stronger results. Then look at which workplace limits hold capable people back.
3 What would answer it
- If reusable tools and saved instructions reliably mark the people with the strongest results, organizations have something concrete to look for.
- If common workplace limits keep capable people at lower levels, that’s something organizations can change.
What could make the study wrong
Every study has limits. These are this one’s, stated up front.
- Who takes part
- People who choose to take part, many from the researcher’s network. It isn’t a random sample of all workers, so findings describe this group.
- The AI’s judgement
- Each report is written by the participant’s own AI and can be wrong or generous. Estimates like “~20%” are rough.
- Different tools see different evidence
- Some AIs review months of history; others only ask a few questions. Each report records what it was based on, so like can be compared with like.
- What goes together, not what causes what
- The study can show which habits and results go together. It can’t prove one causes the other.
- Small numbers
- Any group with fewer than 5 people is combined with others or left out of that breakdown.
- Levels are partly defined by habits
- The four levels are partly described by habits like building tools, so some links between those habits and level are built in. That’s why the study also measures impact, time saved and work that gets used.
- Changes during the pilot
- The questions were refined during the pilot (version 2). Responses to the earlier version are analysed separately.
Help answer these questions
Every level of experience counts, including “I rarely use AI”. About 15 to 20 minutes.
Take part in the study