Research design

How each question will be answered

For each question the study asks: the data that answers it, how it will be analysed, and what kind of result would answer it. It’s set out before the results are in, so the questions can’t be bent to fit them.

Version 2, 13 November 2026 (written during the pilot, before the study opens to everyone; version 2 adds time saved, work that gets used, employer support and experience)

What “real results” means here

No single measure is perfect: an AI can misjudge, and people can be generous about themselves. The study uses three, and is more confident where they agree.

Level

Your AI’s assessment of your level, from novice to expert, based on the evidence it can see.

  • Report summaryYour level: novice, experimenter, practitioner or expert proficiency_level

Impact

Whether AI has made your work better or worse, in your own view.

  • Your answersWhether AI has made your work better or worse (question 5)

Time saved

About how much work time AI saves you in a typical week, in your own estimate.

  • Your answersTime AI saves you in a typical week (question 5)

Work that gets used

Whether there’s evidence that work done with AI was actually used: sent, adopted or run, not just produced.

  • Report summaryEvidence that work done with AI was actually used (sent, adopted or run): none, some, clear or unknown results_evidence

Question 1 of 3

What habits do people who get real results from AI have in common?

1 The data

How you brief AI

  • Report summaryHow long your messages usually are, in characters median_prompt_length_chars
  • Report summaryWhether you ask for a plan before it starts: often, sometimes, rarely or never asks_for_plan_first
  • Report, Part 2How I brief you

How you check its work

  • Report summaryRoughly how often you check its work against reality, for example “~20%” verification_rate
  • Report, Part 2How I check your work

How you push back and keep going

  • Report summaryRoughly how often you question or correct it pushback_rate
  • Report summaryHow many back-and-forths a task usually takes typical_rounds_per_task

How you keep what you learn

  • Report summarySaved instructions it follows: none, basic or extensive persistent_instructions
  • Report summaryHow many saved rules came from fixing its mistakes rules_from_corrections

How often you use it

  • Your answersHow often you use AI (question 4)

To rule out experience alone

  • Your answersHow long you’ve used AI for work (question 4)

2 The analysis

Compare each habit between people with stronger and weaker results, among people with similar tools and similar jobs.

3 What would answer it

  • If a habit is consistently more common among people with stronger results, across different tools and jobs, it’s a real candidate.
  • If no habit stands out, results may depend more on the tools or the work itself, which is question 2.

Question 2 of 3

How much comes down to the person, the tools they have, or the work itself?

1 The data

The person

  • Report, Part 2The habits from question 1

The tools

  • Your answersWhich AI tools you use (question 2)
  • Your answersWhat your AI can do (question 2)
  • Your answersWhether your employer provides your AI tool (question 2)
  • Report summaryWhat your AI could see: your files and history, your history only, or just this chat access_level
  • Report summaryWhether your AI can run code: yes or no env_can_run_code
  • Report summaryWhether it can reach real files or data: yes or no env_can_access_real_data
  • Report summaryWhether it remembers past chats: yes or no env_has_memory
  • Report summaryWhether you use several AIs, or have one review another’s work: yes or no uses_multiple_agents_or_reviews

The work

  • Your answersYour role (question 1)
  • Your answersYour industry (question 1)
  • Your answersOrganization size (question 1)
  • Report summaryYour kind of work in a word or two, for example “operations” or “marketing” primary_work_domain
  • Report summaryWhether you write software for a living: yes, no or unknown user_is_professional_developer

2 The analysis

Compare people with the same tools but different habits, and people with similar habits but different tools or jobs.

3 What would answer it

  • If people with the same tools get very different results, habits matter more.
  • If results rise and fall with what the tools can do, or with the kind of work, those matter more. Most likely it’s a mix; the aim is to estimate how much of each.

Question 3 of 3

How can organizations find and support the people who do this well?

1 The data

What people build and share

  • Report summaryWhether you’ve built anything reusable with AI: yes or no builds_tools
  • Report summaryHow many tools, templates or automations you’ve built that get used more than once things_built_count
  • Report summaryWhether other people reuse anything you’ve built: yes, no or unknown outputs_used_by_others

What helps

  • Your answersTraining, guidance, time and help your workplace gives you (question 3)
  • Your answersWhether your employer provides your AI tool (question 2)

What gets in the way

  • Report summaryLimits your workplace puts on AI, like blocked tools or no access to real data: none seen, some, severe or unknown workplace_limits
  • Report, Part 2What hurts quality or wastes time
  • Report summaryWhether it can try things somewhere safe first: yes or no env_has_safe_test_environment

What would break if they stopped

  • Report, Part 2Fragility (history version)

2 The analysis

Look for signs an organization could see from the outside, like tools people build and others reuse, and check whether they line up with stronger results. Then look at which workplace limits hold capable people back.

3 What would answer it

  • If reusable tools and saved instructions reliably mark the people with the strongest results, organizations have something concrete to look for.
  • If common workplace limits keep capable people at lower levels, that’s something organizations can change.

What could make the study wrong

Every study has limits. These are this one’s, stated up front.

Who takes part
People who choose to take part, many from the researcher’s network. It isn’t a random sample of all workers, so findings describe this group.
The AI’s judgement
Each report is written by the participant’s own AI and can be wrong or generous. Estimates like “~20%” are rough.
Different tools see different evidence
Some AIs review months of history; others only ask a few questions. Each report records what it was based on, so like can be compared with like.
What goes together, not what causes what
The study can show which habits and results go together. It can’t prove one causes the other.
Small numbers
Any group with fewer than 5 people is combined with others or left out of that breakdown.
Levels are partly defined by habits
The four levels are partly described by habits like building tools, so some links between those habits and level are built in. That’s why the study also measures impact, time saved and work that gets used.
Changes during the pilot
The questions were refined during the pilot (version 2). Responses to the earlier version are analysed separately.

Help answer these questions

Every level of experience counts, including “I rarely use AI”. About 15 to 20 minutes.

Take part in the study

What goes in, what comes out · Findings