Study Design Guide

A useful UserTold study has a clear reason to meet users and stays short enough to produce reviewable evidence. It can observe ordinary use, test a specific feature or task, or explore user needs in an open conversation. This guide covers the design choices. Use Study Runtime for advancement behavior and Research Script Templates for more starting points.

Start with the kind of encounter

Choose how you want to learn before writing segments:

  • observe ordinary product use without assuming where the problem is;
  • ask someone to try a specific feature or complete a real task;
  • invite an open conversation about recent behavior and unmet needs.

Then decide who should take part, which product context matters, and what the interviewer should remember. Prefer three to five specific goals. “Understand the user” is too broad. “Learn how first-time buyers navigate checkout and what they expect along the way” gives the interview and reviewer useful context without assuming where a problem exists.

Choose the right mode

Each segment has one job:

ModeUse it forAvoid
talkRecent behavior, motivations, decisions, and a planned debriefHypothetical feature voting or leading questions
speakA short scripted welcome, instruction, transition, or closeBack-and-forth conversation
observeSilent capture while the participant completes one real taskCoaching or rescuing the participant

For usability research, start with speak → observe → talk → speak. Give the task without hints, observe without interruption, then ask about concrete moments from the task.

For discovery research, use a small number of focused talk segments. Ask about the last real occurrence before asking for opinions or imagined future behavior.

Write each segment

Every script uses version: 2, a goals array, and an ordered segments array. Every segment needs a unique id, a title, and a mode.

Add only the fields the mode needs:

  • talk: use talk.system_prompt for focused interviewer behavior and talk.goals to assign study goals.
  • speak: provide concise speak_text.
  • observe: provide a participant-facing instruction or AI-only conductor_context; in practice, use both.

An observe instruction should describe one outcome without revealing the path. For example:

Complete checkout from your cart to the confirmation screen.

conductor_context is not shown to the participant. Use it to describe the expected flow, product terminology, and likely failure points so the planned debrief can interpret what happened.

Make advancement deterministic

An observe segment should have a clear completion path and a time limit. Production supports:

  • url:<substring>;
  • action:<selector-or-pattern>;
  • participant Done / step_done;
  • max_duration_s as a safety valve.

Talk segments end through complete_segment; speak segments advance after scripted playback completes. Goals guide the interview and later analysis, but they are not live advancement rules.

Complete example

{
  "version": 2,
  "goals": [
    {
      "id": "checkout-friction",
      "description": "Identify where buyers hesitate or fail during checkout and what they expected instead."
    }
  ],
  "segments": [
    {
      "id": "intro",
      "title": "Task instructions",
      "mode": "speak",
      "speak_text": "Please complete checkout as you normally would and think aloud while you work."
    },
    {
      "id": "checkout",
      "title": "Complete checkout",
      "mode": "observe",
      "instruction": "Complete checkout from your cart to the confirmation screen.",
      "conductor_context": "Expected flow: cart, delivery, payment, review, confirmation. Preserve hesitation, errors, backtracking, and recovery behavior for the debrief.",
      "advance_when": "url:/checkout/confirmation",
      "max_duration_s": 420
    },
    {
      "id": "debrief",
      "title": "Debrief",
      "mode": "talk",
      "talk": {
        "goals": ["checkout-friction"],
        "system_prompt": "Ask about specific pauses, errors, or backtracking from the observed task. Ask what the participant expected and what happened instead. Do not suggest fixes."
      }
    },
    {
      "id": "thanks",
      "title": "Thanks",
      "mode": "speak",
      "speak_text": "Thanks for completing the interview."
    }
  ]
}

Common failures

ProblemCorrection
The research question covers several workflowsSplit it into separate studies
The instruction tells the participant where to clickState the outcome, not the route
Observation starts without contextAdd a short speak instruction first
Observation ends without a debriefFollow it with a goal-linked talk segment
The interviewer asks generic questionsName the concrete behavior and expected evidence in talk.system_prompt
The observe segment can run indefinitelyAdd a deterministic rule and max_duration_s
The script contains many repeated goals or promptsKeep one responsibility per goal and segment

Validate before launch

Run the study yourself through the same intake and embed participants will use. Confirm:

  1. the participant understands the task without being shown the solution;
  2. every goal belongs to at least one talk segment;
  3. observation stays silent and has an exit path;
  4. the debrief can refer to captured behavior;
  5. the interview reaches completion and produces a reviewable source record;
  6. allowed origins include every product origin where the widget runs.

Then activate the study and its linked intake. After the first real interview, inspect the recording, transcript, timeline, and extracted evidence before increasing recruitment.

Continue