How to Write Decision Questions for Jev and Laya

Quick answer
The question set is the API. The model scores the options and levels you supply, so their wording does most of the work.
Choose the answer shape first: Choice for unordered labels, Score for ordered levels, Noul for a yes/no probability.
Ask one judgment per question. Send several questions in one call; they run in parallel and cost only their tokens.
Describe options by when they apply, not by vague intensity. Give boundary examples where two options are easy to confuse.
Test option order, paraphrases, ambiguous inputs, and calibration before you tune a confidence threshold.

Why question design matters more than the prompt

With a chat model, the prompt is a request and the output is text. With a decision model, the question and its criteria are the entire interface. You are not asking the model to write an answer; you are asking it to score your declared answers against the state. That changes what good instructions look like. A clever prompt with no defined options is a misuse of the tool, while a plain question with sharply written options is often enough.

The practical consequence is that most quality work happens before the API call. If two options overlap, the model will split probability between them. If a score level describes three different things, a state that is high on one and low on another cannot be placed. If a yes/no question hides two conditions, the probability has no clean meaning. The model can only be as precise as the distinctions you give it.

Choose the answer shape first

Pick the type that matches the decision your code has to make. Choice is for an unordered set of labels, such as a department, a document type, or a routing destination. Score is for an ordered spectrum, such as severity, frustration, or experience, where a position between levels is meaningful. Noul is for a single proposition that resolves to a probability, such as whether a message requests a refund or contains personal data.

A useful test is to ask what the next line of code does. If it switches on a label, use Choice. If it compares a number to a threshold, use Score. If it branches on a probability, use Noul. When two types seem to fit, prefer the one whose answer the code can act on without a second conversion. A Choice between refund, rebook, and information maps directly onto three handlers; a Score of frustration needs a threshold before it becomes a branch.

Answer shapes

  CHOICE   one of a known set, no order
           Which team should handle this ticket?
           returns | shipping | billing | other

  SCORE    a position on an ordered rubric
           How severe is the reported bug?
           0 cosmetic | 1 degraded | 2 blocking

  NOUL     probability that one proposition is true
           Does the customer request a refund?
           0.0 no ... 0.5 unsure ... 1.0 yes

  BOX      preview only: where is the object?
           the coffee mug in `photo`
           returns fractional left/top/right/bottom

Write the instructions as one clear question

Write the complete question in instructions, even when the question ID seems self-explanatory. Question IDs are for your code and are not sent to the model. A question such as refund_requested still needs the sentence “Does the customer request a refund?” because the model never sees the ID. Use the same vocabulary your operators use, and avoid pronouns that could refer to several parts of the state.

When the state is structured, name the part being judged with a backtick path such as ticket.messages[0].text or order.charges. That tells the model which field the question is about and keeps unrelated fields from diluting the judgment. For a long document, say which section matters and send only the relevant passage; a larger state is not automatically a better one.

Ask one proposition per question. “Is the customer angry and asking for a refund?” makes the model judge two things at once, and the answer cannot tell you which one drove the probability. Ask two Nouls and combine them in code. If a question needs a condition, such as “If this is a shipping problem, which kind is it?”, make the dependency explicit and only read the answer when the parent condition is true.

Write Choice options that do not overlap

A Choice criterion is a map from the value your code receives to a description of when that value applies. The key is the label; the description is the boundary. Write descriptions as situations, not adjectives. “Payments, invoices, and refunds” separates billing from “outages, errors, and configuration” better than “billing issues” and “technical issues” do.

Include an other or none_of_the_above option when the taxonomy may not cover every input. The model is constrained to your options, so without an escape hatch it must force an unfamiliar input into the closest available label. If other is not appropriate for the business logic, a companion Noul such as “Does the message name any of our products?” can tell your code whether the Choice answer is meaningful.

Option order can matter at the margins, especially when two descriptions are close. Test a reversed order on the same labeled inputs. If the answer changes for more than a small share of cases, the descriptions are not distinct enough yet. Adding examples inside an option description, such as “a duplicate charge, a failed refund, or an invoice question”, is often more effective than adding more adjectives.

Write Score levels that describe situations

A Score has between two and ten ordered levels. Each level is judged on its own, and the model does not see the level number or its neighbors. That means “worse than the level below” is meaningless to it, and a level that says only “medium” gives it nothing to match against. Describe the situation at that level: “Broken or degraded feature, but a workaround exists” is a level; “moderately severe” is not.

Keep each Score to one dimension. A level that says “punctual, smart, and experienced” measures three things, and an input that is high on one and low on another cannot be placed. Split it into separate Scores and combine them with weights in code. If the top of the scale has a rare case that needs different handling, give that case its own level rather than expecting the score to distinguish it.

The returned score is a probability-weighted mean of the level numbers, so it can fall between levels. A score of 1.4 does not mean “40% of the way from good to bad” in any physical sense; it is the average position under the model’s distribution. Read the probabilities and confidence alongside it. If your code needs a discrete level, round explicitly and log the raw score so the rounding is auditable.

Write Noul questions with a clean boundary

A Noul returns the probability that a proposition is true, not a label and not a separate confidence value. Phrase it so a high value means yes. “Does the message contain personal data?” is clear; “Is the message free of personal data?” inverts the meaning and invites a downstream bug. A statement works as well as a question: for “The customer is requesting a refund”, a value near one means the statement is true.

If the boundary between yes and no is subtle, add criteria with true and false descriptions. This is optional for a Noul, and the instruction alone is enough for many questions. Use criteria when reasonable people could disagree about what counts, such as whether a prior contact counts as a repeat contact. Try the question with and without criteria on your own data and keep the version that separates the cases better.

Do not use a Noul to measure a degree. “Is the candidate strong in Python?” returns the probability of “strong”, not a skill level. If you need a position, use a Score with levels such as no experience, some familiarity, regular use, and deep expertise. A Noul value near 0.5 means the model is unsure, not that the candidate is average.

Ask several atomic questions in one call

Decision models evaluate every question in a request independently and in parallel. Adding a question barely changes response time and costs only the tokens for that question. That makes the efficient pattern obvious: put every judgment the code might need into one request, including speculative questions whose answers only matter for some inputs. Your code ignores the ones it does not use.

Atomic questions are also easier to debug. If a routing decision is wrong, you can see whether the department Choice was wrong, whether the urgency Noul was low, or whether the resolution Choice was ambiguous. A compound question collapses those signals into one number and makes the threshold impossible to reason about. The extra question costs a few tokens; the diagnosis is worth much more.

Set thresholds from costs, not from a default

For a Noul, the probability is the answer. Choose the threshold from the cost of a false positive versus a false negative. If acting on a false yes is expensive, such as issuing a refund or paging an engineer, raise the threshold and send the middle band to review. If missing a true yes is expensive, such as failing to flag a safety issue, lower it. The threshold belongs in code, next to the action it gates.

For a Choice or Score, the answer includes a confidence value derived from the probability distribution. Use a three-band policy: high confidence acts automatically, medium confidence asks for confirmation or gathers more information, and low confidence goes to a person or a fallback model. Set different thresholds for different actions in the same request. A read-only action can tolerate more uncertainty than a destructive one.

Do not copy a threshold from a tutorial. The same confidence value can be acceptable for routing a newsletter signup and unacceptable for approving a transfer. Tune on a labeled set that includes ambiguous cases, and record the threshold with the question-set version so a later change is traceable.

Test the question set like code

Start with 100 to 300 labeled inputs from your own traffic. Include easy cases, boundary cases, and cases where your human reviewers disagreed. Measure accuracy against the labels, calibration by confidence band, and the share of answers that fall below the action threshold. A question set that is accurate but always low-confidence is not usable without a review queue; one that is confident and wrong is worse.

Test paraphrases, option order, and state formatting. If the same ticket gets a different label when the options are reversed, the descriptions are not doing enough work. If adding an irrelevant field changes the answer, the question is not pointing at the right part of the state. If a direct instruction in the state can flip the label, treat the input as adversarial and add a policy check outside the model.

Keep the question set, model version, and evaluation set in the repository. When you change a label, a level, or a threshold, that is a new decision schema, not a copy edit. Re-run the evaluation and compare the confusion matrix before shipping. The version you log with each production decision should identify the question set as well as the model.

A complete question set in Python

The example below asks three questions about one support message in a single call. It uses a Choice for the queue, a Noul for refund intent, and a Score for frustration. The code gates on confidence and handles the middle band explicitly. The question IDs are local names; the instructions are the full questions the model sees.

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

QUESTIONS = {
    "queue": Choice(
        instructions="Which team should handle `message`?",
        criteria={
            "billing": "Charges, invoices, refunds, and payment failures.",
            "technical": "Bugs, outages, integrations, and configuration.",
            "account": "Login, password, profile, and access problems.",
            "other": "A request that fits none of the above.",
        },
    ),
    "refund_requested": Noul(
        instructions="Does `message` ask for money back?",
    ),
    "frustration": Score(
        instructions="How frustrated is the customer in `message`?",
        criteria=[
            "Calm and factual.",
            "Concerned or annoyed, but civil.",
            "Angry, using strong language, or threatening to leave.",
        ],
    ),
}

def triage(message: str) -> None:
    with TypeSafeClient() as client:
        result = client.system_one(
            model="jev-1.14.0",
            state={"message": message},
            questions=QUESTIONS,
        )

    queue = result.answers["queue"]
    refund = result.answers["refund_requested"].noul
    frustration = result.answers["frustration"]

    if queue.confidence < 0.4:
        send_to_manual_triage(message)
    else:
        assign(message, queue=queue.choice)

    if refund > 0.8:
        flag_for_refund_review(message)
    elif refund > 0.4:
        ask_customer_to_confirm(message)

    if frustration.score >= 1.8 and frustration.confidence >= 0.5:
        notify_senior_agent(message)

Where this bites vibecoders
A coding assistant will often write the first version as one prompt that asks for a JSON object with a department, a priority, and a refund flag. That throws away the property that makes a decision model useful: separate, typed answers you can inspect and threshold independently. Write the questions first, test them on twenty real inputs, and only then wire the answers into handlers.

Where AI coding assistants get this wrong

Asking one compound question instead of several atomic ones.
Using adjectives such as “urgent” or “severe” without describing the situations that qualify.
Forgetting an other or none-of-the-above option when the taxonomy is incomplete.
Using a Noul to measure a degree instead of a Score.
Setting a confidence threshold from a tutorial instead of from the cost of a wrong action.
Changing an option label without re-running the labeled evaluation set.

Checklist

Write down the code branch before choosing the question type.
Give every option a description that separates it from its neighbors.
Use one proposition per question and one dimension per Score.
Add an escape option or companion Noul for inputs outside the taxonomy.
Test option order, paraphrases, and ambiguous cases.
Record the question-set version with every production decision.
Re-run the evaluation whenever a label, level, or threshold changes.

FAQ

Should the question ID be a full sentence?

No. The ID is a local key for your code and is not sent to the model. Keep it short and stable, and put the complete question in instructions. A readable ID still helps logs and dashboards, but it is not a substitute for a clear instruction.

How many options can a Choice have?

TypeSafe documents up to 255 options for a Choice question. More options cost more tokens and make neighboring descriptions easier to confuse, so use the smallest set that covers the decision. For a large taxonomy, chain Choice questions level by level instead of putting every category in one list.

How many levels should a Score have?

Use as many as you can describe distinctly, up to ten. Three to five is common. Do not add levels you cannot describe with a concrete situation; an undescribed middle level gives the model nothing to match and lowers confidence without improving the decision.

When should I add true and false criteria to a Noul?

Add them when the boundary is subtle, such as whether a previous contact counts as a repeat contact, or when your operators disagree about what qualifies. The instruction alone is enough for many Nouls. Test both versions on your own labeled inputs and keep the one that separates the cases better.

Can I change the questions after launch?

Yes, but treat a change to a label, level, or instruction as a new decision schema. Version the question set, re-run the labeled evaluation, and roll it out behind the same kind of check you would use for a model upgrade. Changing a criterion can change the meaning of every historical answer.