Jev Score: The Ordered-Levels Question Type

Last verified against the TypeSafe docs on 22 September 2026 · model jev-1.13.0

What a Score question looks like

A Score question has `type` set to `"score"`, an `instructions` string saying what is being rated, and `criteria`: an ordered array of level descriptions running from the low end of the scale to the high end. The docs say it should have at least two levels, and the API accepts up to 10.

JSON

{
  {
  "state": : "The export button crashes the settings page in Safari. It works in Chrome, but a few of our customers only use Safari.",
  ,
  "questions": {
    : {
    "bug_severity": {
      : {
      "type": : "score",
      ,
      "instructions": : "How severe is the reported issue?",
      ,
      "criteria": [
        : [
        "Cosmetic; no impact to functionality",
        ,
        "Broken or degraded feature, but workaround exists",
        ,
        "Blocking issue; no workaround exists"
      ]
    }
  }
}
      ]
    }
  }
}

Python

from typesafe_sdk  typesafe_sdk import Score, TypeSafeClient

 Score, TypeSafeClient

with TypeSafeClient()  TypeSafeClient() as client:
    response = client.system_one(
        state= client:
    response = client.system_one(
        state="The export button crashes the settings page in Safari. It works in Chrome, but a few of our customers only use Safari.",
        questions={
            ,
        questions={
            "bug_severity": Score(
                instructions=: Score(
                instructions="How severe is the reported issue?",
                criteria=[
                    ,
                criteria=[
                    "Cosmetic; no impact to functionality",
                    ,
                    "Broken or degraded feature, but workaround exists",
                    ,
                    "Blocking issue; no workaround exists",
                ],
            ),
        },
    )

    print(response.answers[,
                ],
            ),
        },
    )

    print(response.answers["bug_severity"].score)].score)

A level's number is its position in the array, starting at 0, so the three entries above are levels 0, 1 and 2. The order of the array is the numbering. The model gets the descriptions and nothing else: it does not see a level's number or its neighbours, and judges each level on its own against the state.

What comes back

JSON

{
  {
  "model": : "jev-1.13.0",
  ,
  "answers": {
    : {
    "bug_severity": {
      : {
      "type": : "score",
      ,
      "score": : 1.43,
      ,
      "confidence": : 0.35,
      ,
      "legend": {
        : {
        "0": : "Cosmetic; no impact to functionality",
        ,
        "1": : "Broken or degraded feature, but workaround exists",
        ,
        "2": : "Blocking issue; no workaround exists"
      },
      
      },
      "probabilities": {
        : {
        "0": : 0.0,
        ,
        "1": : 0.57,
        ,
        "2": : 0.43
      }
    }
  },
  
      }
    }
  },
  "usage": {
    : {
    "input_tokens": : 332,
    ,
    "output_tokens": : 18
  }
}
  }
}

Four fields beyond `type`. `probabilities` gives the probability of each level, keyed by level number, summing to 1. `score` is each level number multiplied by its probability, added up: `0 × 0.0 + 1 × 0.57 + 2 × 0.43 = 1.43`. `legend` maps each level number back to its description. `confidence` is 0 to 1, computed from how the probability is spread.

A score of 1.43 means the model is split between levels 1 and 2, leaning to level 1. That matches the report: the export is broken, switching to Chrome is a workaround for most customers, but not for the ones who only use Safari.

The range is 0 to the top level, not 0 to 1

This is the most common thing to get wrong about a Score. The value is a position on your level number line. Three levels means 0 to 2. Four levels means 0 to 3. Ten levels means 0 to 9.

It matters as soon as you combine two Scores. A four-level scale and a three-level scale do not produce comparable numbers, so divide each score by its own top level number — `len(criteria) - 1` — before weighting them against each other.

Two more things the number will not tell you. Different distributions can produce the same score: a 1.0 can mean all the probability is on level 1, or half on level 0 and half on level 2. And the fractional part is not a measurement of the world — in the Safari example, 1.43 does not mean 43% of customers lack a workaround. Read `probabilities` and `confidence` alongside the score.

How the score moves with the input

These are the values the TypeSafe docs record for the three-level severity question above, across five bug reports:

State	`score`	`confidence`	L0	L1	L2
The export button is misaligned by a few pixels on the settings page.	0.0	1.0	1.0	0.0	0.0
The PDF export button does nothing when clicked. I can still export to CSV and convert it myself, but that takes ages.	1.0	1.0	0.0	1.0	0.0
Export to PDF fails with a spinner that never finishes. Some of our team say CSV export still works for them, others say it fails too.	1.11	0.84	0.0	0.89	0.11
The export button crashes the settings page in Safari. It works in Chrome, but a few of our customers only use Safari.	1.43	0.35	0.0	0.57	0.43
Nobody on our team can log in since this morning. We get a 500 error on every attempt.	2.0	1.0	0.0	0.0	1.0

TypeSafe docs, Score

Confidence 1.0 in the first two rows means all the probability landed on one level. That describes the model's answer, not a guarantee that the answer is correct.

Writing levels that work

Describe situations, not degrees. "Broken or degraded feature, but workaround exists" gives the model something to match the state against. "Moderately severe" does not.
Numbers in the descriptions do nothing. Each level is evaluated separately and the model never sees the numbering, so "worse than the previous level" is meaningless to it. The TypeSafe docs show what happens with numbers alone: on a report about a button misaligned by a few pixels, levels of `["0", "1", "2"]` with the instruction "Rate severity from 0 to 2, where 2 is worst" returned a score of 0.55 at confidence 0.33, splitting between levels 0 and 1. The same report with three descriptive levels returned 0.0 at confidence 1.0.
One dimension per question. A level that reads "punctual and smart and experienced" is measuring three things, and an input that is high on one and low on another cannot be placed anywhere.
Give rare extremes their own level. A sentiment scale that stops at "very angry" will score an abusive message near the top as well, and the score alone will not separate them. Add the level if you need to act on it differently.
Use as many levels as you can describe distinctly, up to 10. Three is fine. Do not add levels you cannot tell apart in words.

Adding examples to a level

When the model keeps landing between two levels on inputs you think are clear, give each level an object instead of a string, with a field for what the level covers and a field holding a few example situations. Use the same field names on every level.

The effect depends entirely on whether the examples look like your real inputs. The TypeSafe docs run the same Safari report three ways:

Level description writing	`score`	`confidence`
Pure strings, no examples	1.43	0.35
With the matching example "export fails in one browser but works in another"	1.03	0.96
With the unrelated example "search fails, but browsing categories still works"	1.43	0.35

TypeSafe docs, Score

The matching example concentrates almost all the probability on one level. The unrelated example changes nothing. And note what this does not prove: higher confidence does not establish that the answer is correct. Choose examples with known expected levels, then test the revised descriptions on separate inputs before keeping them.

Combining several Scores

A judgment that depends on several things is best split into one Score per thing, then combined in your own code with weights you control. Send them in one request, since questions are evaluated in parallel.

Python

def normalized(answers, question_id: str) -> float:
     normalized(answers, question_id: str) -> float:
    """Put a score on 0 to 1 by dividing by its top level number."""
    top_level = len(TRIAGE_QUESTIONS[question_id].criteria) - 
    top_level = len(TRIAGE_QUESTIONS[question_id].criteria) - 1
    
    return answers[question_id].score / top_level


severity = normalized(answers,  answers[question_id].score / top_level


severity = normalized(answers, "severity")
frustration = normalized(answers, )
frustration = normalized(answers, "frustration")
report_quality = normalized(answers, )
report_quality = normalized(answers, "report_quality")

priority = )

priority = 0.6 * severity +  * severity + 0.3 * frustration +  * frustration + 0.1 * report_quality * report_quality

On the TypeSafe docs' worked example the three normalized scores are 0.62, 0.64 and 1.0, giving a priority of `0.6 × 0.62 + 0.3 × 0.64 + 0.1 × 1.0 = 0.664`.

The weights live in your code, which is the point: when the ranking does not match what your team would decide, you change a number and run again instead of rewriting a prompt.

Score, Choice or Noul

	Noul	Choice	Score
Asks	Is this true?	Which of these options?	Which level?
`criteria` shape	Optional object with `true` and `false` descriptions	Map of option name to description	Ordered array of level descriptions
Returns	`noul`	`choice`, `probabilities`, `confidence`	`score`, `legend`, `probabilities`, `confidence`
Value range	0 to 1	One of your option keys	0 to the top level number
Has `confidence`	No	Yes	Yes
Size limit	—	Up to 255 options	At least 2 levels, up to 10
Your code does	An `if`	Branches per option	Compares to a threshold

flowchart TD
    A[What shape is the answer?] --> B{Yes or no?}
    B -- Yes --> C[Noul]
    B -- No --> D{Do the answers<br/>have an order?}
    D -- No --> E[Choice]
    D -- Yes --> F{Can you describe<br/>each level?}
    F -- Yes --> G[Score]
    F -- No --> E

If there is no in-between at all and the answer is one of a few discrete categories, use a Choice, or split the question into several Nouls. If the answer is yes or no, use a Noul — and remember that a Noul's 0-to-1 value is a probability, not a degree, which is exactly what a Score is for.

Related primitives

Jev Noul

Jev Choice

All Jev concepts

FAQ

What range does a Jev score use?

0 to the top level number, which is `len(criteria) - 1`. A three-level scale returns 0 to 2. It is not a 0-to-1 value.

How many levels can a Score have?

At least two, and the API accepts up to 10. Use as many as you can describe distinctly.

Why did I get a score between two levels?

Because the score is a probability-weighted mean of the level numbers. A 1.43 means probability is split between levels 1 and 2, leaning to 1.

What is `legend` for?

It maps each level number back to the description you sent, so your code and your logs can show the level in words without keeping a copy of the criteria.

Can I use a Score as a rating out of 10?

Only if you write ten distinct level descriptions. The model never sees numbers — it matches the state against each description — so a scale built from bare numbers performs worse than one built from situations.

Score or Choice?

Score when the answers have an order you can describe in steps. Choice when they are distinct categories with no order between them.