Jev Choice: The Classification Question Type

What comes back

{
  {
  "model": : "jev-1.13.0",
  ,
  "answers": {
    : {
    "department": {
      : {
      "type": : "choice",
      ,
      "choice": : "returns",
      ,
      "confidence": : 1.0,
      ,
      "probabilities": {
        : {
        "shipping": : 0.0,
        ,
        "returns": : 1.0,
        ,
        "billing": : 0.0
      }
    }
  },
  
      }
    }
  },
  "usage": {
    : {
    "input_tokens": : 328,
    ,
    "output_tokens": : 34
  }
}
  }
}

This ticket is an easy one: all of the probability sits on `returns` and confidence is 1.0. A ticket that mentioned both a wrong size and a missing refund would split probability between `returns` and `billing`, and the confidence would drop.

Reading confidence

The ticket genuinely belongs to two teams, and the 0.42 confidence on `department` reflects that. The resolution question is weaker still at 0.20: the double charge suggests money back, the wrong size suggests a swap, and the customer never says which they want. That is a case for asking rather than guessing.

A low confidence value is a routing decision, not a verdict on the answer. Note too that confidence 1.0 describes the model's distribution, not a guarantee that the answer is correct.

Acting on the distribution, not just the winner

Because `probabilities` covers every option, your code can do more than read the winner:

department = answers[department = answers["department"]

]

if department.confidence <  department.confidence < 0.3:
    send_to_manual_triage(ticket)
    :
    send_to_manual_triage(ticket)
    return

assign(ticket, team=department.choice)



assign(ticket, team=department.choice)

# A second team with a real share of the probability gets a copy

for team, probability  team, probability in department.probabilities.items():
     department.probabilities.items():
    if team != department.choice  team != department.choice and probability >  probability > 0.25:
        notify(ticket, team=team):
        notify(ticket, team=team)

For the ticket above this assigns it to returns, and copies billing in because its 0.35 share clears the 0.25 bar.

Writing the option set

• Give the full list. A Choice question accepts up to 255 options, and each option costs only a few tokens, so send the whole list of teams, categories or products rather than a shortlist.
• Add an escape hatch. When the list might not cover every input, include an `other` or `none of the above` option so the model can say that none of the others fit. A Choice always names a winner, so without one, an input that fits nothing still gets assigned somewhere.
• Descriptions can be null. When the option names speak for themselves, `null` is a valid description. The TypeSafe docs use `{"calm": null, "frustrated": null, "angry": null}` for a tone question.
• Use objects when two options keep getting confused. Instead of one line per option, give each a structured description with a field for what it covers, a field for what belongs to the neighbouring option instead, and a few examples:

"criteria": {
  : {
  "return_policy": {
    : {
    "what": : "Whether and how an item can be returned",
    ,
    "not_for": : "Progress of a return already sent",
    ,
    "examples": [
      : [
      "Can I return shoes I've worn once?",
      ,
      "How long do I have to return an order?"
    ]
  },
  
    ]
  },
  "return_status": {
    : {
    "what": : "Progress of a return already sent",
    ,
    "not_for": : "Whether and how an item can be returned",
    ,
    "examples": [
      : [
      "Has my return arrived yet?",
      ,
      "When will my refund be paid?"
    ]
  }
}
    ]
  }
}

The field names `what`, `not_for` and `examples` are not part of the API and none of them are reserved. You pick them, the same way you pick option names, and the model sees the names along with the values. On the message "I sent the shoes back a week ago. When do I get my money?", this question returns `return_status` at confidence 1.0.

Deep taxonomies chain. For a large hierarchy, ask a Choice per level and let each answer decide which options the next question offers, rather than flattening everything into one question.

Choice, Noul or Score

flowchart TD
    A[What shape is the answer?] --> B{Yes or no?}
    B -- Yes --> C[Noul]
    B -- No --> D{Do the answers<br/>have an order?}
    D -- No --> E[Choice]
    D -- Yes --> F{Can you describe<br/>each level?}
    F -- Yes --> G[Score]
    F -- No --> E