A new Python library from Nokia’s applied research arm gives open language models a narrow job. Rather than writing prose, they choose one answer from a fixed list and attach a confidence figure. The project is called AnyJev, and it needs no training.
Restricting a model’s next token to the option labels and reading the scores is the usual shortcut. Nokia’s team flags two problems with it. A different ordering of the same options can produce a different answer. Those probabilities also mislead if taken at face value.
The interface comes from Jev, the decision model TypeSafe AI shipped this month. A caller hands over a typed question and gets back a choice plus a probability fit for thresholding, read out of the model’s next-token distribution. No generation step is involved, no parsing, and no fine-tuning.
Two levels ship today. L0 costs one prefill per option, batched over a shared prefix, at about 0.25 seconds per decision on a single H100 with 20 labels. L1 handles 100 to 500 labels and layers temperature scaling on top, storing fitted values in a small JSON artifact. Reshaping confidence that way does not change the ranking.
The library installs from PyPI under Apache-2.0 and supports transformers and vLLM backends, with prefix caching suggested for serving.
