A question is a JSON object. You send one image and up to 64 of them; you get back one Answer per question, each a probability distribution over things you named yourself. Nothing is generated, so nothing has to be parsed: you compare a number to a threshold. There are four types. noul is “no-or-yes level”: the one number between no and yes.

​
How the examples here were run

There is no network for model weights in the environment these docs were written in, so every example below runs a deterministic toy encoder in place of SigLIP: a 16-dim hash of the input bytes. It produces real embeddings, so every shape, every field and every error message on this page is real output from a real run. The numbers mean nothing. The toy has no idea what “indoor” looks like. Swap Toy() for jevimage.load() and the shapes are identical; only the numbers become meaningful.
Every example loads it at temperature=0.5. A real encoder’s cosines live in a narrow band and the default temperature is tuned for that; the toy’s cosines are spread over the whole range, so at the default every answer saturates to 1.000 / 0.000 and the printed distributions stop being readable. See Temperature.
ds/ is the 12-image fixture these pages share: flat 64x64 squares, ds/red/0..5.png and ds/blue/0..5.png. Write it next to toy.py with examples/make_fixtures.py before running anything below. The toy hashes the image’s raw pixels, so the numbers on this page are a function of those exact squares - point img at a photo of your own and every shape stays the same while every number moves.

​
The shape every question takes

instructions on a choice or a score is silently dead weight; it is read but never used, so a question phrased as instructions plus bare labels will not do what it looks like it does:
Everything the model sees is in criteria and prompt. Put the discriminating words there.

​
Questions arrive as a dict or a list

A list gets q1, q2, … unless an entry carries its own id. Two entries with the same id is an error, not a silent overwrite.

​
Answers

An Answer is a dict subclass with attribute access, so a.choice and a["choice"] are the same thing and json.dumps(a) needs no encoder. Printing one gives a short repr, not the full record:
Reading a field that this answer type does not have raises AttributeError naming what it does have. a.score on a choice answer says 'choice' answers have no 'score'; this one has: type, choice, probabilities, confidence.

​
choice

Pick exactly one of 2..255 named options. Use it whenever the options are mutually exclusive and you want the winner plus how sure the model was. Request
criteria is an object mapping the label you want back to the caption the model sees. Your downstream code branches on "indoor", and you can rewrite the caption to something wordier without touching that code. A bare list is shorthand for “label == caption”:
Two identical captions in that list is an error rather than a distribution over fewer options than you listed. Answer Choosing a threshold. Branch on confidence, not on probabilities. The maximum probability is not comparable between a 2-option question and a 30-option one; confidence is rescaled so it is. Start at confidence >= 0.5 for “act on it” and route the rest to a human, then move it once you have looked at a few hundred of your own images. Also check choice against the runner-up when two options are near- synonyms. The model splitting 0.45/0.44 between “bus” and “coach” is a caption problem, not an uncertainty problem.

​
score

Place the image on an ordered rubric of 2..10 levels. Use it when the answer is a magnitude rather than a category: empty → packed, pristine → destroyed, clean → filthy. Request
criteria is an array, low end first. Order is meaningful: the score is the expected index, so putting the levels out of order gives a number that averages across a rubric that has no direction.
Answer Choosing a threshold. score is a mean, so it hides its own shape: a bimodal distribution over “empty” and “packed” lands mid-rubric and looks like “moderately busy”. Threshold on score and require confidence above a floor, or read probabilities directly and sum the top levels (p["2"] + p["3"] > 0.6) when you need “at least busy” rather than a point estimate. Note the scale ends at len(criteria)-1, not at 1 or 100, so three levels means 0..2.

​
noul

One probability, no distribution to interpret. Use it for the yes/no gates that make up most real pipelines: is this blurry, is there a person in frame, is this a screenshot. noul has two forms, and they are not interchangeable.

​
The contrast pair (works on every encoder)

Two captions that mean opposite things is a real two-way decision, scored exactly like a 2-option choice, and noul is the probability of the true side. Needs no calibration, so it works on SigLIP, CLIP, MetaCLIP, DFN5B and every ensemble.
Write the false side as a real alternative, not as a negation. “a sharp, in-focus photo” is a caption an image can match; “not a blurry photo” is not. Dual encoders have no negation operator, and that caption embeds close to the one it was meant to oppose. A pair with only one side filled in is an error, not a fallback to the sigmoid form.

​
The single-caption sigmoid (SigLIP only)

One caption, and the probability comes from sigmoid(cosine * logit_scale + logit_bias), the encoder’s own trained readout. This needs an encoder publishing both logit_scale and logit_bias. SigLIP and SigLIP2 have both, because they were trained with a pairwise sigmoid loss that had to output a calibrated probability per caption. CLIP-family models publish a scale and no bias. An Ensemble publishes the mean of its members’ scales when every member has one (the concatenated cosine is the mean of the members’ cosines, so the mean scale is what reads it), but never a bias: a bias was fitted against one member’s own negatives and does not average across a concatenation. So an ensemble still refuses the single-caption form, whatever its members are. Asked of an encoder that cannot do it, you get a refusal rather than a plausible-looking number:
With a calibrated encoder the same question answers:
1.000000 there is the toy being a hash function: its random cosines are far outside the range SigLIP’s scale and bias were fitted for, so the sigmoid saturates. A real SigLIP does not saturate, but it does not give you a usable probability either. Measured on siglip2-base-224, against a photo of a man using a laptop at an outdoor picnic table:
The ordering is right, and the range is not. SigLIP’s bias is -16.75, fitted for retrieval against millions of negatives, so everything lands near zero: a threshold of 0.5 reads every caption as false, and two captions that are both true (“a person using a laptop”, “a photo taken outdoors”) differ by a factor of 4000. Use the sigmoid form for ranking captions against one image, and the {true, false} pair for anything you threshold. basis tells the two forms apart in the answer, so a consumer can log which readout produced a number. Temperature does not apply to the sigmoid form. The scale is the encoder’s, not yours. Answer (both forms) There is no confidence on a noul: with one number, the distance from 0.5 is the confidence. Choosing a threshold. The pair form is a contrast, not an absolute: 0.5 means the two captions fit equally well, which is not the same as “50% chance this is blurry”. Sweep the threshold on 50-100 of your own labelled images and pick the point that gives you the precision you need. For a gate that drops images, err high (0.8+); for a gate that flags for review, err low. The criteria pair is also the form to prefer if you care about stability across encoder swaps, since the sigmoid form silently stops being available the moment you move to an ensemble.
Run a linear head you trained on your own labelled images. Same answer shape as choice, plus provenance. Request
head names a head in the head directory. name works as a spelling of the same field, and so does instructions if neither is present, a leftover convenience rather than the shape to write. There is no criteria, no prompt: the classes and their meanings are in the head file, fixed when you trained it. See training.md.
Answer accuracy rides along in every answer on purpose: a 0.99-confidence answer from a head whose held-out accuracy is 0.62 is a confident answer from a bad classifier, and you can only see that if both numbers are in front of you. A head records the encoder that fitted it and refuses to run under a different one:
That check is on the encoder name, not the width, because two encoders of the same dimensionality produce embeddings in unrelated spaces and the answer would be silently wrong rather than loudly broken. Choosing a threshold. Use confidence, as with choice, but calibrate it against accuracy and per_class from compare() rather than against intuition. A head trained on 12 examples is a suggestion; a head with 200 per class and 0.95 held-out is a decision.

​
Captions: what the model reads

​
The template

Each caption is substituted into prompt before it is embedded. The default is "{}" (the caption, unchanged). The template must contain {}; without a placeholder every option would get the same string, so the distribution would be uniform over identical captions and choice would be whichever you listed first. That is refused:
Write captions, not questions. Dual encoders are trained on image/caption pairs scraped from the web. Their text tower has seen “a photo of a golden retriever on a lawn” tens of millions of times and “Is there a dog in this image?” approximately never. A question embeds near other questions, which is not where your image is. Phrase every criterion as the caption someone would have written under the photo:
Keep the shared words in prompt and the discriminating words in criteria: the shared prefix contributes the same component to every option’s embedding and cancels in the softmax, while the differing part is what the answer is made of.

​
Flattening: a criterion can be a string, a list or an object

Any criterion value is flattened to exactly one caption before embedding: Nesting composes: lists inside objects inside lists all collapse by the same two rules. This is visible in the planner (internal, shown here only to make the flattening concrete; jevimage.core.plan is not part of the public API):
The object form injects your keys (front:, back:) into the caption, which is text the encoder scores, useful when the keys are meaningful words and noise when they are a/b/c. The null fallback means a criterion you forgot to fill in does not error; it quietly becomes its own label. If your labels are already good captions, the bare-list form says so more clearly.

​
Confidence and thresholds

confidence is not the maximum probability. It is that maximum rescaled so uniform is 0 and certain is 1 at any number of options:
Raw max-probability is not comparable across questions. 0.5 is lukewarm over two options and emphatic over fifty. The rescaling removes K, so one threshold works for every question you ask:
The same 0.70 maximum is 0.400 over two options and 0.694 over fifty, which is the correct ordering: being 70% sure among fifty candidates is much stronger evidence than being 70% sure between two. jevimage.confidence(p) is public and takes any sequence of probabilities, so you can apply the same scale to distributions you computed yourself. A single-element sequence returns 1.0 by definition. What confidence is not: a probability of being correct. It measures how peaked the distribution is, and a badly-worded caption set can be sharply peaked on the wrong option. Calibrate a threshold against your own labelled sample; do not port one from another project’s question set.

​
Asking many questions at once

N questions about one image cost barely more than one. The image is encoded once; each question is then a matmul against that vector. Captions are cached per string across calls, so a repeated question set costs no text encoding at all after the first call. Counting the encoder’s forward passes directly (subclassing the toy to count):
The four-question call did one image pass and one text pass for the five new captions (2 + 2 + 3 = 7 total, minus the 2 already cached). Asking the identical set again encoded no text at all. The head question contributes zero captions; it is pure matmul. The ceiling is 64 questions per call. Above that, split the call and reuse the embedding:
ask_embedding exists on Jev and not on Remote, deliberately: over HTTP there is nothing to skip, so a remote caller sends the image and asks everything in one /v1/ask. The text cache holds 50,000 captions, evicting oldest-first, a bound that matters only to a long-lived server whose callers generate captions dynamically.

​
Temperature

Temperature divides the logits before the softmax. Lower is sharper, higher is flatter. It never changes which option wins, only how confident the answer looks.
(None here equals 0.5 because that is what this Jev was loaded with.) You almost certainly should not touch it. The default is the encoder’s own 1 / logit_scale (the temperature the model was trained to be read at), falling back to 0.01 for encoders that publish no scale, which is the CLIP-family convention. A raw cosine lives in roughly [0, 0.3] and softmaxes to nearly uniform, which is why some scaling is needed at all and why the model’s own is the right one. Turning temperature down to make answers “more confident” does not make them more correct; it makes your confidence threshold meaningless and hides exactly the uncertainty you built a probability-first API to see. The honest reasons to set it:
  • you are reproducing another system’s calibration,
  • you swapped in a custom encoder with no logit_scale and 0.01 is visibly wrong for its cosine range (as with the toy on this page),
  • you are deliberately flattening a distribution to feed an ensemble vote downstream.
Where it applies: That split surprises people, so here it is measured on the colour head above:
Must be a finite number greater than 0. Zero, negative, infinite, True and strings are all refused: a negative temperature inverts every softmax so the least similar option wins while still reporting high confidence, and an infinite one returns a perfectly well-formed answer carrying no signal. Both are worse than an error, because nothing downstream can tell.

​
Over HTTP

The wire shape is the same object. POST /v1/ask:
Those scene numbers are identical to the local run earlier in this page, against the same encoder: connect() and load() answer the same question the same way. Send images (a list, up to 8) instead of image and answers becomes a list, one entry per image, and the questions are asked of every image. One field or the other: sending both is a 422, send 'image' or 'images', not both: 'image' returns one answer object, 'images' returns a list, because the choice picks the response shape as well as the pixels. Every error in the table below arrives as HTTP 422 with the same sentence in detail, except the two the engine never sees: questions that is neither an object nor a list, and a temperature that is not a number, are both typed fields on the request model, so FastAPI rejects them with its own field report instead. The unreadable head file is the third exception: it is a 404, with the heads directory stripped out of the sentence. (temperature: 0 does reach the engine, but as a float, so the sentence reads got 0.0 over the wire and got 0 in Python.)

​
From the CLI

jev ask builds these questions from flags; each flag repeats, and all of them ride the one encode.
(The choice saturates because this server runs the toy encoder at its default temperature, rather than the 0.5 the Python blocks above load it at; see above.) --noul takes two captions for the pair form or one for the sigmoid form. Repeated flags get ids choice, choice2, …; --head NAME uses the head’s name as its id. The CLI covers the common shapes only; prompt templates, nested criteria and per-call temperature are Python and HTTP.

​
Errors a malformed question can raise

All of these are ValueError unless marked, and surface as HTTP 422 with the text below (see the two exceptions above). The one KeyError is a 404 instead, and the server strips the heads directory out of its text on the way to the client. Each message is verbatim. None of them is free: ask() encodes the image first and plans the questions afterwards, so one bad question in a batch of 64 fails the whole call after paying for the encode rather than instead of it. Most are raised by the planner, which is before the captions are encoded. Seven are not: the single-caption noul refusal, the unreadable head file, the four head checks and the nan guard are raised while the answer is being read, after every caption in the request has been encoded too. The embedding-norm check sits in between, after the image and before the text. The last two only reach you through a custom encoder: jevimage.encoders.unit() is what the built-in encoders wrap their output in, and a non-finite answer is refused rather than emitted because json.dumps turns nan into invalid JSON that fails three layers downstream.
See also: encoders.md for what to ask these questions of, and training.md for where head questions come from.
← Quickstart · Docs index · Training a head →