Patterns
A few shapes that come up again and again when decisions are made from probabilities. Each one is a handful of lines around a single API call.
Routing by confidence
Let the model handle the cases it is sure about and send the rest to a person. One choice question picks the queue; the confidence decides whether the pick is used.
from typesafe_sdk import Choice, TypeSafeClient # Reads TYPESAFE_BASE_URL (https://api.vakyn.com) and TYPESAFE_API_KEY from the environment.client = TypeSafeClient()QUEUES = { "billing": "Charges, invoices and refunds", "shipping": "Delivery, tracking and damaged parcels", "account": "Sign-in, password and profile",} def route(message: str) -> str: result = client.system_one( state=message, questions={"queue": Choice(instructions="Which queue should handle this?", criteria=QUEUES)}, ) answer = result.choices["queue"] if answer.confidence >= 0.8: return answer.choice # sure enough: route automatically return "triage" # unsure: a person decides print(route("I changed my email address and now the password reset link never arrives."))- Measure how many messages fall under the threshold. If it is too many, the options are probably unclear or incomplete; fix the criteria before lowering the bar.
- Log the distribution, not only the pick. When a person corrects a route, you can see whether the model had the right answer in second place.
Example: moderating product reviews
Routing by confidence with several questions at once. Each review gets a noul (fake?), a choice (topic) and a score (how negative). The code acts only when every answer is sure enough; anything else waits for a moderator. The How VAKYN works page walks through it.
Questionsfakenoultopicchoicenegativescore
Clear case
Acts automaticallyStopped working after five weeks. 1 star. I bought the Tallow & Pine CafeDuo 2 in March and used it every morning for two coffees. In week five the steam wand stopped heating, then the pump started rattling and the machine shut itself off mid-shot. I descaled it twice with the brand's own tablets and followed the reset steps in the manual exactly. Nothing helped. For a machine at this price I expected years, not weeks. Do not buy. Marta K., verified purchase
Question Answer Confidence fake yes 0.04 0.96 topic quality 0.97 negative 3.94 of 4 0.95 Every answer is at least 0.70 sure (lowest: negative 0.95), so your code acts on it.
Borderline case
Goes to a personDecent kettle for what I paid. It took ages to show up and the box was a bit crushed, but it boils fine. Would maybe buy again if it goes on sale. J.
Question Answer Confidence fake yes 0.03 0.97 topic quality 0.23 negative 1.93 of 4 0.94 topic is only 0.23 sure, under 0.70, so a person decides.
Clear case
curl -s "https://api.vakyn.com/v1/systemone" \ -H "Authorization: Bearer $TYPESAFE_API_KEY" \ -H "Content-Type: application/json" \ -o answer.json \ -d @- <<'JSON'{ "model": "jev-latest", "state": "Stopped working after five weeks. 1 star.\n\nI bought the Tallow & Pine CafeDuo 2 in March and used it every morning for two coffees. In week five the steam wand stopped heating, then the pump started rattling and the machine shut itself off mid-shot. I descaled it twice with the brand's own tablets and followed the reset steps in the manual exactly. Nothing helped. For a machine at this price I expected years, not weeks. Do not buy.\n\nMarta K., verified purchase", "questions": { "fake": { "type": "noul", "instructions": "Is this review likely fake: paid for, written by the seller, or not based on using the product?" }, "topic": { "type": "choice", "instructions": "What is the review mainly about?", "criteria": { "quality": "How the product works, feels or holds up", "delivery": "Shipping, packaging and arrival", "price": "Price and value for money", "service": "Support, returns and refunds", "other": null } }, "negative": { "type": "score", "instructions": "How negative is the review?", "criteria": [ "Positive", "Mostly positive", "Mixed", "Mostly negative", "Very negative" ] } }}JSON # Act when every answer is at least 0.70 sure; otherwise a person decides.jq -r '.answers as $a | {fake: ([$a.fake.noul, 1 - $a.fake.noul] | max), topic: $a.topic.confidence, negative: $a.negative.confidence} | to_entries | min_by(.value) as $weakest | if $weakest.value >= 0.70 then "act: \(if $a.fake.noul >= 0.5 then "remove" else "publish" end), topic \($a.topic.choice), negative \($a.negative.score) of 4" else "human: \($weakest.key) is only \($weakest.value * 100 | round / 100) sure" end' answer.jsonThe borderline case runs the same code with its own state. topic is only 0.23 sure, under 0.70, so it prints a human line and a person decides. Change the threshold to match what a wrong answer costs you.
Example: approving expense reports
The same rule on structured data. The report goes in as JSON with the policy next to it; a choice says approve, flag or reject, a noul says whether any line breaks the policy, and a score rates the risk of paying it as filed. Reports the model is sure about are paid or returned by code; the rest go to finance. The How VAKYN works page walks through it.
Questionsdecisionchoiceover_policynoulriskscore
Clear case
Acts automatically{ "policy": { "rail": "Second class, booked through the travel desk", "hotel": "Up to EUR 180 a night", "meals": "Up to EUR 60 a day while travelling, no alcohol", "receipts": "A receipt for every line over EUR 25" }, "report": { "employee": "Ines Varga, field engineer", "trip": "Customer site visit in Lyon, 6 October 2026", "lines": [ { "date": "2026-10-06", "item": "Flight, Brussels to Lyon and back, business class, booked privately", "amount_eur": 1240, "receipt": false }, { "date": "2026-10-06", "item": "Hotel, three nights for a one-day visit", "amount_eur": 1260, "receipt": false }, { "date": "2026-10-06", "item": "Dinner, alone, two bottles of wine", "amount_eur": 310, "receipt": false } ], "total_eur": 2810 } }
Question Answer Confidence decision reject 0.82 over_policy yes 1.00 1.00 risk 2.97 of 3 0.97 Every answer is at least 0.70 sure (lowest: decision 0.82), so your code acts on it.
Borderline case
Goes to a person{ "policy": { "rail": "Second class, booked through the travel desk", "hotel": "Up to EUR 180 a night", "meals": "Up to EUR 60 a day while travelling, no alcohol", "receipts": "A receipt for every line over EUR 25" }, "report": { "employee": "Pieter Claes, account manager", "trip": "Trade fair in Milan, 13 to 14 October 2026", "lines": [ { "date": "2026-10-13", "item": "Rail, Brussels to Milan, second class, booked by the travel desk", "amount_eur": 164, "receipt": true }, { "date": "2026-10-13", "item": "Hotel, one night, city tax EUR 9 included", "amount_eur": 189, "receipt": true }, { "date": "2026-10-13", "item": "Dinner with two people from a prospect", "amount_eur": 118, "receipt": true, "note": "Talked about a pilot" }, { "date": "2026-10-14", "item": "Taxi to the fair", "amount_eur": 27, "receipt": false, "note": "Driver's card reader was broken" } ], "total_eur": 498 } }
Question Answer Confidence decision reject 0.65 over_policy yes 0.94 0.94 risk 2.52 of 3 0.52 risk is only 0.52 sure, under 0.70, so a person decides.
Clear case
curl -s "https://api.vakyn.com/v1/systemone" \ -H "Authorization: Bearer $TYPESAFE_API_KEY" \ -H "Content-Type: application/json" \ -o answer.json \ -d @- <<'JSON'{ "model": "jev-latest", "state": { "policy": { "rail": "Second class, booked through the travel desk", "hotel": "Up to EUR 180 a night", "meals": "Up to EUR 60 a day while travelling, no alcohol", "receipts": "A receipt for every line over EUR 25" }, "report": { "employee": "Ines Varga, field engineer", "trip": "Customer site visit in Lyon, 6 October 2026", "lines": [ { "date": "2026-10-06", "item": "Flight, Brussels to Lyon and back, business class, booked privately", "amount_eur": 1240, "receipt": false }, { "date": "2026-10-06", "item": "Hotel, three nights for a one-day visit", "amount_eur": 1260, "receipt": false }, { "date": "2026-10-06", "item": "Dinner, alone, two bottles of wine", "amount_eur": 310, "receipt": false } ], "total_eur": 2810 } }, "questions": { "decision": { "type": "choice", "instructions": "What should finance do with this expense report?", "criteria": { "approve": "Pay it as filed", "flag": "Hold it until a person checks it", "reject": "Send it back to the employee to fix" } }, "over_policy": { "type": "noul", "instructions": "Does any line in the report break the expense policy?" }, "risk": { "type": "score", "instructions": "How risky is it to pay this report as filed?", "criteria": [ "No risk", "Low risk", "Some risk", "High risk" ] } }}JSON # Act when every answer is at least 0.70 sure; otherwise a person decides.jq -r '.answers as $a | $a | map_values(if .type == "noul" then [.noul, 1 - .noul] | max else .confidence end) | to_entries | min_by(.value) as $weakest | if $weakest.value >= 0.70 then "act: \($a.decision.choice), over policy \(if $a.over_policy.noul >= 0.5 then "yes" else "no" end), risk \($a.risk.score) of 3" else "human: \($weakest.key) is only \($weakest.value * 100 | round / 100) sure" end' answer.jsonThe borderline case runs the same code with its own state. risk is only 0.52 sure, under 0.70, so it prints a human line and a person decides. Change the threshold to match what a wrong answer costs you.
Guardrails for agents
Put VAKYN between an agent and the APIs it calls. Before a call goes out, send the call, the agent's job and the context as the state, and ask whether to allow it, hold it for a person or block it, with the facts that decide it as their own questions: can it be undone, how much damage could a mistake do, is it the agent's job. The call runs only when every answer is sure; the rest wait for a person. The How VAKYN works page walks through it.
Questionsactionchoiceirreversiblenouldamagescorewithin_jobnoul
Clear case
Acts automatically{ "agent": { "name": "Returns assistant at Orla Cycles", "job": "Handles returns. May refund returned items to the original payment method, up to EUR 1,000 per order, once the warehouse has checked the return in." }, "call": { "api": "POST /payments/refunds", "body": { "order": "OC-58213", "amount": 900, "currency": "EUR", "to": "original payment method", "reason": "Returned, unused" } }, "context": { "order": "OC-58213, one e-bike battery, paid EUR 900.00 by card on 2026-09-28", "return": "Checked in by the warehouse on 2026-10-09: sealed, unused, serial number matches the order", "earlier_refunds_on_order": 0, "customer": "Customer since 2022, 11 orders, no earlier refunds, same card on file since 2022" } }
Question Answer Confidence action allow 1.00 irreversible yes 0.14 0.86 damage 0.21 of 3 0.79 within_job yes 0.99 0.99 Every answer is at least 0.70 sure (lowest: damage 0.79), so your code acts on it.
Borderline case
Goes to a person{ "agent": { "name": "Returns assistant at Orla Cycles", "job": "Handles returns. May refund returned items to the original payment method, up to EUR 1,000 per order, once the warehouse has checked the return in." }, "call": { "api": "POST /payments/refunds", "body": { "order": "OC-60477", "amount": 900, "currency": "EUR", "to": "original payment method", "reason": "Parcel never arrived" } }, "context": { "order": "OC-60477, one e-bike battery, paid EUR 900.00 by card on 2026-10-02", "return": "None: the customer says the parcel never arrived", "carrier": "Tracking says delivered to a parcel locker on 2026-10-05", "earlier_refunds_on_order": 0, "customer": "First order, account opened 2026-10-01" } }
Question Answer Confidence action block 0.93 irreversible yes 0.29 0.71 damage 0.77 of 3 0.23 within_job yes 0.42 0.58 damage is only 0.23 sure, under 0.70, so a person decides.
Clear case
curl -s "https://api.vakyn.com/v1/systemone" \ -H "Authorization: Bearer $TYPESAFE_API_KEY" \ -H "Content-Type: application/json" \ -o answer.json \ -d @- <<'JSON'{ "model": "jev-latest", "state": { "agent": { "name": "Returns assistant at Orla Cycles", "job": "Handles returns. May refund returned items to the original payment method, up to EUR 1,000 per order, once the warehouse has checked the return in." }, "call": { "api": "POST /payments/refunds", "body": { "order": "OC-58213", "amount": 900, "currency": "EUR", "to": "original payment method", "reason": "Returned, unused" } }, "context": { "order": "OC-58213, one e-bike battery, paid EUR 900.00 by card on 2026-09-28", "return": "Checked in by the warehouse on 2026-10-09: sealed, unused, serial number matches the order", "earlier_refunds_on_order": 0, "customer": "Customer since 2022, 11 orders, no earlier refunds, same card on file since 2022" } }, "questions": { "action": { "type": "choice", "instructions": "What should the guardrail do with this API call?", "criteria": { "allow": "Let the agent make the call now", "ask_human": "Hold the call until a person approves it", "block": "Stop the call and tell the agent why" } }, "irreversible": { "type": "noul", "instructions": "Would this call be hard or impossible to undo once made?" }, "damage": { "type": "score", "instructions": "If this call is a mistake, how hard is the damage to put right?", "criteria": [ "Easy", "Takes some work", "Hard", "Impossible" ] }, "within_job": { "type": "noul", "instructions": "Is this call part of the agent's job as described?" } }}JSON # Act when every answer is at least 0.70 sure; otherwise a person approves the call.jq -r '.answers as $a | $a | map_values(if .type == "noul" then [.noul, 1 - .noul] | max else .confidence end) | to_entries | min_by(.value) as $weakest | if $weakest.value >= 0.70 then "act: \($a.action.choice) (damage \($a.damage.score) of 3)" else "human: \($weakest.key) is only \($weakest.value * 100 | round / 100) sure" end' answer.jsonThe borderline case runs the same code with its own state. damage is only 0.23 sure, under 0.70, so it prints a human line and a person decides. Change the threshold to match what a wrong answer costs you.
Fan-out: many questions, one reading
A request reads the state once and answers every question from that reading, so a checklist of ten yes/no checks costs about the same as one. Build the questions from data and send them together.
from typesafe_sdk import Noul, TypeSafeClient # Reads TYPESAFE_BASE_URL (https://api.vakyn.com) and TYPESAFE_API_KEY from the environment.client = TypeSafeClient()CHECKS = { "has_total": "Does the invoice state a total amount?", "has_due_date": "Does the invoice state a payment due date?", "has_vat_number": "Does the invoice show the supplier's VAT number?", "mentions_late_fee": "Does the invoice mention a fee for late payment?",} invoice = { "supplier": "Northwind Paper Co.", "lines": [{"item": "A4 paper, 40 boxes", "amount": "1,180.00 EUR"}], "text": "Total due: 1,180.00 EUR. Payment within 30 days of the invoice date.",} # One request, one reading of the invoice, every check answered.result = client.system_one( state=invoice, questions={name: Noul(instructions=q) for name, q in CHECKS.items()},)missing = [name for name, a in result.nouls.items() if a.noul < 0.5]print("missing:", missing)The same idea works across types: a noul for "is this a complaint", a choice for the product and a score for severity, all in one call. Keep each question about one thing; the request may hold as many as fit in the token limits.
Example: eight job applications in one request
Eight applications for one role in one state, and ten questions built from the data: a priority score for each application, who to interview first and whether any application shows a red flag. The How VAKYN works page walks through it.
Questionspriority_1scorepriority_2scorepriority_3scorepriority_4scorepriority_5scorepriority_6scorepriority_7scorepriority_8scorebestchoicered_flagsnoul
Clear case
Acts automatically{ "role": { "title": "Night shift lead, distribution center in Rotterdam", "must_have": [ "Two years or more leading a warehouse team", "A valid forklift certificate", "Can work nights, Sunday to Thursday" ], "nice_to_have": [ "Dutch and English", "Has used a warehouse management system" ] }, "applications": [ { "application": 1, "name": "Joost Verhagen", "summary": "Four years as night shift lead at a grocery distribution center, team of 14. Forklift certificate valid to 2028. Wants to stay on nights. Dutch and English. Daily work in a warehouse management system." }, { "application": 2, "name": "Mila Petrova", "summary": "Recent graduate in logistics, no work experience in a warehouse yet. No forklift certificate. Prefers day shifts." }, { "application": 3, "name": "Sam O'Brien", "summary": "Barista for three years. No warehouse or forklift experience. Available weekends only." }, { "application": 4, "name": "Fatima Zahra El Idrissi", "summary": "Office manager for six years. No warehouse experience, no forklift certificate. Looking for a day job close to home." }, { "application": 5, "name": "Kees de Wit", "summary": "Retired in 2025 after thirty years as a truck driver. No team lead experience. Wants two days a week." }, { "application": 6, "name": "Lucas Moreau", "summary": "Software developer looking for a career change into tech sales. Never worked in a warehouse." }, { "application": 7, "name": "Hanna Kowalski", "summary": "Student, wants a summer job. No experience and no forklift certificate. Available in July and August only." }, { "application": 8, "name": "Ravi Menon", "summary": "Chef for ten years in restaurants. No warehouse experience. Cannot work nights." } ] }
Question Answer Confidence priority_1 1.98 of 2 0.97 priority_2 0.00 of 2 1.00 priority_3 0.00 of 2 1.00 priority_4 0.00 of 2 1.00 priority_5 0.01 of 2 0.99 priority_6 0.00 of 2 1.00 priority_7 0.00 of 2 1.00 priority_8 0.00 of 2 1.00 best application_1 0.96 red_flags yes 0.09 0.91 Every answer is at least 0.70 sure (lowest: red_flags 0.91), so your code acts on it.
Borderline case
Goes to a person{ "role": { "title": "Night shift lead, distribution center in Rotterdam", "must_have": [ "Two years or more leading a warehouse team", "A valid forklift certificate", "Can work nights, Sunday to Thursday" ], "nice_to_have": [ "Dutch and English", "Has used a warehouse management system" ] }, "applications": [ { "application": 1, "name": "Anouk Jansen", "summary": "Three years as shift lead in a parcel hub, team of 9, mostly evenings. Forklift certificate valid to 2027. Open to nights. Dutch and English." }, { "application": 2, "name": "Daniel Mensah", "summary": "Five years as warehouse team lead on day shifts, team of 20. Forklift certificate expired in 2025, says he will renew it. Can work nights. English only." }, { "application": 3, "name": "Eva Lindgren", "summary": "Two years leading a night team of 6 in a cold store. Forklift certificate valid. Uses a warehouse management system daily. English and some Dutch." }, { "application": 4, "name": "Tom Bakker", "summary": "Forklift driver for seven years, no team lead role. Certificate valid. Night shifts now. Dutch." }, { "application": 5, "name": "Yusuf Demir", "summary": "Says he led a team of 30 for eight years; his dates show four years in total at two companies. Forklift certificate attached." }, { "application": 6, "name": "Sofia Russo", "summary": "Two years as assistant shift lead at a fashion warehouse. Forklift certificate valid. Prefers nights. English and Italian." }, { "application": 7, "name": "Bram Visser", "summary": "Nine years as a warehouse supervisor, then three years out of work for family reasons. Certificate valid to 2026. Can work nights." }, { "application": 8, "name": "Lena Hofmann", "summary": "Store manager in retail for five years, team of 12. No forklift certificate. Can work nights." } ] }
Question Answer Confidence priority_1 1.85 of 2 0.77 priority_2 0.29 of 2 0.56 priority_3 1.81 of 2 0.71 priority_4 0.05 of 2 0.93 priority_5 0.77 of 2 0.27 priority_6 0.12 of 2 0.82 priority_7 1.00 of 2 0.49 priority_8 0.01 of 2 0.99 best application_3 0.49 red_flags yes 0.93 0.93 priority_5 is only 0.27 sure, under 0.70, so a person decides.
Clear case
curl -s "https://api.vakyn.com/v1/systemone" \ -H "Authorization: Bearer $TYPESAFE_API_KEY" \ -H "Content-Type: application/json" \ -o answer.json \ -d @- <<'JSON'{ "model": "jev-latest", "state": { "role": { "title": "Night shift lead, distribution center in Rotterdam", "must_have": [ "Two years or more leading a warehouse team", "A valid forklift certificate", "Can work nights, Sunday to Thursday" ], "nice_to_have": [ "Dutch and English", "Has used a warehouse management system" ] }, "applications": [ { "application": 1, "name": "Joost Verhagen", "summary": "Four years as night shift lead at a grocery distribution center, team of 14. Forklift certificate valid to 2028. Wants to stay on nights. Dutch and English. Daily work in a warehouse management system." }, { "application": 2, "name": "Mila Petrova", "summary": "Recent graduate in logistics, no work experience in a warehouse yet. No forklift certificate. Prefers day shifts." }, { "application": 3, "name": "Sam O'Brien", "summary": "Barista for three years. No warehouse or forklift experience. Available weekends only." }, { "application": 4, "name": "Fatima Zahra El Idrissi", "summary": "Office manager for six years. No warehouse experience, no forklift certificate. Looking for a day job close to home." }, { "application": 5, "name": "Kees de Wit", "summary": "Retired in 2025 after thirty years as a truck driver. No team lead experience. Wants two days a week." }, { "application": 6, "name": "Lucas Moreau", "summary": "Software developer looking for a career change into tech sales. Never worked in a warehouse." }, { "application": 7, "name": "Hanna Kowalski", "summary": "Student, wants a summer job. No experience and no forklift certificate. Available in July and August only." }, { "application": 8, "name": "Ravi Menon", "summary": "Chef for ten years in restaurants. No warehouse experience. Cannot work nights." } ] }, "questions": { "priority_1": { "type": "score", "instructions": "How high should application 1 go on the interview list?", "criteria": [ "Do not interview", "Interview if needed", "Interview first" ] }, "priority_2": { "type": "score", "instructions": "How high should application 2 go on the interview list?", "criteria": [ "Do not interview", "Interview if needed", "Interview first" ] }, "priority_3": { "type": "score", "instructions": "How high should application 3 go on the interview list?", "criteria": [ "Do not interview", "Interview if needed", "Interview first" ] }, "priority_4": { "type": "score", "instructions": "How high should application 4 go on the interview list?", "criteria": [ "Do not interview", "Interview if needed", "Interview first" ] }, "priority_5": { "type": "score", "instructions": "How high should application 5 go on the interview list?", "criteria": [ "Do not interview", "Interview if needed", "Interview first" ] }, "priority_6": { "type": "score", "instructions": "How high should application 6 go on the interview list?", "criteria": [ "Do not interview", "Interview if needed", "Interview first" ] }, "priority_7": { "type": "score", "instructions": "How high should application 7 go on the interview list?", "criteria": [ "Do not interview", "Interview if needed", "Interview first" ] }, "priority_8": { "type": "score", "instructions": "How high should application 8 go on the interview list?", "criteria": [ "Do not interview", "Interview if needed", "Interview first" ] }, "best": { "type": "choice", "instructions": "Which application should we interview first?", "criteria": { "application_1": null, "application_2": null, "application_3": null, "application_4": null, "application_5": null, "application_6": null, "application_7": null, "application_8": null } }, "red_flags": { "type": "noul", "instructions": "Does any application show a red flag, such as claims that contradict each other or a certificate that looks made up?" } }}JSON # Act when every answer is at least 0.70 sure; otherwise a person decides.jq -r '.answers as $a | $a | map_values(if .type == "noul" then [.noul, 1 - .noul] | max else .confidence end) | to_entries | min_by(.value) as $weakest | if $weakest.value >= 0.70 then "act: interview \($a.best.choice) first, red flags \(if $a.red_flags.noul >= 0.5 then "yes" else "no" end), " + "priorities \([range(1; 9) as $i | $a["priority_\($i)"].score | round] | map(tostring) | join(" "))" else "human: \($weakest.key) is only \($weakest.value * 100 | round / 100) sure" end' answer.jsonThe borderline case runs the same code with its own state. priority_5 is only 0.27 sure, under 0.70, so it prints a human line and a person decides. Change the threshold to match what a wrong answer costs you.
Composite scores
When a decision depends on several criteria, ask one score per criterion and combine them in your code with weights you control. You can explain the result ("strong on budget, weak on timing") and change the weights without touching the questions.
from typesafe_sdk import Score, TypeSafeClient # Reads TYPESAFE_BASE_URL (https://api.vakyn.com) and TYPESAFE_API_KEY from the environment.client = TypeSafeClient()LEVELS = ["Not at all", "Partly", "Clearly"]WEIGHTS = {"budget": 0.4, "timeline": 0.3, "decision_maker": 0.3} lead = ( "Hi, I run operations at a 40-person logistics firm. We set aside budget for a scheduling " "tool this quarter and want something live before the summer peak. I'll loop in our CFO.")result = client.system_one( state=lead, questions={ "budget": Score(instructions="Does the lead have budget for a purchase?", criteria=LEVELS), "timeline": Score(instructions="Does the lead need a solution soon?", criteria=LEVELS), "decision_maker": Score(instructions="Is the writer able to approve the purchase?", criteria=LEVELS), },)# Each score runs from 0 to 2; normalise to 0-1 and weigh.fit = sum(WEIGHTS[k] * result.scores[k].score / 2 for k in WEIGHTS)print(f"fit: {fit:.2f}")Use the expected score rather than the most likely level: it moves smoothly as the evidence changes, which makes the combined number easier to rank and to threshold.
Long documents
A request may hold up to 32,768 tokens of state plus its longest question, and 65,536 tokens overall. The server never cuts a document short: over the limit it answers 400 with max_tokens_exceeded. For longer documents, split them and combine the answers. For a question like "does any part say X?", the document says X unless every part says it does not:
from typesafe_sdk import Noul, TypeSafeBadRequestError, TypeSafeClient # Reads TYPESAFE_BASE_URL (https://api.vakyn.com) and TYPESAFE_API_KEY from the environment.client = TypeSafeClient()QUESTION = {"auto_renews": Noul(instructions="Does this text say the contract renews automatically?")} def ask(text: str) -> float: try: return client.system_one(state=text, questions=QUESTION).nouls["auto_renews"].noul except TypeSafeBadRequestError as e: if "max_tokens_exceeded" not in str(e.body): raise # Over the limit: split on paragraphs and ask each half. paragraphs = text.split("\n\n") if len(paragraphs) < 2: raise mid = len(paragraphs) // 2 halves = ["\n\n".join(paragraphs[:mid]), "\n\n".join(paragraphs[mid:])] # "Does any part say so?": yes unless every part says no. p_no = 1.0 for half in halves: p_no *= 1 - ask(half) return 1 - p_no contract = "\n\n".join(f"Clause {i}. The parties agree to clause {i}." for i in range(1, 6000))print(f"{ask(contract):.2f}")Many requests at once
A server admits a fixed number of requests at once, counting those running and those waiting their turn; beyond that it answers 429 with a retry-after header, and both SDKs wait and retry on their own. On open-server the number is --max-queue (32 by default). Send work concurrently and let the queue absorb it:
import asyncio from typesafe_sdk import AsyncTypeSafeClient, Noul MESSAGES = [ "Where is my parcel?", "Please cancel my subscription at the end of the month.", "Your app crashes when I open settings.",] async def main() -> None: # Reads TYPESAFE_BASE_URL (https://api.vakyn.com) and TYPESAFE_API_KEY from the environment. async with AsyncTypeSafeClient() as client: # The server queues what it can take and answers 429 beyond that; # the SDK retries 429 after the delay the server asks for. results = await asyncio.gather(*( client.system_one(state=m, questions={"cancel": Noul(instructions="Does the customer want to cancel?")}) for m in MESSAGES )) for m, r in zip(MESSAGES, results): print(f"{r.nouls['cancel'].noul:.2f} {m}") asyncio.run(main())