Skip to content
Patterns

Long documents

When a document does not fit in one request, split it into parts, ask each part the same question and combine the answers in your code.

When to use it

  • Your documents are sometimes longer than the limit: contracts, reports, long email threads, logs.
  • The question is about the whole document, such as "does it say X anywhere?".
  • You would rather get an answer for every document than reject the long ones.

The limits

A request may hold up to 32,768 tokens of state plus its longest question, and 65,536 tokens overall (see the API limits). The server never cuts a document short: over the limit it answers 400 with max_tokens_exceeded; split the document when you get it.

How it works

Split the document on natural boundaries and ask each part the same noul. How you combine the answers depends on the question. For "does any part say X?", the document says X unless every part says it does not: the probability of no is the product of each part's no.

  1. Document

    Too long for one request

    • 400 max_tokens_exceeded

  2. Split

    Parts on natural boundaries

    • sections, paragraphs, pages

  3. Ask

    Same noul, each part

    • p₁, p₂, … pₙ

  4. Combine

    In your code

    • any: 1 − Π(1 − pᵢ)
    • every: Π pᵢ
Split, ask each part, combine the probabilities.

This version tries the whole document first and splits in half only when the server says it is too long, so short documents still cost one request:

from typesafe_sdk import Noul, TypeSafeBadRequestError, TypeSafeClient # Reads TYPESAFE_BASE_URL (https://api.vakyn.com) and TYPESAFE_API_KEY from the environment.client = TypeSafeClient()QUESTION = {"auto_renews": Noul(instructions="Does this text say the contract renews automatically?")}  def ask(text: str) -> float:    try:        return client.system_one(state=text, questions=QUESTION).nouls["auto_renews"].noul    except TypeSafeBadRequestError as e:        if "max_tokens_exceeded" not in str(e.body):            raise        # Over the limit: split on paragraphs and ask each half.        paragraphs = text.split("\n\n")        if len(paragraphs) < 2:            raise        mid = len(paragraphs) // 2        halves = ["\n\n".join(paragraphs[:mid]), "\n\n".join(paragraphs[mid:])]        # "Does any part say so?": yes unless every part says no.        p_no = 1.0        for half in halves:            p_no *= 1 - ask(half)        return 1 - p_no  contract = "\n\n".join(f"Clause {i}. The parties agree to clause {i}." for i in range(1, 6000))print(f"{ask(contract):.2f}")

Tips

  • If a part needs context from elsewhere (the parties to a contract, the title of a report), repeat that short context at the top of every part.
  • The parts can go out at the same time; see many requests at once.
  • Questions that need the whole document at once, like "what is the overall tone?", do not split well. Ask them on a summary or on the parts that matter instead.

Loading the docs…