Many requests at once

When to use it

• You have a backlog to work through: last month's tickets, a folder of contracts, a product catalog.
• Many users or workers call VAKYN at the same time.
• You split a long document and want the parts answered together.

How it works

A server admits a fixed number of requests at once, counting those running and those waiting their turn. Beyond that it answers 429 with a retry-after header, and both SDKs wait and retry on their own. On open-server the number is --max-queue (32 by default; see flags). So you do not need your own rate limiter: send work concurrently and let the server queue it.

This example answers three messages concurrently with the async client:

import asyncio from typesafe_sdk import AsyncTypeSafeClient, Noul MESSAGES = [    "Where is my parcel?",    "Please cancel my subscription at the end of the month.",    "Your app crashes when I open settings.",]  async def main() -> None:    # Reads TYPESAFE_BASE_URL (https://api.vakyn.com) and TYPESAFE_API_KEY from the environment.    async with AsyncTypeSafeClient() as client:        # The server queues what it can take and answers 429 beyond that;        # the SDK retries 429 after the delay the server asks for.        results = await asyncio.gather(*(            client.system_one(state=m, questions={"cancel": Noul(instructions="Does the customer want to cancel?")})            for m in MESSAGES        ))    for m, r in zip(MESSAGES, results):        print(f"{r.nouls['cancel'].noul:.2f}  {m}")  asyncio.run(main())

Tips

• For a very large batch, cap how many requests you keep in flight (an asyncio.Semaphore works) so you do not hold thousands of open connections waiting on retries.
• Several questions about the same document belong in one request, not many; see fan-out.
• Errors and headers, including retry-after-ms, are listed in the API reference.