Open source · MIT

Make every
LLM response
reliable.

Production-grade reliability middleware for structured LLM outputs. Extract JSON, validate with Pydantic, repair malformed responses and return typed Python objects.

Get startedView source

llm-reliability-engine

  1. Prompt

  2. 2

    Raw Response

  3. 3

    JSON Extraction

  4. 4

    JSON Parsing

  5. 5

    Pydantic Validation

  6. 6

    Deterministic Repair

  7. 7

    LLM Repair

  8. 8

    StructuredResult

Raw LLM Response

malformed
Sure! Here is the user:
```json
{
  "name": "Ada Lovelace",
  "email": "ada@analytical.dev",
  "age": "36",
  "roles": ["admin", "engineer",],
}
```

Successfully validated after 2 attempts

StructuredResult<User> returned

Attempts
2
Parse Method
deterministic
Latency
812 ms
Provider
openai:gpt-4o-mini

repaired: trailing comma removed · age coerced to int

Features

Everything between the model and your types.

Provider agnostic

One interface across OpenAI, Anthropic, Mistral, Ollama and any HTTP endpoint. Swap providers without touching parsing logic.

Automatic JSON repair

Deterministic fixes for fenced blocks, trailing commas, single quotes and truncated objects before a retry is ever spent.

Pydantic validation

Responses are coerced into your Pydantic models, so downstream code receives typed Python objects instead of dictionaries.

Production ready

Attempt-level metadata, latency tracking, structured logging and bounded retries with exponential backoff built in.

Quick start

Four lines from prompt to typed object.

from pydantic import BaseModel
from llm_reliability_engine import ReliableClient

class User(BaseModel):
    name: str
    email: str
    age: int
    roles: list[str]

client = ReliableClient(provider="openai:gpt-4o-mini", max_attempts=3)

result = client.structured(
    prompt="Extract the user profile from this email thread.",
    schema=User,
)

print(result.value.name)
print(result.attempts)
print(result.parse_method)

Playground

Watch a broken response become a typed object.

Prompt

Raw Response

unparseable

Structured Output

validated