Using Jev in LangChain and LangGraph

Which API to use

“Beta” is LangChain's own marker: the classifier is decorated @beta() and emits a LangChainBetaWarning when first constructed. Both middleware live under langchain_typesafe.experimental, which the package says “may change without notice.”

TypeSafeClassifier

Construct the classifier once and pass state and questions together on every call. Questions moved into the call in 0.0.1a3 (PR #40659 ↗), so an example that passes questions= to the constructor is from an earlier alpha. In 0.0.1a3 that raises a validation error, and invoking with only a state string raises TypeError.

from langchain_typesafe import Choice, Noul, TypeSafeClassifier

classifier = TypeSafeClassifier()  # reads TYPESAFE_API_KEY

result = classifier.invoke(
    {
        "state": "I was charged twice in September. Refund one?",
        "questions": {
            "team": Choice(
                instructions="Which team should handle this ticket?",
                criteria={
                    "billing": "Charges, invoices, refunds, plans.",
                    "technical": "Bugs, errors, integrations.",
                    "other": "Anything else.",
                },
            ),
            "refund": Noul(
                instructions="Is the customer asking for money back?"
            ),
        },
    }
)

team = result.choices["team"]  # ChoiceAnswer
if team.confidence >= 0.7:
    print("route to", team.choice)
else:
    print("send to triage")

print(result.nouls["refund"].noul)  # probability of yes, 0-1
print(result.model, result.usage, result.request_id)

Questions must be Choice, Noul, or Score objects. Plain dicts such as {"type": "noul", ...}, which TypeSafe's own SDK accepts, raise AttributeError here. For which primitive fits a decision, see Choice, Score, and Noul.

Reading the result

Thresholds like the 0.7 above are yours to set from your own data. The API design behind them is covered in the classification task.

Contract details

ModelRouterMiddleware Experimental

Jev picks which of your chat models handles an agent run. Once, before the agent starts (before_agent), the middleware asks a single Choice with one label per ModelChoice. It stores the full ChoiceAnswer in agent state as model_route, then swaps that model into every model call for the run (wrap_model_call).

from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import (
    ModelChoice,
    ModelRouterMiddleware,
)

router = ModelRouterMiddleware(
    choices={
        "fast": ModelChoice(
            model=fast_model,  # chat model or "provider:model"
            criteria="Lookups, extraction, short edits.",
        ),
        "strong": ModelChoice(
            model=strong_model,
            criteria="Multi-step reasoning or high stakes.",
        ),
    },
    instructions="Pick the cheapest model that can do the task.",
)

agent = create_agent(fast_model, middleware=[router])
question = {"role": "user", "content": "Prove √2 is irrational."}
result = agent.invoke({"messages": [question]})

route = result["model_route"]  # the full ChoiceAnswer
print(route.choice, route.confidence)

Whether routing saves money or time depends on your models and traffic; the package makes no such claim and neither do we. The design questions (label criteria, fallbacks, measuring misroutes) are on the model routing task page.

AutoModeMiddleware Experimental

Jev estimates whether a proposed tool call is risky or insufficiently authorized, and the middleware blocks it if so. It hooks wrap_tool_call. It classifies only the tools you list, and each call separately.

from langchain.agents import create_agent
from langchain.tools import tool
from langchain_typesafe import NoulCriteria
from langchain_typesafe.experimental.middleware import (
    AutoModeMiddleware,
)


@tool
def delete_backups(env: str) -> str:
    """Delete every backup in an environment."""
    return backups.delete_all(env)  # your code still authorizes


@tool
def read_status(env: str) -> str:
    """Read an environment's health status."""
    return status.read(env)


auto_mode = AutoModeMiddleware(
    tools=[delete_backups],  # read_status is never classified
    criteria=NoulCriteria(
        true="Deletes, overwrites, publishes, or changes access.",
        false="Only reads data the user asked about.",
    ),
)

agent = create_agent(
    model,
    tools=[delete_backups, read_status],
    middleware=[auto_mode],
)

What Jev decides, what you enforce

AutoMode is a risk estimate, not a security boundary. TypeSafe's own jev-1.13 limitations page ↗ says injected instructions or text arguing for its own classification “can move the answer.” Those messages and tool results are exactly what this middleware sends. LangChain also warns against putting secrets in tool arguments or state unless sending them to TypeSafe is acceptable.

Both middleware: configuration

Each middleware builds its own TypeSafeClassifier() from the environment when you construct it. The constructors take no API key, Jev model, timeout, or HTTP client. So TYPESAFE_API_KEY must be set before the middleware is created, calls use jev-latest, and TYPESAFE_BASE_URL is honored. The instance is exposed as .classifier. Replacing it (to pin a Jev version or inject a test transport) works in 0.0.1a3 but isn't a documented option.

Both need the [experimental] extra, which adds langchain (create_agent). Without it the import fails with an explicit install hint. Both set a trace policy (omit_payload) that keeps the middleware step's inputs out of traces.

LangGraph

There is no LangGraph-specific Jev API. Two things are true instead. An agent from create_agent runs as a LangGraph graph, so both middleware already work there. And in a hand-built StateGraph, TypeSafeClassifier is an ordinary Runnable: call it in a node and branch on the answer in a conditional edge. Low confidence becomes an explicit route instead of a silent guess.

from typing import TypedDict

from langgraph.graph import START, StateGraph
from langchain_typesafe import Choice, TypeSafeClassifier

classifier = TypeSafeClassifier()

TEAM = Choice(
    instructions="Which team should handle this ticket?",
    criteria={
        "billing": "Charges, invoices, refunds, plans.",
        "technical": "Bugs, errors, integrations.",
        "other": "Anything else.",
    },
)


class Ticket(TypedDict, total=False):
    text: str
    team: str
    confidence: float


def classify(state: Ticket) -> Ticket:
    result = classifier.invoke(
        {"state": state["text"], "questions": {"team": TEAM}}
    )
    answer = result.choices["team"]
    return {"team": answer.choice, "confidence": answer.confidence}


def route(state: Ticket) -> str:
    if state["confidence"] < 0.7:
        return "triage"
    return state["team"]


graph = StateGraph(Ticket)
graph.add_node("classify", classify)
graph.add_node("billing", billing)  # your existing nodes
graph.add_node("technical", technical)
graph.add_node("other", other)
graph.add_node("triage", triage)
graph.add_edge(START, "classify")
graph.add_conditional_edges("classify", route)
app = graph.compile()

Errors, retries, tracing

HTTP failures raise TypeSafeAPIError subclasses that also inherit LangChain's standard model errors: a 429 is both TypeSafeRateLimitError and ModelRateLimitError, with retry_after_ms and request_id. Import the TypeSafe classes from langchain_typesafe.client. The package README imports them from the top level, which fails in 0.0.1a3.

There is no built-in retry, unlike TypeSafe's SDK. Use the Runnable wrapper (it backs off on its own schedule and ignores retry_after_ms):

from langchain_core.exceptions import ModelRateLimitError
from langchain_typesafe import TypeSafeClassifier
from langchain_typesafe.client import TypeSafeAPIError  # not top-level

classifier = TypeSafeClassifier().with_retry(
    retry_if_exception_type=(ModelRateLimitError,),
    stop_after_attempt=3,
)

With LangSmith tracing on, each call is recorded as an llm run with provider typesafe, the Jev model name, and token usage. Without tracing nothing extra happens; a tracing failure never fails a classification.

LangChain or the TypeSafe SDK?

Neither is better in general. The same Jev call sits underneath both; what differs is what you get around it.

For keys, endpoints, and limits, see API & access. For Node, see the TypeScript SDK guide.

LangChain.js and LangSmith

LangChain.js has a separate package, @langchain/typesafe (0.0.1 on npm). Its contract differs from Python's: questions go to the TypeSafeClassifier constructor and invoke takes only state. Don't port the examples on this page line for line; use its README ↗.

LangSmith can also use Jev as an evaluator that scores traces, set up in the LangSmith UI with a TypeSafe key. That is an evaluation workflow, separate from this runtime package; see Jev as a LangSmith evaluator.