What Is Laya? The Open-Source Jev Alternative You Can Run Locally

Quick answer
Laya is an Apache-2.0 decision engine from Convai Innovations that runs typed choice, score, and noul questions on your own hardware.
It is a separate family of encoders, not a compressed Jev: 322M-421M parameters across three checkpoints, with a router that picks one per request.
The English and typed-decisions checkpoints use ModernBERT-large; the multilingual checkpoint uses mmBERT-base and covers 100+ languages.
It runs on CPU, Apple Metal, CUDA, or Intel XPU, and ships a CLI, an HTTP server that speaks the Jev wire protocol, and an MCP stdio server.
Zero-shot quality trails hosted decision models on harder semantic tasks; fine-tuning on your own labeled decisions is where it becomes competitive.

What is Laya?

Laya is an open-source decision engine from Convai Innovations, released under the Apache 2.0 license. You give it a state and a set of typed questions, and it returns a choice, a score, or a yes/no probability in a single forward pass. It does not generate text, and it does not require a hosted API: the checkpoints run on your machine, on a server you control, or in a container.

Laya is often described as an open-source alternative to Jev, and that is accurate at the interface level. It supports the same three text question primitives and its HTTP server implements the Jev wire protocol, so a client written for Jev can point at a local Laya server. It is not the same model, and API compatibility is not a quality guarantee. Treat it as a different implementation with different trade-offs, not as a free clone.

Which checkpoints does Laya ship?

The package includes three checkpoints, and a Router chooses between them per request. The English checkpoint is a 421M-parameter ModernBERT-large encoder with a 512-token context. The multilingual checkpoint is a 322M-parameter mmBERT-base encoder with a 1,024-token context, extendable to 8,192 tokens for long documents, and covers more than one hundred languages. The typed-decisions checkpoint is another 421M-parameter ModernBERT-large model tuned for the typed-decision workflows.

The router uses language and script detection to choose a checkpoint, which is why the same call works for an English ticket and a Spanish one. You can override it by pinning a checkpoint name, forcing a language code, or naming a workflow. Pinning is useful for reproducibility, but it also means you take responsibility for sending an input the pinned checkpoint can handle; an English checkpoint given a long non-English document will not route itself out of trouble.

Laya's three checkpoints

  laya                 421M  ModernBERT-large   512 tokens    English
  laya-multilingual    322M  mmBERT-base        1,024 tokens  100+ languages
                                                    up to 8,192
  laya-typed-decisions 421M  ModernBERT-large   1,024 tokens  typed-decision workflows

The Router picks one from the input's language and script unless you pin model=.

What does running it locally buy you?

The obvious benefit is cost: there is no per-call fee, so the marginal cost of a decision is the hardware time it consumes. The less obvious benefit is control. Inputs do not leave your infrastructure, the model runs offline, and you can version the checkpoint alongside your code. For an app handling health data, customer records, or anything under a data-residency requirement, that can decide the architecture before quality does.

Local inference is also fast on the right hardware. The model is a single forward pass rather than a token-by-token decode, so latency is predictable and does not grow with the length of an answer. The catch is that you now own the operational work: downloading checkpoints, sizing memory, choosing a device, keeping the server healthy, and evaluating a model that will not improve unless you fine-tune it.

Where does Laya fit in a vibecoded app?

The same places a hosted decision model fits: support triage, intent routing, moderation, retrieval-domain selection, lead scoring, and guardrails before an agent acts. Laya also has a practical advantage for local coding agents, because it can run beside the agent without sending every tool call to a third party. The MCP server exposes the decision tools over stdio, so an agent can ask for a typed judgment without a network round trip.

The design rule does not change. Laya returns a judgment; your code owns the policy. A high-confidence answer is not permission to run a destructive command, issue a refund, or delete a record. Use the answer to choose a branch, then validate the branch with ordinary code. The local deployment moves the model inside your trust boundary, but it does not make the model’s judgment infallible.

What are the downsides?

The main one is quality. Laya’s base checkpoints are small encoders trained for zero-shot typed decisions, and on harder semantic tasks they trail the hosted models. The project’s own benchmark shows a large gap between the base checkpoints and a fine-tuned checkpoint, which is a useful signal: if your task is domain-specific, budget for fine-tuning and a labeled dataset, not just an install. If your task is generic, test before assuming the free model is good enough.

The second is operational weight. The checkpoints are hundreds of megabytes each, Router() keeps up to two resident by default, and the first prediction downloads from Hugging Face. CPU latency is usable but not free, especially on long inputs. There is no vendor SLA, no managed upgrades, and no one to call when a checkpoint behaves differently after an update. Those are reasonable costs for some teams and disqualifying for others.

How do you try it in five minutes?

Install the package on Python 3.10 or newer, create a Router, and send one state with a few questions. The first call downloads the checkpoint the router selects, so it takes longer than later calls. If you only want to see the routing decision, the CLI can do that without downloading a checkpoint at all. The full setup, including the HTTP server and the MCP server, is in the linked tutorial.

Start with a question you can check by eye, then compare the answer with what you expected. If the model is unsure, inspect the probability distribution before changing the threshold. A flat distribution usually means the options overlap or the state contains more than one request. That is a question-design problem, and no amount of local inference speed fixes it.

python3 -m venv .venv
.venv/bin/python -m pip install laya
.venv/bin/laya "I was charged twice, please refund the duplicate" --preset triage

Where this bites vibecoders
The appeal of Laya is that it makes the decision layer free, and the trap is believing that free means finished. The install is easy; the evaluation is the work. An assistant will generate a confident-looking integration from the README and never test it on the messy, multilingual, half-redacted inputs the app actually receives. Run a labeled set, compare against the rules or LLM you are replacing, and only then decide whether local inference is enough.

Where AI coding assistants get this wrong

Assuming API compatibility with Jev means the same accuracy or the same behavior.
Pinning the English checkpoint for multilingual traffic and blaming the model for bad answers.
Sending a long document without raising max_len, then wondering why the answer ignores the end.
Running the HTTP server on a public interface without setting LAYA_API_KEY.
Skipping fine-tuning on a domain-specific task and concluding the model is unusable.
Treating a local model as exempt from evaluation because the data never leaves the machine.

Checklist

Confirm the Apache-2.0 license and checkpoint terms for your use.
Install on Python 3.10+ and run one prediction before building around it.
Check which checkpoint the router selects for your real inputs.
Measure CPU or GPU latency at the percentile your app needs.
Set a confidence threshold and a fallback for low-confidence answers.
Budget for fine-tuning if the task is domain-specific.
Put authentication in front of any HTTP deployment.

FAQ

Is Laya really free?

The software and checkpoints are Apache 2.0, so there is no per-call license fee. You still pay for the hardware, electricity, and engineering time to run and evaluate it. If you use a hosted Laya endpoint from a provider, that provider’s price applies. “Free” describes the license, not the total cost of ownership.

Is Laya better than Jev?

It depends on the constraint. Laya wins on data residency, offline operation, per-call cost at high volume, and the ability to fine-tune. Jev wins on zero-shot quality on harder semantic tasks and on not having to operate a model. Neither is universally better; test both on your labeled inputs if both are viable.

Does Laya work in other languages?

Yes. The multilingual checkpoint covers more than one hundred languages, and the router sends non-English text to it automatically. Long documents need max_len=8192 and an explicit multilingual model, because the default 1,024-token limit truncates them. Accuracy varies by language, so test the languages your users actually write.

Can I use the Jev SDK with Laya?

Against the HTTP server, often yes, because laya-serve implements the Jev /v1/systemone wire protocol. That is useful for avoiding a rewrite, but confirm the response fields your client depends on and keep an adapter in your code. The SDKs are not guaranteed to expose every Laya-specific field such as routing metadata.

How much memory does it need?

It depends on the checkpoint and precision, but plan for a few gigabytes of disk for the checkpoints and enough RAM to hold one or two of them. Router() keeps up to two resident by default and evicts the least recently used; Router(max_loaded=1) lowers the footprint at the cost of reloads. Measure on your target machine rather than sizing from parameter count alone.