On this page
  1. What is Laya AI?
  2. Laya at a glance
  3. Which Laya checkpoint should you use?
  4. How does the Laya router choose a checkpoint?
  5. How fast is Laya?
  6. What does Laya cost?
  7. What is Laya good at?
  8. Where does Laya fail?
  9. How does Laya compare with Jev?
  10. How do you install and run Laya?
  11. How do you fine-tune and secure Laya?
  12. Where AI coding assistants get this wrong
  13. Checklist
  14. FAQ
    1. What is Laya AI in one sentence?
    2. Is Laya AI free?
    3. Does Laya need a GPU?
    4. How accurate is Laya?
    5. Can I use Laya with the Jev SDK?
    6. How much memory does Laya use?
    7. How do I improve Laya’s accuracy?
  15. Related topics
  16. Sources
pillar

Laya AI: The Complete Guide to the Open-Source Decision Model

Laya AI explained: checkpoints, router, local setup, speed, cost, fine-tuning, security, and how it compares with Jev and other decision models.

Software Engineeringbeginner11 min readUpdated 2026-10-21

Quick answer

  • Laya AI is an Apache-2.0 decision engine from Convai Innovations that runs typed choice, score, and noul questions on your own hardware.
  • It ships three checkpoints: English (421M), multilingual (322M, 100+ languages, up to 8,192 tokens), and typed-decisions (421M). A router picks one per request.
  • It runs on CPU, Apple Metal, CUDA, or Intel XPU, and exposes a CLI, a Jev-compatible HTTP server, and an MCP stdio server.
  • The software is free. The real costs are hardware, operations, evaluation, and the fine-tuning work that makes it competitive on a narrow task.
  • Zero-shot quality trails hosted models on harder semantic tasks. Fine-tuning on your own labeled decisions is where accuracy improves.

What is Laya AI?

Laya AI is an open-source decision engine from Convai Innovations, released under the Apache 2.0 license. You give it a state and a set of typed questions, and it returns a choice, a score, or a yes/no probability in a single forward pass. It does not generate text and does not require a hosted API. The checkpoints run on your machine, a server you control, or a container.

Laya is often described as an open-source alternative to Jev, and that is accurate at the interface level: it supports the same three text question primitives, and its HTTP server implements the Jev wire protocol. It is not the same model, and API compatibility is not a quality guarantee. Treat it as a separate implementation with different trade-offs, not as a free clone of a hosted model.

Laya at a glance

The facts that decide whether Laya fits are the license, the checkpoint size, the language coverage, the deployment surface, and the operational burden. Laya is a small encoder family rather than a large generative model, so the hardware is modest but the accuracy on hard zero-shot tasks is lower than a hosted frontier decision model.

Laya AI at a glance

  Project           Convai Innovations / NandhaKishorM
  License           Apache 2.0
  Model type        non-autoregressive decision engine
  Checkpoints       English 421M, multilingual 322M, typed-decisions 421M
  Context           512 (English), 1,024 (multilingual/typed), 8,192 max for long docs
  Languages         100+ with the multilingual checkpoint
  Devices           CPU, Apple Metal, CUDA, Intel XPU
  Interfaces        Python SDK, CLI, HTTP server, MCP stdio server
  Pricing           free software; you pay for hardware and operations
  Fine-tuning       laya-train from CSV or JSONL
  Best fit          local/offline triage, routing, guardrails, data residency

Which Laya checkpoint should you use?

The English checkpoint is the default for English text and has a 512-token context. The multilingual checkpoint is smaller, covers more than one hundred languages, and accepts up to 8,192 tokens when you pass max_len=8192; it is the one to use for long documents or non-English inputs. The typed-decisions checkpoint is tuned for the benchmark’s decision workflows and is the best starting point when your task resembles typed classification.

The router chooses a checkpoint from the input’s language and script unless you pin one. Pinning is useful for reproducibility, but it also means you are responsible for the mismatch: an English checkpoint given a long Spanish document will not route itself out of trouble. If your inputs are consistently one language, pinning avoids a reload; if they are mixed, let the router work and budget for two resident checkpoints.

How does the Laya router choose a checkpoint?

The router detects the script and language of the state, then selects the checkpoint that fits. Non-English text goes to the multilingual checkpoint; English text goes to the English checkpoint. You can override the decision with a model name, a language code, or a language hint, and a task name can force the typed-decisions workflow instead of detecting it. The response includes a routing block that records the model, repository, and reason, which is the first thing to log when an answer looks wrong.

Routing alone does not download a checkpoint, which makes the CLI’s route mode a cheap way to see what would happen before loading weights. The first prediction downloads the selected checkpoint from Hugging Face and caches it; later calls reuse the cache. A router keeps up to two checkpoints resident by default, and Router(max_loaded=1) trades memory for reloads when the language switches.

How fast is Laya?

Laya’s advantage is that it computes a decision in a single forward pass rather than generating tokens one at a time. The project reports 33 ms for one question and 7.2 ms per question batched on a T4, with CPU and Apple Metal numbers that depend on the machine. Those are useful starting points, not a promise for your deployment: the checkpoint, input length, batch size, thread count, and device all move the number.

Measure a warm process on your target hardware, at the input length you actually receive. A cold start includes a checkpoint download or load and is much slower. On CPU, cap Torch’s intra-op threads at or below the physical core count; oversubscribing logical cores usually makes inference slower. If the decision is on a user-facing path, report p95 rather than the median and keep a fallback for the cold case.

What does Laya cost?

The software and checkpoints are Apache 2.0, so there is no per-call license fee. The costs are the machine that runs the model, the electricity, the engineering time to deploy and monitor it, the labeled data and GPU time for fine-tuning, and the evaluation work that decides whether it is good enough. At low volume, those fixed costs can exceed a hosted API’s per-call bill; at high volume or under a data-residency constraint, the local option can be the only viable one.

The honest comparison is total cost per resolved decision, not price per call. A hosted model includes the serving, upgrades, and a support relationship. A local model includes a checkpoint you pin, a server you secure, a GPU or CPU you size, and a fine-tune you may have to repeat when the distribution changes. Budget for all of it before choosing Laya on the license price alone.

What is Laya good at?

Laya is a strong fit when the decision is local, bounded, and repeated: triage, intent routing, moderation, retrieval-domain selection, lead scoring, and guardrails before an agent acts. It is especially useful when the data cannot leave your infrastructure, when the app must work offline, or when the per-call price of a hosted model would dominate the cost of the feature.

It is also a good fit for coding agents, because the MCP server exposes the decision tools over stdio and keeps the model beside the agent rather than behind a third-party API. The agent can ask for a typed judgment about a command, a diff, or a support message without sending the content to a hosted decision service. The agent still owns the action; Laya supplies the judgment.

Where does Laya fail?

The base checkpoints are small encoders trained for zero-shot typed decisions, and on harder semantic tasks they trail hosted models. Laya’s own benchmark shows the base checkpoints near chance on the typed-decisions task and a large improvement after fine-tuning. That is the key caveat: if your task is domain-specific, budget for labeled data and a fine-tune, not just an install. If it is generic, test before assuming the free model is enough.

The operational burden is the second limitation. The checkpoints are hundreds of megabytes each, Router() keeps up to two resident, and the first prediction downloads from Hugging Face. There is no vendor SLA, no managed upgrade path, and no one to call when a checkpoint behaves differently after an update. The HTTP server also binds 0.0.0.0:8000 with no authentication unless you set LAYA_API_KEY. Those are reasonable costs for some teams and disqualifying for others.

How does Laya compare with Jev?

Jev is hosted and managed, with text as its production input and images as an opt-in preview feature. Its published pricing is $0.042 per million input tokens, with vendor-reported 70-500 ms latency. Laya is local, Apache 2.0, multilingual, and free to run, with lower zero-shot quality on hard tasks and an operations burden you own. Jev is the shortest path to a production text-decision layer; Laya is the shortest path to a decision layer that never leaves your infrastructure.

The choice often comes down to one constraint. If the data can leave and the task is text, Jev is cheaper in total at low-to-medium volume and requires almost no operations. If the data cannot leave, the app must work offline, or the volume is high enough that per-call pricing dominates, Laya is the better fit. For a balanced comparison including Clef and OpenAI, use the linked comparison guide.

How do you install and run Laya?

Install on Python 3.10 or newer with pip install laya, create a Router, and send one state with a few questions. The first call downloads the routed checkpoint; later calls reuse the cache. For a quick check without writing a script, the CLI installs with the package and can route or predict from the terminal. The HTTP server and MCP server are optional extras rather than part of the core install.

The full tutorial covers the virtual environment, the first prediction, the CLI presets, laya-serve with authentication, the MCP configuration, and the memory and device settings. Start there if you want a working local endpoint; come back to the fine-tuning and security guides once the basic call is reliable.

How do you fine-tune and secure Laya?

Fine-tuning is where Laya becomes competitive on a narrow task. The package includes laya-train, which reads a CSV or JSONL of labeled decisions, fine-tunes a checkpoint, fits calibration temperatures on a held-out slice, and writes a checkpoint that laya.load can open. Start with a dry run, keep a test set you do not train on, and evaluate calibration before gating on confidence.

Security is a separate project from accuracy. Pin a checkpoint revision, verify artifact digests if you need integrity, set LAYA_API_KEY before exposing laya-serve, and treat the state as untrusted text. A local model is not immune to prompt injection or to a poisoned checkpoint, and a high-confidence answer is not authorization for an action. The linked guides cover both the training loop and the deployment boundary.

Where this bites vibecoders

Laya’s install is easy enough to make it feel finished after the first prediction. The work that decides whether it belongs in production is the labeled set, the fine-tune, the calibration check, and the server boundary. An assistant will generate the first script in one pass and skip all four. Do the unglamorous part before you put the model on a user path.

Where AI coding assistants get this wrong

  • Assuming API compatibility with Jev means the same accuracy or behavior.
  • Pinning the English checkpoint for multilingual traffic.
  • Sending a long document without max_len=8192 and assuming the model read the end.
  • Exposing laya-serve without LAYA_API_KEY.
  • Skipping fine-tuning on a domain-specific task and concluding the model is unusable.
  • Gating on confidence before fitting calibration temperatures.

Checklist

  • Confirm the Apache-2.0 license and checkpoint terms for your use.
  • Run one prediction and check the routing decision.
  • Measure warm p95 latency on your target hardware.
  • Decide whether the task needs fine-tuning or works zero-shot.
  • Set LAYA_API_KEY and a concurrency limit before exposing the server.
  • Pin the checkpoint revision and verify digests where integrity matters.
  • Keep the action in code; treat the model output as a recommendation.

FAQ

What is Laya AI in one sentence?

Laya AI is an Apache-2.0 decision engine that answers typed choice, score, and yes/no questions locally, using a small encoder and a router instead of a hosted text-generation model.

Is Laya AI free?

The software and checkpoints are Apache 2.0, so there is no per-call license fee. You still pay for hardware, electricity, deployment, monitoring, and the engineering time to evaluate and fine-tune. A hosted Laya endpoint from a provider has that provider’s price.

Does Laya need a GPU?

No. It runs on CPU, Apple Metal, CUDA, and Intel XPU, and selects an available device automatically. A GPU improves latency and throughput, but CPU is usable for development and low-volume inference. Fine-tuning is the part that realistically wants a GPU.

How accurate is Laya?

On Laya’s typed-decisions benchmark, the base checkpoints score around 0.36 against a 0.318 random baseline, while the fine-tuned checkpoint reaches 0.766. That is a vendor-run benchmark, not a guarantee for your task. Zero-shot performance on hard semantic tasks is lower than hosted models; fine-tuning on your own labels is where the gain is.

Can I use Laya with the Jev SDK?

Against laya-serve, often yes, because the server implements the Jev /v1/systemone wire protocol. Check the response fields your client depends on and keep an adapter in your code. API compatibility covers the request and answer shapes, not every Laya-specific field such as routing metadata.

How much memory does Laya use?

Plan for a few gigabytes of disk for the checkpoints and enough RAM to hold one or two of them. Router() keeps up to two resident by default and evicts the least recently used; Router(max_loaded=1) lowers the footprint at the cost of reloads. Measure on your target machine rather than sizing from parameter count alone.

How do I improve Laya’s accuracy?

Fine-tune on labeled decisions from your domain with laya-train, then evaluate on a held-out set. Keep a calibration slice, fit temperatures, and check option-order stability for questions with many options. More high-quality labels usually help more than a larger base model for a narrow task.

Sources

Share: