AKAS
Compiler-first model inference.
Compose models in pure Python, fuse a whole model at compile time, and serve AOT-compiled .tbin programs without PyTorch or TileLang at runtime.
- Version
- 0.1.0 (pre-1.0)
- License
- MIT
- Language
- Python 3.12
- Built on
- Tensor
The model is compiled once, then served from a lock.
- 01
Declare
Write the model in Python as a declarative graph, with quantization codecs fused straight into the ops.
- 02
Compile
Whole-graph AOT compilation fuses the model, prunes dead prefill work, and lifts row-local state into slot pools.
- 03
Optimize
akas optimize takes a checkpoint, a workload and a budget, then searches behind a three-tier gate.
- 04
Serve
akas serve reads the locked deployment.json and serves immediately — it never compiles at request time.
Three assumptions other stacks make.
These are not benchmarks. They are what the code actually depends on.
Tensor plus one regex package
AKAS's dependency list is Tensor and one regex package. No torch.nn, no torch.compile, and no TileLang at runtime.
Declarative graphs, inline codecs
Models are written in Python. Quantization codecs — Q4_0, Q4_K, Q6_K, FP8, Q16 — fuse straight into CUDA and WebGPU ops, so you are not hand-writing a shader matrix.
A three-tier gate
Candidates must clear a numerical gate against the baseline, then an engine gate on native phases, then an HTTP burst gate on a live server, before a deployment is written.
Compiled once, served from a lock
Give it a checkpoint, a workload and a time budget. It searches geometries, chunking, policies and schedules, then writes the deployment.
The numbers, and what they do not establish.
- prefill vs llama.cpp
- +74%
- context lengths led on decode
- 8 / 8
- tok/s at full context
- 79.55
- peak GPU memory
- 7.71 GiB
at a 130,944-token prefix · LFM2.5-24B-A2B, Q4_0, NVIDIA L4
LFM3-1.7B, Q4_0 · AMD Radeon RX 6700 XT, WebGPU vs Vulkan
vs 74.63 tok/s for llama.cpp (+6.59%)
vs 7.73 GiB · full 131,072-token context reserved
What these numbers are not
- These are phase rates, not end-to-end serving throughput. Complete-burst HTTP throughput was not measured.
- On CUDA, AKAS and llama.cpp time different boundaries. The Vulkan decode comparison is the one that uses a matched protocol.
- One request, 128 output tokens, one GPU per campaign. Other models, devices and workloads differ.
- The WebGPU path establishes coverage and correctness on AMD hardware — not tensor-core performance or native FlashAttention parity.
Every figure here comes from a study in the repository, with hardware, protocol, retained samples and limitations recorded. Read the method, not just the headline.
Run it locally.
Use Python 3.12. The compiler is needed to build programs; precompiled programs run without the compiler packages.
git clone https://github.com/kreasof-ai/akas && cd akas/uv sync --locked --extra compiler --extra servingakas optimize --model model.gguf --workload standard-v1 --budget 30m --out build/deploymentakas serve --deployment build/deployment/deployment.json