ashhart/TensorFold
- Stars
- 1,923
- Language
- Zig
- License
- Apache-2.0
- Daily
- rank 7 · +94 stars gained
Repository metrics were observed on this edition date and may have changed.
Key Features An inference engine whose Zig binary reads checkpoints, tokenizes requests, and serves language models through Metal and CUDA kernels. Core serving does not require Python or MLX installation, and it provides Responses and Anthropic Messages APIs, request queuing, and concurrency limits. Use Cases The documentation explains how to download a model, serve it through a local server, and send requests to the chat API, as well as how to connect existing local checkpoints or a Hugging Face cache. A separate local web app supports model selection, execution, and chat. Costs and Limitations The engine code is licensed under Apache-2.0, while model weights have separate licenses. Local serving should be distinguished from external paid APIs; the default examples use local models. Hardware, memory, storage, and electricity costs are separate, and model-specific cost conditions need to be confirmed. Adoption Considerations The Linux NVIDIA distribution requires glibc 2.28 or later and an R580 or later driver. Check each model's supported backends and memory requirements; the web app may require a separate installation. Living Weights is experimental, modifies checkpoint files, and has stated recall and speed limitations. Why It Stands Out It was observed at No. 7 on daily Trending, with a star increase of 94. Version 1.2.7, listed on 2026-11-12, describes improved state restoration for long conversations, WebSocket mode for the Responses API, and LightOnOCR-3 image chat on CUDA, providing reasons to examine its API integration and conversation resumption features.
View on GitHub