Havuz

Fastest MoE/LLM inference runtime for consumer and edge Blackwell GPUs. SN74 on Gittensor.

Reproducible on-device LLM benchmarks for Apple Silicon (iPhone 17 Pro, M4 Max): Apple Core AI, MLX, llama.cpp, LiteRT-LM and Core ML on the

Run Kimi K3 (2.8 T parameters,1.56 TB) on iPhone 16 Pro? Weights stream from your SSD through a memory budget you set. Free, local, on Mac a

edge-dit.cpp — a native C/C++ inference engine for Diffusion Transformers (DiT), designed for local and resource-constrained devices with au

Hands-free on-device voice loop for macOS: Apple SFSpeechRecognizer + cloned-voice TTS, continuous listening, zero cloud. Originally built t

On-device shell command generator for macOS Tahoe. Uses Apple's 3B model with dynamic few-shot retrieval from 21k tldr examples.

Android SDK and MCP server: 157 tools for calendar, contacts, SMS, files, camera, sensors, notifications and UI automation, for on-device LL

3D scanner running fully on-device: 8 photos to a spinnable gaussian-splat scan in ~3s on an iPhone, via Depth Anything 3 + react-native-exe

Custom converter: open-weight Hugging Face LLMs/VLMs -> on-device LiteRT-LM (.litertlm), with one-command reproduction

Fast WebGPU runtime for 1-bit (binary-weight) LLMs in the browser. Bit-exact, zero runtime dependencies.

Free, open-source Mac dictation built for accessibility. Press a key, speak, and it types in any app. Whisper and AI cleanup run fully on yo

[Archived] AAOSP, predecessor to JibarOS. Early experiment in OS-level on-device AI (MCP + local LLM on Android 15). See JibarOS for the act

Android 16 fork. AI as a platform primitive. Twelve capabilities, one shared runtime, every app. OEM-pluggable. Apache 2.0.

A local-first macOS app to catalog, organize, and clone the open-source repos you actually use. No account, no cloud.

Run GPT-OSS-120B on a 16 GB Mac. Streams mixture-of-experts weights from SSD so only the 2 GiB trunk stays resident. Swift + Metal, no depen

Pi agent for Android — sideload the APK and run a Pi coding agent on your phone. Single-APK, on-device Termux runtime, Claude Pro/Max OAuth,

Qwen3.8-27B on one laptop CPU: up to 2.52 token/s, 8 GB tested, no runtime accuracy loss. Native OpenAI-compatible API with function tools;

Private on-device voice assistant — STT, LLM, TTS fully local. Android + desktop from one core, built on Tether QVAC SDK.

CubicLM is a unified cross-platform AI workspace with local GPU acceleration (GGUF, LiteRT-LM). Supports on-device image generation, 23+ clo

P2P semantic matching for on-device AI agents. No server, no cloud.

Use many on device and cloud models with Apple's Foundation Models framework in iOS and macOS with a full chat experience.

Tiny, Apache-2.0 wake-word detection you train yourself and run on-device in Python, the browser, and on mobile.

AI agents and workflows for Kotlin Multiplatform, as small typed graphs: loops, parallel steps, streaming, save points and human approval. I

Native iOS DeepSeek Harness port: BYOK inference, on-device Agent tools, iSH sandbox, Cordis plugins, and trajectory.

A ternary, zero-heap tiny language model that runs inside a $2 microcontroller — bit-exact Python <-> C99 <-> Cortex-M3 (QEMU) parity. Apach

A private Claude-Code-style coding agent for Apple Silicon — run chat, code, and local model workflows on-device. MLX-native, Ollama/OpenAI

A practical lab for exploring Apple's Core AI framework, model assets, specialization, and on-device inference.

Run a 28.9M-parameter TinyLM on ESP32-S3 with an RP2040 OLED display node for fully local embedded AI inference.

Free, open-source AI coding agent & multi-agent teams for macOS — runs 100% on-device via Ollama or LM Studio. Autovisor, an autonomous Supe

Translate and rewrite without switching apps. Free, open-source, on-device writing for Mac. No account or API key.

Provider-neutral bounded agent runtime for Android/JVM: controlled context (trust/priority/budgets), profile-scoped tools, OpenAI-compatible

Mobile-native agent SDK for building on-device AI agents with sandboxed execution, extensible tools, and x-app / x-agent / x-channel connect

Native AI assistant for macOS, built with SwiftUI and llama.cpp. Run local LLMs offline, work with PDFs, and create custom AI assistants. Op

Fast, private, on-device dictation for Apple silicon Macs

100% offline voice dictation for macOS, Windows & Linux — Whisper & Parakeet on-device, no cloud, open source.

Run the native 284B-A13B DeepSeek-V4-Flash-0731 LLM locally on one laptop CPU: pure C, 8 GB RAM minimum, no GPU, best TPOT 0.892 s/token, re

The first native Swift inference engine for Gemma 4 on Apple Silicon / iOS

🧠 One LLM, split across a Mac (Apple MPS) + a Windows PC (NVIDIA CUDA) — heterogeneous pipeline inference over a framework-neutral wire, bit

A local-first second brain for scientists, researchers, coders, and nerds.

web-ai-sdk: TypeScript SDK for the Web AI surface (Prompt, Writer, Rewriter, Proofreader, Translator, Summarizer, Language Detector, WebMCP)

On-device streaming speech-to-text for macOS, built on Qwen3-ASR via MLX

Native macOS image generator for FLUX.2 Klein — fully on-device on Apple Silicon via MLX. No Python, no ComfyUI, no server.

A free, fully local, minimal-setup voice dictation app for macOS - built specifically for developers.

SDK for building on-device AI applications on Arm with reusable building blocks across inference runtimes.

Faraday, the on-device AI voice agent for Android. Gemma 4 runs entirely on your phone: real voice conversation, long-term memory, and tools

Framework to talk to AI models and intelligently route

Your own voice + text AI assistant, embedded with one script tag, running 100% in the visitor's browser.

Run stock llama.cpp with the stock Unsloth Qwen3.6-35B-A3B 4-bit on a 16 GB Mac.

Offline LLM app for iPhone: run open models from Hugging Face on-device with llama.cpp, MLX, Core ML and Apple Foundation Models. No account

Tidytalk：Mac 本地语音输入工具。按住说话，松开即可得到整理后的文本。语音识别完全本机运行，可搭配 Ollama 本地大模型润色转写内容，数据不会离开你的电脑。免费开源。| Private on-device voice input for Mac. Hold to s