Filter by type:
Filter by tags:
Attention: Trust Remote Code is required for this model
wemm-embedding-2b
WeMM-Embedding-2B is Tencent's Apache-2.0 multilingual embedding model
built on Qwen3.5. This entry serves the original bfloat16 safetensors with
LocalAI's Transformers backend and produces 2,048-dimensional normalized
embeddings for text retrieval, semantic search, and RAG.
The upstream model can also embed images and videos. LocalAI currently
exposes text input through its embeddings API for this backend.
Repository: localaiLicense: apache-2.0
Attention: Trust Remote Code is required for this model
wemm-embedding-4b
WeMM-Embedding-4B is Tencent's mid-sized Apache-2.0 multilingual embedding
model built on Qwen3.5. This entry serves the original bfloat16 safetensors
with LocalAI's Transformers backend and produces 2,560-dimensional
normalized embeddings for text retrieval, semantic search, and RAG.
The upstream model can also embed images and videos. LocalAI currently
exposes text input through its embeddings API for this backend.
Repository: localaiLicense: apache-2.0
Attention: Trust Remote Code is required for this model
wemm-embedding-9b
WeMM-Embedding-9B is Tencent's largest Apache-2.0 multilingual embedding
model built on Qwen3.5. This entry serves the original bfloat16 safetensors
with LocalAI's Transformers backend and produces 4,096-dimensional
normalized embeddings for text retrieval, semantic search, and RAG.
The upstream model can also embed images and videos. LocalAI currently
exposes text input through its embeddings API for this backend.
Repository: localaiLicense: apache-2.0
dfm-mimir:vllm
DFM Mimir is an Apache-2.0, instruction-tuned HRM-Text model from Danish
Foundation Models. It has about 1 billion parameters and a 4,096-token
context window. The model focuses on Danish and English chat, reasoning,
mathematics, and code generation, and uses only permissible post-training
data. This entry serves the official BF16 safetensors checkpoint with vLLM.
Repository: localaiLicense: apache-2.0
carnice-v2-27b
# Carnice-V2-27B for Hermes Agent
Carnice-V2-27B is a full merged BF16 SFT of `Qwen/Qwen3.6-27B` for Hermes-style agent traces. This repository contains the standalone merged model weights, not only a LoRA adapter.
## BF16 Transformers Loading Fix
The BF16 safetensors were republished with corrected `Qwen3_5ForConditionalGeneration` tensor prefixes. The original merge artifact accidentally serialized an extra Unsloth wrapper prefix, which caused direct HF Transformers loads to report the real weights as unexpected keys and initialize expected layers randomly. GGUF files were not affected because the GGUF conversion path normalized those prefixes.
## Benchmarks
The benchmark artifact bundle is included under `benchmarks/`. It contains the rendered graph, extracted `metrics.json`, benchmark scripts, and raw result files used to make the chart.
Scope note: the IFEval run is a short `limit=20` A/B smoke benchmark, not an official full leaderboard score. Held-out loss/perplexity is the exact assistant-only training-format validation metric from the SFT script. The raw BFCL two-case smoke files are included for auditability, but they are too small to use as a model-quality claim.
...
Repository: localaiLicense: apache-2.0
qwen3.8-27b-exl3-vllm-cpp
Qwen3.8-27B EXL3 3.5bpw served by vllm.cpp, LocalAI's C++ vLLM-style
runtime. The published checkpoint generates on CUDA and measured 16.7
tokens/s on GB10 with the pinned revision and the limits configured here.
This is the target-only setup. Use the DFlash2 variant for the measured
speculative-decoding configuration. The entry downloads the complete
revision-pinned repository, including its configuration, tokenizer, index,
and safetensors shards.
Repository: localai
granite-4.2-3b:vllm
Granite 4.2 3B is IBM's compact dense reasoning model for code generation,
tool calling, agentic workflows, multilingual chat, and long-context tasks.
This entry serves the bfloat16 safetensors with vLLM and supports a
128K-token context. It is the smallest fallback in a family that also offers
the higher-capacity 8B and 30B checkpoints as variants.
Repository: localaiLicense: apache-2.0
granite-4.2-8b:vllm
Granite 4.2 8B is IBM's mid-sized dense reasoning model for code generation,
tool calling, agentic workflows, multilingual chat, and long-context tasks.
This entry serves the higher-capacity bfloat16 safetensors with vLLM and
supports a 128K-token context.
Repository: localaiLicense: apache-2.0
granite-4.2-30b:vllm
Granite 4.2 30B is IBM's largest dense Granite 4.2 reasoning model for code
generation, tool calling, agentic workflows, multilingual chat, and
long-context tasks. This entry serves the bfloat16 safetensors with vLLM and
supports a 128K-token context.
Repository: localaiLicense: apache-2.0
Attention: Trust Remote Code is required for this model
phonellm-alpha-1:vllm
PhoneLLM Alpha 1 is a 30B-parameter, 3.5B-active mixture-of-experts model
tuned by the Pipecat team for low-latency voice agents, multi-turn
conversations, and accurate tool use without thinking. This entry serves
the bfloat16 safetensors with vLLM and supports a 262K-token context.
PhoneLLM is released under BSD-2-Clause and derives from NVIDIA Nemotron 3
Nano 30B-A3B. The underlying work remains subject to the NVIDIA Nemotron
Open Model License and its redistribution and attribution requirements.
Repository: localaiLicense: bsd-2-clause
Attention: Trust Remote Code is required for this model
phonellm-alpha-1:sglang
PhoneLLM Alpha 1 is a 30B-parameter, 3.5B-active mixture-of-experts model
tuned by the Pipecat team for low-latency voice agents, multi-turn
conversations, and accurate tool use without thinking. This entry serves
the bfloat16 safetensors with SGLang and supports a 262K-token context.
PhoneLLM is released under BSD-2-Clause and derives from NVIDIA Nemotron 3
Nano 30B-A3B. The underlying work remains subject to the NVIDIA Nemotron
Open Model License and its redistribution and attribution requirements.
Repository: localaiLicense: bsd-2-clause
ltx-2.3-22b-dev-ggml
LTX-2.3 22B dev - DiT-based audio-video foundation model from Lightricks,
GGUF-quantized for the stable-diffusion.cpp backend. Generates synchronized
video and audio from a text prompt (T2V), a reference image (I2V), or
first/last frame pairs (FLF2V). Uses gemma-3-12b-it as the text encoder
and ships dedicated video and audio VAEs plus an embeddings_connectors
safetensors that bridges the LLM hidden states to the diffusion model.
This entry uses the dynamic (UD) Q4_K_M quantization of the 22B model
(~16 GB) paired with the UD-Q4_K_XL QAT Gemma encoder (~7.4 GB).
Recommended generation: width=1280, height=720, video_frames=33,
fps=24, sampler=euler, cfg_scale=6.0.
Repository: localaiLicense: ltx-2-community-license-agreement