Ollama

Raspberry Pi

The most popular single-board computer for lightweight self-hosting: a Raspberry Pi 5 runs Pi-hole, Home Assistant, WireGuard, and small local models around the clock for a few dollars a month in electricity, with no monthly hosting bill.

Paid

Hosting & Cloud

3/5

Ollama can run on a Raspberry Pi 5 with enough RAM to serve small open-weight models locally, a common lightweight self-hosted AI setup for developers who already run other services on a Pi.

Unsloth

Open-source desktop app, web UI, and Python library for running and fine-tuning open-weight models locally. It serves models through an OpenAI-compatible API and trains them about twice as fast with roughly 70% less memory than standard pipelines.

Freemium

Data & ML Libraries

4/5

Unsloth exports fine-tuned models to GGUF for local serving in Ollama, a common last step after training.

DeepSeek

Chinese open-weight LLM family under the MIT license, led by DeepSeek-V4.1-Flash, offering frontier-level results at very low API prices, with full self-hosting through vLLM, SGLang, and Ollama.

Freemium

LLM

4/5

Ollama can pull and run DeepSeek's open-weight models locally with a single command, exposing them through its OpenAI-compatible API.

Xiaomi MiMo

Xiaomi's open-weight, MIT-licensed LLM family, led by the omnimodal MiMo-V2.6-Pro (1.02T MoE, 42B active), which ranks among the strongest open-weight models for agentic and coding work at a low per-token price.

Freemium

LLM

4/5

MiMo runs locally through Ollama via community GGUF quantizations, no cloud API required.

GLM

GLM is Z.ai's (formerly Zhipu AI) family of bilingual Chinese and English LLMs, with open weights on Hugging Face, an OpenAI-compatible API with free Flash models, and a flat-rate GLM Coding Plan for coding agents.

Freemium

LLM

4/5

Ollama can pull and run GLM's open-weight models locally with a single command, exposing them through its OpenAI-compatible API.

Qwen

Alibaba's LLM family, from sub-1B models to a 2.4-trillion-parameter flagship, covering reasoning, coding, vision, and long context, mostly under Apache 2.0 open weights with a hosted API on Alibaba Cloud.

Freemium

LLM

4/5

Ollama can pull and run Qwen's open-weight models locally with a single command, exposing them through its OpenAI-compatible API.

Langflow

An open-source, low-code visual builder for AI agents and RAG pipelines — drag-and-drop components on a canvas, model- and vendor-agnostic, with every flow exportable as an API or Python code. One of the most-starred AI projects on GitHub.

Open Source

Agentic AI

4/5

Langflow includes an Ollama component for wiring locally served open models into a flow.

Dify

Open-source platform for building AI workflows, RAG chatbots, and agents on a visual canvas, then publishing them as web apps or APIs. Self-hosted with Docker Compose or used as the managed Dify Cloud.

Freemium

Agentic AI

4/5

Dify's official Ollama model provider plugin connects a self-hosted Dify to models served locally by Ollama, so workflows, knowledge bases, and agents run without a hosted model API.

LlamaIndex

Open-source framework for building retrieval-augmented and document-centric AI agents in Python and TypeScript. Its companion LlamaCloud service adds hosted document parsing, extraction, indexing, and agent deployment.

Freemium

Agentic AI

3/5

LlamaIndex's llama-index-llms-ollama package points its query engines and agents at models served locally by Ollama, for RAG that stays on your own hardware.

AutoGPT

The project that popularized autonomous AI agents in 2023, now a platform of ready-made AI Experts coordinated by a chat agent, with a visual workflow builder and a community marketplace. Available as a hosted service or self-hosted.

Freemium

Agentic AI

3/5

AutoGPT includes Ollama as a built-in model provider, so a self-hosted install can run workflow AI blocks on locally served open models with no model API key.

LM Studio

Desktop app for downloading and running open-weight language models locally on macOS, Windows, and Linux, with a chat interface and a local OpenAI-compatible API server. It is free for personal and work use.

Freemium

AI Infrastructure

5/5

LM Studio leads with a graphical app for finding, downloading, and chatting with local models; Ollama is open source and CLI- and server-first, which suits scripts and headless machines.

llama.cpp

Open-source C/C++ engine that runs GGUF language and vision models on CPUs, Apple Silicon, and consumer GPUs. Its llama-server binary serves them through an OpenAI-compatible API with a built-in web UI.

Open Source

AI Infrastructure

5/5

Ollama adds a model library, a pull command, and a managed background server on top of the same GGML engine family; llama.cpp is the engine itself, with every quantization, sampling, and GPU-offload setting exposed. Pick Ollama for convenience, llama.cpp for control and the newest model support.

vLLM

Open-source, high-throughput inference and serving engine for large language models, exposing an OpenAI-compatible API. It is the most common way to self-host open-weight models in production on GPU and accelerator clusters.

Open Source

AI Infrastructure

4/5

vLLM is a datacenter serving engine that batches many concurrent requests across GPUs; Ollama runs a model on a laptop or single server with one command. Use Ollama for local and small-scale use, vLLM once many users hit the same model.

Groq

Fast LLM inference on Groq's own LPU hardware: unlike aggregators such as OpenRouter, Groq runs the compute itself, with low time-to-first-token and per-token pricing among the lowest of the major providers.

Usage Based

AI Infrastructure

4/5

Groq serves open models on its own LPU hardware as a hosted API with very low latency; Ollama runs open-weight models locally on your own machine. Groq for fast hosted inference, Ollama for private, offline, free local use.