Home Lab Ep. 1: Building a Self-Hosted LLM with Ollama and LibreChat

In this episode, I’m setting up a private, self-hosted AI stack: a local LLM runner (Ollama) paired with a full-featured chat interface (LibreChat). By the end, I had a working ChatGPT-style web app running entirely inside my lab, talking to models that live on my own hardware.

The Architecture

The setup has two main pieces.

1. The Inference Engine: Ollama

Ollama is the local model runner. It manages my open LLMs and serves them for inference. It doesn’t ship with an interactive chat web interface, but it does expose a local HTTP REST API on port 11434. You can poke at it directly with curl:

curl http://10.10.10.100:11434/v1/models

2. The Chat Interface: LibreChat

LibreChat is a full-featured, multi-provider AI client and management stack. Think of it as your own self-hosted ChatGPT or Claude web app. It’s made up of several parts:

Web Frontend: A modern chat UI for entering prompts, rendering Markdown/LaTeX/code blocks, and selecting models.
Backend API & Orchestration: Manages user authentication, presets, prompt templates, conversation branching, and tool use/RAG.
Data Persistence: Uses MongoDB to store users, conversation threads, and settings.
Fast Full-Text Search: Uses Meilisearch to index and instantly search past conversations.
Multi-Provider Hub: While connected to my local Ollama instance for offline inference, LibreChat can also connect to external APIs (OpenAI, Anthropic, OpenRouter, Google) whenever I want, all from one unified UI.

Setting Up LibreChat

Clone the repository, then pull the container images and start the services:

# Clone the LibreChat repository
git clone https://github.com/danny-avila/LibreChat.git
cd LibreChat

# Pull container images and run the services
docker compose up -d

Connecting LibreChat to Ollama

I configured LibreChat to talk directly to Ollama’s REST API using librechat.yaml:

version: 1.3.17
cache: true

endpoints:
  custom:
    - name: "Ollama"
      apiKey: "ollama"
      baseURL: "http://X.X.X.X:11434/v1"
      models:
        default: [ "MODEL_NAME" ]
        fetch: true
      titleConvo: true
      titleModel: "MODEL_NAME"
      modelDisplayLabel: "Ollama"

Then I bind-mounted the config file into the container using docker-compose.override.yml, naming the service api. That way, whenever I need to reload the configuration, I only have to restart that one service:

services:
  api:
    volumes:
    - type: bind
      source: ./librechat.yaml
      target: /app/librechat.yaml

Configuring Authentication

Since this is a safe, local environment, I created a single local admin user for now. I’ll replace it with a proper, controlled IAM architecture later.

First, I generated cryptographic secrets. LibreChat uses them to sign session tokens (JWT) and encrypt user credentials in MongoDB. Setting explicit, persistent keys also prevents sessions from being invalidated every time the container restarts.

# Run 3 times: JWT secret / JWT refresh secret / credential encryption key
openssl rand -hex 32

# For the credential IV
openssl rand -hex 16

Then I added everything to the .env file:

# --- Application Domain ---
DOMAIN_CLIENT=http://X.X.X.X:3080
DOMAIN_SERVER=http://X.X.X.X:3080

# --- Authentication & Session Secrets ---
JWT_SECRET=<JWT_SECRET>
JWT_REFRESH_SECRET=<JWT_REFRESH_SECRET>
CREDS_KEY=<CREDENTIAL_KEY>
CREDS_IV=<CREDENTIAL_IV>

# --- Authentication Controls ---
ALLOW_EMAIL_LOGIN=true
ALLOW_REGISTRATION=true
ALLOW_SOCIAL_LOGIN=false
ALLOW_SOCIAL_REGISTRATION=false

# --- Security Hardening ---
# Enforces TOTP 2FA for all local accounts (turned off for now)
ENFORCE_TWO_FACTOR_AUTHENTICATION=false

Registering the Admin Account

After troubleshooting Proxmox processor mismatches and some missing variables in .env, I was finally able to register an admin account. I connected through a Kali host in my local lab environment. And just like that, I have a working LLM linked to a chat interface under a registered account.

Troubleshooting Cheat Sheet

A few commands that saved me along the way:

# Make sure Ollama is on the LibreChat network
docker network connect librechat_default ollama

# Restart the API container to load endpoint configurations
docker compose restart api

# View logs
docker compose logs -f

# Update LibreChat
git pull
docker compose pull
docker compose up -d

What’s Next

Private LLM Hardening

Internal DNS: Point librechat.yaml at an internal DNS name for the Ollama REST API instead of a raw IP.
Choose an authentication model: Local email/password, LDAP/Active Directory, or OAuth2/social logins.
Enforce 2FA: Flip on TOTP for all local accounts.
Wazuh integration: Link authentication and logging prompts into Wazuh for monitoring.