Local Model Verifier

What this does

Answer three questions and this produces a small set of files. One of them blocks the model from reaching the internet. Another one checks whether that actually worked, which is the part most guides leave out.

Every file explains itself and tells you how to undo it. You do not need to know systemd to use them.

Run this after applying the settings below. It checks two things: that the model still answers, and that it cannot reach the internet. Both have to pass.

#!/usr/bin/env bash
# Checks whether the isolation applied to ollama.service actually works.
#
# 3 checks, all of which have to pass. They test different things on
# purpose, because each one alone can pass while you are still exposed:
#
#   Is the service still working?      A lockdown that breaks the service looks
#                                      the same from outside as one that works.
#   Is the policy on THIS service?     Proving the kernel can block egress says
#                                      nothing about whether your service got it.
#   Does blocking work here?           A kernel can accept the setting and then
#                                      not enforce it.
#
# Run it with:  bash verify-isolation.sh
#
# TO UNDO everything this applied:
#   sudo rm /etc/systemd/system/ollama.service.d/isolation.conf
#   sudo systemctl daemon-reload
#   sudo systemctl restart ollama.service
#
# Exit codes, for running this in CI: 0 all checks passed, 1 the lockdown is not
# in force, 2 the service is broken.
set -uo pipefail

echo "Isolation check for ollama.service"
echo "==============================================="
echo

n=1
echo "Check $n of 3: is the service still working?"
echo "  Asking it for its model list."
if ! curl -sf localhost:11434/api/tags >/dev/null 2>&1; then
  echo "  FAIL: the service did not answer."
  echo
  echo "  What this means: the lockdown broke the service instead of protecting"
  echo "  it. The usual cause is ReadWritePaths pointing at the wrong directory,"
  echo "  so the service cannot write where it keeps its models."
  echo
  echo "  Look at the error with:"
  echo "    systemctl status ollama.service"
  echo
  echo "  TO UNDO everything this applied:"
  echo "    sudo rm /etc/systemd/system/ollama.service.d/isolation.conf"
  echo "    sudo systemctl daemon-reload"
  echo "    sudo systemctl restart ollama.service"
  exit 2
fi
echo "  PASS: the service answered."
echo

# Advisory, not a check. Running as root does not defeat ProtectSystem=strict,
# so this is not a failure and must not exit non-zero on a working setup. It is
# reported because it changes what a compromised runtime could reach, and a
# reader deciding how much to trust this result should know.
SVC_USER=$(systemctl show -p User --value ollama.service 2>/dev/null)
if [ -z "$SVC_USER" ] || [ "$SVC_USER" = "root" ]; then
  echo "  NOTE: this service runs as root."
  echo "  The settings above still apply, and root inside the unit cannot edit"
  echo "  /etc while ProtectSystem=strict is in force. But if a malicious model"
  echo "  file achieves code execution while loading, it starts from a stronger"
  echo "  position. Running the service as its own user is worth doing."
  echo
fi

n=2
echo "Check $n of 3: is the lockdown actually attached to this service?"
echo "  Asking systemd what policy ollama.service is running under."
DENY=$(systemctl show -p IPAddressDeny --value ollama.service 2>/dev/null)
if [ -z "$DENY" ]; then
  echo "  FAIL: systemd reports no address policy on ollama.service."
  echo
  echo "  What this means: the file is not being read. Either it is in the"
  echo "  wrong place, or systemd was not told to reload after you added it."
  echo
  echo "  Check what systemd thinks the unit looks like:"
  echo "    systemctl cat ollama.service"
  echo "  The contents of isolation.conf should appear at the bottom. If not:"
  echo "    sudo systemctl daemon-reload"
  echo "    sudo systemctl restart ollama.service"
  exit 1
fi
echo "  PASS: systemd is enforcing an address policy on this service."
echo

n=$((n + 1))
echo "Check $n of 3: does blocking actually work on this kernel?"
echo "  Running a throwaway service under the same policy and trying to reach"
echo "  a website. This catches a kernel that accepts the setting and ignores it."
if systemd-run --pty --collect --quiet \
     --property=IPAddressDeny=any \
     --property=IPAddressAllow=localhost \
     curl -s --max-time 5 https://example.com >/dev/null 2>&1; then
  echo "  FAIL: the connection succeeded."
  echo
  echo "  What this means: systemd accepted the policy but your kernel is not"
  echo "  enforcing it. This needs CONFIG_CGROUP_BPF, which most distributions"
  echo "  enable. Use the firewall rules instead of the settings file."
  exit 1
fi
echo "  PASS: the connection was refused, which is what we want."
echo

echo "RESULT: isolation holds, and the service still works."
echo
echo "This says nothing about whether the model itself is trustworthy. It says"
echo "the model cannot send anything anywhere."
exit 0

Install this at /etc/systemd/system/ollama.service.d/isolation.conf. The file tells you the exact commands, and how to undo it.

# Isolation for Ollama (ollama.service)
#
# This file stops the model from reaching the network. It does not stop it
# working: you still talk to it normally on this machine.
#
# TO INSTALL:
#   sudo mkdir -p /etc/systemd/system/ollama.service.d
#   sudo cp isolation.conf /etc/systemd/system/ollama.service.d/isolation.conf
#   sudo systemctl daemon-reload
#   sudo systemctl restart ollama.service
#   bash verify-isolation.sh
#
# TO UNDO everything this applied:
#   sudo rm /etc/systemd/system/ollama.service.d/isolation.conf
#   sudo systemctl daemon-reload
#   sudo systemctl restart ollama.service
#
# THIS FILE ASSUMES your model files live in /var/lib/ollama.
# If they do not, change ReadWritePaths below or the service will not start.
#
# NOT SET: MemoryDenyWriteExecute
#   Most systemd hardening guides tell you to turn this on. Do not.
#   It breaks CUDA, which JIT compiles kernels at runtime.
#   With it set, the model falls back to the CPU or fails to start.

[Service]
# Block all network access for this service...
IPAddressDeny=any
# ...except to this machine itself. This is how you talk to the model.
IPAddressAllow=localhost
# Stop the service gaining extra privileges if something inside it is exploited.
NoNewPrivileges=yes
# Make the whole filesystem read-only for this service, apart from the
# exception listed below.
ProtectSystem=strict
# Let it read your home directory but never write to it.
ProtectHome=read-only
# The one place it is allowed to write: where it keeps model files.
ReadWritePaths=/var/lib/ollama
# Allow only ordinary local and internet sockets, IPv4 and IPv6. Blocks the
# exotic kinds a service like this has no reason to use. AF_INET6 is included
# deliberately: leaving it out blocks IPv6 outright, which makes an IPv6
# IPAddressAllow above a dead letter and breaks any service listening on ::1.
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6