Pi 5 with Large Language Models (LLMs)

 

Last Update: 2/17/2025

Video: Setup DeepSeek / TinyLlama LLMs LOCALLY on a Raspberry Pi 5

This page is for testing AI Large Language Models (LLM’s) on a Pi 5. It provides a step-by-step tutorial on updating the Pi, installing the Ollama LLM software, and interacting with the model through terminal commands. Additionally, the page offers example prompts to demonstrate practical applications of LLMs on the Raspberry Pi 5 platform.

Q&A

A close up of the menu bar with buttons.
  • What do I need to experiment with AI on a Raspberry Pi 5? At a minimum, you’ll need a Raspberry Pi 5 with 8GB of RAM (or more), a monitor/TV, microHDMI to HDMI cable, Pi Power Adapter, active/passive cooling, and a keyboard + mouse. You will also need Raspberry Pi OS Desktop (64-bit) installed to either a microSD, SSD or external USB drive. You can find more details with links on the Raspberry Pi 5 guide here.
  • Where can I find more models to try? You can find more models on the Ollama website.
  • Can you explain what an LLM is in an easy to understand way? Sure, Imagine you have a super-smart robot friend who has read millions of books, websites, and stories. This robot doesn’t have eyes or a brain like ours, but it’s really good at understanding words and figuring out how to use them. This robot is called an LLM, which stands for Large Language Model.
    • Here’s how it works:
      • You can ask the robot questions or tell it to write a story, and it will use all the things it has “read” to give you an answer or create something new.
      • It doesn’t really understand things like we do€”it’s more like a super advanced word puzzle solver. It looks at the words you give it and guesses what words should come next to make sense.
      • It’s like having a magical book that can talk back to you and help you with homework, tell jokes, or even make up cool adventures!
    • So, an LLM is like a super-smart word robot that helps people with writing, learning, and having fun with words!
  • Why would I want to use an LLM on a Pi 5? Running an LLM (Large Language Model) on a Raspberry Pi 5 can be a fun and educational project, but it’s important to know that the Raspberry Pi 5, while powerful for its size, has limited resources compared to a full-sized computer or server. Still, here are 5 reasons why you might want to run an LLM on a Raspberry Pi 5:
    • Learn About AI and Machine Learning – Running an LLM on a Raspberry Pi is a great way to learn how AI models work, how they process language, and how to optimize them for smaller devices. It’s like a hands-on science experiment for understanding cutting-edge technology!
    • Privacy and Offline Use – If you run an LLM on your Raspberry Pi, you don’t need to rely on the internet or big companies’ servers. This means your data stays private, and you can use the LLM even without an internet connection.
    • Custom Projects and Prototyping – A Raspberry Pi is perfect for building custom projects. You could create a voice assistant, a chatbot for your room, or even a smart device that answers questions or helps with tasks€”all powered by your own LLM.
    • Energy Efficiency – The Raspberry Pi 5 uses very little power compared to a regular computer or server. If you want to run an LLM in a way that’s eco-friendly or cost-effective, the Pi is a great choice.
    • Challenge and Fun – Running an LLM on a Raspberry Pi is a technical challenge because the Pi isn’t as powerful as a regular computer. If you enjoy problem-solving, optimizing the model to work on the Pi can be a rewarding and fun experience!
  • What is Ollama? Ollama is a lightweight and optimized framework for running Large Language Models (LLMs) locally on personal devices, including computers and single-board computers like the Raspberry Pi 5. It is designed to make deploying and interacting with AI models seamless without requiring cloud-based processing.
  • When was DeepSeek first released? DeepSeek released its first chatbot on January 10, 2025. The chatbot, DeepSeek-R1, was based on the DeepSeek-V3 model. DeepSeek is a Chinese AI startup that has released a number of models since 2023. 
  • Can I use the Raspberry Pi AI Kit with Ollama? While you can install it into your Pi 5, Ollama isn’t currently taking advantage of the the NPU. At present, you won’t see any boost in performance by using it (as of 2/15/2025).

Running LLM’s on the Raspberry Pi 5

IMPORTANT NOTE – You must be running a 64-bit operating system to run Ollama on the Pi 5. During my testing, I used Raspberry Pi OS (64-bit). To check which version of Pi OS you’re running, enter “uname -m” into the terminal and press ENTER. If it says aarch64, you’re good to go. If not and you need help setting it up, please see this video.

A close up of the menu bar with buttons.

Step 1 – Update the Pi

The first step is to make sure the Pi5 is up-to-date. Open a terminal (click the icon on the toolbar or press Ctrl+ALT+T at the same time). Then issue each command below, followed by ENTER.

  • sudo apt update
  • sudo apt upgrade

Step 2 – Install Ollama

  • To install Ollama, issue the following command:
    • curl -fsSL https://ollama.com/install.sh | sh
  • To check the installed version of Ollama, run this command:
    • ollama -v

Step 3 – Run the LLM

The first time you run a new LLM, it will download the model from the internet locally to your Pi5. This process will take some time, however once downloaded, the next time you issue the command it will be much faster as the model will have been downloaded to your Pi 5. Once downloaded, the model runs 100% locally, that is you can disconnect your WiFi or Ethernet cable and the LLM will work. Below are some models I’ve personally tested on the Pi 5 and the commands you’ll issue to run them.

  • Run the following command to TinyLlama (1.1 billion parameters / 637MB) – reasonably fast and useful.
    • ollama run tinyllama
  • Use the following command to run DeepSeek-R1 (1.5 billion parameters / 1.1GB) on the Pi 5 – reasonably fast:
    • ollama run deepseek-r1:1.5b
  • Issue the following command to run DeepSeek Model 7b (7 billion parameters / 4.7GB) on the Pi 5 but quite slow:
    • ollama run deepseek-r1:7b

*NOTE – DeepSeek models distilled from DeepSeek-R1 based on Llama and Qwen. More details here.

Step 4 – Ask the LLM a question

To ask the LLM a question, simply type your question and press ENTER. Running an LLM on the Pi 5 is a lot slower than you may be used to by running it from a a website. Please be patient while awaiting the response. The questions you ask it are called prompts, I have a few example prompts listed here if you want to try them.

Ollama LLM Commands

The following commands can be run once Ollama has been started with an LLM:

Command Description
/set Set session variables
/show Show model information
/load <model> Load a session or model
/save <model> Save your current session
/clear Clear session context
/bye Exit
/?, /help Help for a command
/? shortcuts Help for keyboard shortcuts

Exit Ollama

  • You can exit the LLM running in Ollama by:
    • Type: /bye
    • Press Ctrl+D

Ollama Terminal Commands

The following commands are run outside the LLM and from the terminal session.

List Installed Models

  • To list all installed models, enter the following command:
    • ollama list

Remove Model

  • To remove a model from your Pi, issue the following command for the models installed (see List Installed Models to see what is installed first):
    • ollama rm deepseek-r1:1.5b
    • ollama rm deepseek-r1:7b
    • ollama rm tinyllama:latest

Remove Ollama

To remove Ollama from your Pi, issue each of the following commands in the terminal and press ENTER:

  1. sudo systemctl stop ollama
  2. sudo systemctl disable ollama
  3. sudo rm /etc/systemd/system/ollama.service

Example Prompts

A prompt is basically a way of communicating with an AI model to generate code, content or responses based on your input. Below are a few examples I used while testing (Copy+paste after running the LLM):

Prompt
What programming languages do you understand?
what is your favorite programming language?
Explain what an AI LLM is to a 12 year old.
Write a brief story about a dog and a cat and make it funny.
what is the distance from earth to mars?
what is 10+5*10?

Create a number Guessing Game

Prompt: Provide the program itself with no commentary. Write a python program to allow a user to enter a number from 1-100. The program picks a random number between 1-100. The user enters a number between 1 and 100 to try to guess the number the computer picked. The user has 5 guesses to pick the number the computer chose. The computer responds with higher or lower or answers if the correct guess was picked. If the user guesses 5 times and doesn’t pick the correct number, the program says sorry you lost, displays the number the computer guessed and exits the program. If the user picks the correct number, the computer says congratulations, you picked the right number and states the number of guesses taken to guess the correct number, then exits.

  • Copy the resulting program from the response.
  • Then, under Programming Thonny, paste the program into the editor. Note – if the program has errors, it’s likely due to line endings. Make sure that longer lines of text all appear on one line.
  • Click the Run button to run the python program.

Change Log

  • 2025-02-17 – Prepare for public release.
  • 2025-02-16 – Continuation.
  • 2025-02-13 – Start to this guide.