How To Run LLMs Locally?

You can run any LLM(obviously open source) locally. Today’s newsletter breaks down how to run large language models like DeepSeek and Llama using Ollama, Docker, and OpenUI.

I added content, news, and resources about deep-dive LLMs, DeepSeek locally, Sam Altman’s 3 AI observations, 10x Engineer by Andrew Ng, and more.

In today’s edition:

• AI Roundup— deep dive into LLMs by Andrej Karpathy

• Dive Deep Drill— how to run LLMs locally

• Build Together— here’s how I can help you

AI Roundup

I found these resources, content, and news this week.

— [news] AI will make it more vertical.

— [content] Sam Altman’s 3 AI Observations

— [resource] AI Agents and RAG tutorials on GitHub.
— [content] 10x Engineer by Andrew Ng
— [resource] Mastering AI Agents.

Dive Deep Drill

This has recently become a common question: running LLMs locally.

You and I are covering 2 questions today:

• how to run LLMs locally

• how to build an AI agent to browse the web

How To Run LLMs Locally

Follow these steps on Mac, for windows I added the link to the documentation.

These are the steps we are going to follow:

1. Installing Ollama

2. Downloading any models on the local system

3. Install Docker

4. Install Open WebUI for a User Interface

5. Install Web-UI for an AI agent to Browse the Web

1. Installing Ollama

Ollama is an open-source AI engine that helps you run models locally.

Run the following command on your Mac:

brew install ollama

You can also download it directly from the website here.

Once done, check the version, it will show you 0.5.7 

ollama --version

Now you can download any model on your local system.

Select the model you want to install locally from the website.

2. Downloading any models on the local system

For now, let’s download deepseek-r1, there are different versions available you can download the smaller one as you are using your laptop.

I downloaded the 8b version which is 4.9GB

ollama pull deepseek-r1:8b

You can check all the models installed in your system by this command:

ollama list

Run the model:

ollama run deepseek-r1:8b

You can interact with DeepSeek on the terminal.

Now we want it to have a User Interface like ChatGPT.

3. Install Docker

Docker is an open-source platform that uses containers to package software and its dependencies.

Think of it as a folder that contains all the files, libraries, and dependencies of your project.

Go to this website and download docker.

I selected the Apple silicon version.

You can select the Intel version depending on your laptop.

Once it is downloaded, install it and you will get an interface like this:

4. Install Open WebUI for a User Interface

Open WebUI is an AI interface by which you do not need to interact with models through the terminal.

It’s open source, check GitHub.

Run the following command:

pip install open-webui

As Ollama is already on your computer, use this command to run open webUI on docker localhost

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

It will look like this, pretty cool no?

Build an AI Agent to Browse the Web

5. Install Web-UI for an AI agent to Browse the Web

Now if you want to run an AI agent to browse the internet, you need Web-UI.

This open-source will help you run AI Agent in your browser.

To install it clone the GitHub repo and get into the web-ui directory(folder).

git clone https://github.com/browser-use/web-ui.git
cd web-ui

Now all you need to do is run this command and you can access the interface on the localhost on a different port.

docker compose up --build

The localhost will look like this:

You can select the LLM configuration to DeepSeek and don’t need any API key because we already installed it locally.

Now you can go to the Run Agent section:

When you click on the Run agent after giving the prompt, it will automatically execute the task you have given to it.

One last thing, you can check the docker desktop app, in which all the containers running:

Cheers, you have done it!

Want to work together? Here’s How I can help you

• AI Engineering & Consulting (B2B) at Dextar—[Request a Brainstorm]

• You are a leader?—join the [AI leadership academy]

• AI Training for Enterprise Team—[MasterDexter]

• Get in front of 50k+ AI leaders & professionals—[Sponsor this Newsletter]

I use BeeHiiv to send this newsletter.

Paper Unfold

A series in which you will get a breakdown of complex research papers into easy-to-understand pointers. If you missed the previous ones:

• AI Agents for Computer Use [Paper Unfold]

• Future Of AI - Large Concept Models [Paper Unfold]

• DeepSeek-R1 - A Step Towards AGI [Paper Unfold]

• Agents are Not Enough [Paper Unfold]

• Attention Is All You Need [Paper Unfold]

Question for You

Which industry will need private LLMs the most?