OLLAMA on Arch Linux: A Guide to CPU/GPU-Centric Computing

How2's
1

In recent years, the need for high-performance computing has become increasingly important in various fields, including scientific research, data analysis, and machine learning. One key component of this effort is the utilization of Graphics Processing Units (GPUs) for general-purpose computing tasks. OLLAMA (OpenLP ARchitectures), a GPU-centric operating system, offers an alternative to traditional x86-based systems for accelerating workloads on NVIDIA GPUs.

Why Install OLLAMA?

OLLAMA provides several benefits over traditional Linux distributions:

  • Native Support: OLLAMA natively supports the latest NVIDIA drivers and features, ensuring optimal performance and compatibility with cutting-edge GPUs.
  • GPU-Centric Design: The operating system is specifically designed for GPU-based computing, providing optimized hardware abstraction layers and a streamlined user interface.
  • Customization: OLLAMA’s modular design allows users to easily integrate custom hardware support, extend the software stack, or modify existing components.

Installing OLLAMA on Arch Linux

To install OLLAMA on Arch Linux, follow these steps:

  1. Download the latest version of OLLAMA manually
curl -fsSL https://ollama.com/install.sh | sh
  1. Be aware when you install it by Version 1.) your ollama never got updated. Here comes Arch.

IF you have no GPU and want run ollama by your CPU

pacman -S ollama

IF you have a already a nvidia system

make sure you have nvidia-cuda already installed by:

pacman -S nvidia-cuda

followed by

pacman -S ollama-cuda
  1. Configure your system by editing the ~/.config/olllama.conf file, adding the following configuration option to enable CUDA support:
[archlinux]
cuda_support = true

Post-Installation Steps

After installing and configuring OLLAMA, perform these tasks to ensure optimal performance:

  • Update Your System: Run sudo pacman -Syyu to update all installed packages.
  • Install Essential Tools: Use sudo pacman -S base-devel for basic system tools like git, curl, make, and others.

By following this guide, you can successfully install OLLAMA on Arch Linux and leverage its GPU-centric features for accelerated computing tasks. Running a LLM on GPU is factor 1000 faster compared to CPU.

By default ollama run on localhost only. Allow ollama listening on all local interfaces, you can follow these steps:

  1. If you’re running Ollama directly from the command line, use the
    OLLAMA_HOST=0.0.0.0 ollama serve command to specify that it should listen on all local interfaces

  2. Edit the service file: Open /usr/lib/systemd/system/ollama.service and add the following line inside the [Service] section:

Environment="OLLAMA_HOST=0.0.0.0"

Once you’ve made your changes, reload the daemons using the command

sudo systemctl daemon-reload
and then restart the service with
sudo systemctl restart ollama