mirror of
https://github.com/lone-cloud/gerbil
synced 2026-06-03 19:54:44 -07:00
prettify the README, misc minor changes
This commit is contained in:
parent
d3ff055ca6
commit
35be08bf59
4 changed files with 64 additions and 44 deletions
76
README.md
76
README.md
|
|
@ -1,15 +1,29 @@
|
||||||
# Gerbil <img src="src/assets/icon.png" alt="Gerbil Icon" width="32" height="32" />
|
<div align="center">
|
||||||
|
|
||||||
A desktop app to easily run Large Language Models locally.
|
<img src="src/assets/icon.png" alt="Gerbil Icon" width="80" height="80" />
|
||||||
|
|
||||||
|
# Gerbil
|
||||||
|
|
||||||
|
**The simplest way to run Large Language Models on your own hardware**
|
||||||
|
|
||||||
|
[](https://www.gnu.org/licenses/agpl-3.0)
|
||||||
|
[](https://github.com/lone-cloud/gerbil/releases)
|
||||||
|
[](https://github.com/lone-cloud/gerbil/stargazers)
|
||||||
|
[](https://aur.archlinux.org/packages/gerbil)
|
||||||
|
|
||||||
|
[Download](https://github.com/lone-cloud/gerbil/releases/latest) • [Features](#features) • [Screenshots](#demo--screenshots) • [Installation](#installation)
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
<!-- markdownlint-enable MD033 -->
|
<!-- markdownlint-enable MD033 -->
|
||||||
|
|
||||||
## Core Features
|
Gerbil provides a graphical interface for running Large Language Models locally. It handles the technical complexity of managing backends, model downloads, and hardware acceleration - letting you focus on using AI rather than configuring it.
|
||||||
|
|
||||||
- **Run LLMs locally** - Powered by [KoboldCpp](https://github.com/LostRuins/koboldcpp), a feature-rich fork of [llama.cpp](https://github.com/ggml-org/llama.cpp)
|
## Features
|
||||||
|
|
||||||
|
- **Run LLMs locally** - Powered by [KoboldCpp](https://github.com/LostRuins/koboldcpp), a fork of [llama.cpp](https://github.com/ggml-org/llama.cpp)
|
||||||
- **Cross-platform** - Native support for Windows, macOS, and Linux (including Wayland)
|
- **Cross-platform** - Native support for Windows, macOS, and Linux (including Wayland)
|
||||||
- **Offline capable** - Import pre-downloaded KoboldCpp binaries and run entirely without an internet connection
|
- **Offline capable** - Import pre-downloaded KoboldCpp binaries and run entirely without an internet connection
|
||||||
- **Automatic backend updates** - Download and keep your KoboldCpp binary up-to-date effortlessly
|
|
||||||
- **Flexible hardware support** - Works on CPU-only systems while also supporting GPU acceleration (CUDA, ROCm, Vulkan, CLBlast, Metal)
|
- **Flexible hardware support** - Works on CPU-only systems while also supporting GPU acceleration (CUDA, ROCm, Vulkan, CLBlast, Metal)
|
||||||
- **Image generation** - Built-in presets for Flux, Chroma, Qwen Image, and Z-Image workflows
|
- **Image generation** - Built-in presets for Flux, Chroma, Qwen Image, and Z-Image workflows
|
||||||
- **Integrated HuggingFace search** - Browse models, view model cards, and download GGUF files directly from the app
|
- **Integrated HuggingFace search** - Browse models, view model cards, and download GGUF files directly from the app
|
||||||
|
|
@ -17,20 +31,26 @@ A desktop app to easily run Large Language Models locally.
|
||||||
- **OpenWebUI integration** - Launch OpenWebUI for a modern web-based chat interface (requires [uv](https://docs.astral.sh/uv/getting-started/installation/))
|
- **OpenWebUI integration** - Launch OpenWebUI for a modern web-based chat interface (requires [uv](https://docs.astral.sh/uv/getting-started/installation/))
|
||||||
- **Privacy-focused** - Everything runs locally with no external data transmission or telemetry
|
- **Privacy-focused** - Everything runs locally with no external data transmission or telemetry
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
1. **[Download Gerbil](https://github.com/lone-cloud/gerbil/releases/latest)** for your platform
|
||||||
|
2. **Launch the app** - No installation needed for portable versions
|
||||||
|
3. **Download a model** - Use the default model, use the built-in HuggingFace search by clicking on the looking glass icon, or import your own
|
||||||
|
4. **Start generating** - Text and image generation is supported
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
### Pre-built Binaries
|
### Pre-built Binaries
|
||||||
|
|
||||||
Download the latest release for your platform from the [GitHub Releases page](https://github.com/lone-cloud/gerbil/releases/latest):
|
Download the latest release from the [GitHub Releases page](https://github.com/lone-cloud/gerbil/releases/latest):
|
||||||
|
|
||||||
- **Windows**: `Gerbil-Portable-X.X.X.exe` (portable executable)
|
- **Windows**: `Gerbil-Portable-X.X.X.exe` or `Gerbil-Setup-X.X.X.exe`
|
||||||
- **Windows**: `Gerbil-Setup-X.X.X.exe` (installer executable)
|
- **macOS**: `Gerbil-X.X.X.dmg`
|
||||||
- **macOS**: `Gerbil-X.X.X.dmg` (disk image)
|
- **Linux**: `Gerbil-X.X.X.AppImage`
|
||||||
- **Linux**: `Gerbil-X.X.X.AppImage` (portable application)
|
|
||||||
|
|
||||||
#### Linux - AUR (Arch Linux)
|
### Linux - AUR (Arch Linux)
|
||||||
|
|
||||||
For Arch Linux users, install from the AUR using your preferred AUR helper:
|
**Recommended for Arch users** - Automatic updates through your package manager:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Using yay
|
# Using yay
|
||||||
|
|
@ -45,9 +65,9 @@ cd gerbil
|
||||||
makepkg -si
|
makepkg -si
|
||||||
```
|
```
|
||||||
|
|
||||||
The AUR package automatically handles installation, desktop integration, and system updates. This is the ideal way to run Gerbil on Linux.
|
The AUR package handles desktop integration and keeps Gerbil updated with your system.
|
||||||
|
|
||||||
#### Open a Mac app from an unknown developer
|
### macOS Security Note
|
||||||
|
|
||||||
After installing on macOS, you will notice that the app will not run as it was not verified by Apple. You will need to [follow these instructions](https://support.apple.com/en-ca/guide/mac-help/mh40616/mac) to remove it from your system's quarantine to be able to run it. Alternatively you can run `xattr -r -d com.apple.quarantine /Applications/Gerbil.app` in the terminal.
|
After installing on macOS, you will notice that the app will not run as it was not verified by Apple. You will need to [follow these instructions](https://support.apple.com/en-ca/guide/mac-help/mh40616/mac) to remove it from your system's quarantine to be able to run it. Alternatively you can run `xattr -r -d com.apple.quarantine /Applications/Gerbil.app` in the terminal.
|
||||||
|
|
||||||
|
|
@ -110,20 +130,16 @@ https://github.com/user-attachments/assets/9e7ecfb3-3576-443c-8cef-a14e06ab5b60
|
||||||
|
|
||||||
<!-- markdownlint-enable MD033 MD022 -->
|
<!-- markdownlint-enable MD033 MD022 -->
|
||||||
|
|
||||||
### Future features
|
|
||||||
|
|
||||||
Not all koboldcpp features have currently been ported over to the UI. As a workaround one may use the "Additional arguments" on the "Advanced" tab of the launcher to provide additional command line arguments if you know them.
|
|
||||||
|
|
||||||
## CLI Mode
|
## CLI Mode
|
||||||
|
|
||||||
The `--cli` argument allows you to use the Gerbil binary as a proxy to the downloaded KoboldCpp binary. This enables you to run KoboldCpp from the command line using the same binary that the GUI has downloaded.
|
The `--cli` argument allows you to run Gerbil in the terminal without the UI. This will run the same backend that the GUI was using.
|
||||||
|
|
||||||
### CLI Arguments
|
### CLI Arguments
|
||||||
|
|
||||||
Gerbil supports the following command-line arguments:
|
Gerbil supports the following command-line arguments:
|
||||||
|
|
||||||
- `--version` - Display the Gerbil version and exit
|
- `--version` - Display the Gerbil version and exit
|
||||||
- `--cli` - Run in CLI mode (proxy to KoboldCpp binary). All arguments after `--cli` are passed to KoboldCpp
|
- `--cli` - Run in CLI mode
|
||||||
- `--minimized` - Start Gerbil GUI minimized to system tray (requires system tray to be enabled in settings)
|
- `--minimized` - Start Gerbil GUI minimized to system tray (requires system tray to be enabled in settings)
|
||||||
|
|
||||||
### Considerations
|
### Considerations
|
||||||
|
|
@ -135,14 +151,14 @@ You might want to run CLI Mode if you're looking to use a different frontend, su
|
||||||
**Linux/macOS:**
|
**Linux/macOS:**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Basic usage - launch KoboldCpp launcher with no arguments
|
# Basic usage - launch the KoboldCpp launcher with no arguments
|
||||||
gerbil --cli
|
gerbil --cli
|
||||||
|
|
||||||
# Pass arguments to KoboldCpp
|
# Pass arguments to the backend
|
||||||
gerbil --cli --help
|
gerbil --cli --help
|
||||||
gerbil --cli --port 5001 --model /path/to/model.gguf
|
gerbil --cli --port 5001 --model /path/to/model.gguf
|
||||||
|
|
||||||
# Any KoboldCpp arguments are supported
|
# Any backend arguments are supported
|
||||||
gerbil --cli --model /path/to/model.gguf --port 5001 --host 0.0.0.0 --multiuser 2
|
gerbil --cli --model /path/to/model.gguf --port 5001 --host 0.0.0.0 --multiuser 2
|
||||||
|
|
||||||
# CLI inception (Gerbil CLI calling KoboldCpp CLI mode)
|
# CLI inception (Gerbil CLI calling KoboldCpp CLI mode)
|
||||||
|
|
@ -154,7 +170,7 @@ gerbil --cli --cli --model /path/to/model.gguf --gpulayers 57 --contextsize 8192
|
||||||
|
|
||||||
CLI mode will only work correctly on Windows if you install Gerbil using the Setup.exe from the github releases. Otherwise there is currently a technical limitation with the Windows portable .exe which will cause it to not display the terminal output correctly nor will it be killable through the standard terminal (Ctrl+C) commands.
|
CLI mode will only work correctly on Windows if you install Gerbil using the Setup.exe from the github releases. Otherwise there is currently a technical limitation with the Windows portable .exe which will cause it to not display the terminal output correctly nor will it be killable through the standard terminal (Ctrl+C) commands.
|
||||||
|
|
||||||
You can use the CLI mode on Windows in exactly the same way as in the Linux/macOS examples above, except you'll be calling the "Gerbil.exe". Note that it will not be on your system PATH by default, so you'll need to manually specify the full path to it when callig it from the Windows terminal.
|
You can use the CLI mode on Windows in exactly the same way as in the Linux/macOS examples above, except you'll be calling the "Gerbil.exe". Note that it will not be on your system PATH by default, so you'll need to manually specify the full path to it when calling it from the Windows terminal.
|
||||||
|
|
||||||
## Local Dev
|
## Local Dev
|
||||||
|
|
||||||
|
|
@ -194,14 +210,18 @@ You can use the CLI mode on Windows in exactly the same way as in the Linux/macO
|
||||||
|
|
||||||
- OS: [Archlinux](https://archlinux.org/) + KDE + Wayland
|
- OS: [Archlinux](https://archlinux.org/) + KDE + Wayland
|
||||||
- Gerbil: installed from [AUR](https://aur.archlinux.org/packages/gerbil) via [yay](https://github.com/Jguer/yay)
|
- Gerbil: installed from [AUR](https://aur.archlinux.org/packages/gerbil) via [yay](https://github.com/Jguer/yay)
|
||||||
- KoboldCpp: nocuda build, vulkan backend
|
- Backend: nocuda running the Vulkan acceleration
|
||||||
- Hardware: RX 7900 GRE (16GB VRAM, undervolted, overclocked), Ryzen 7 7800X3D, ASUS B650-E, 32GB RAM (EXPO 1: 6000Mhz)
|
- Hardware: RX 7900 GRE (16GB VRAM, undervolted, overclocked), Ryzen 7 7800X3D, ASUS B650-E, 32GB RAM (EXPO 1: 6000Mhz)
|
||||||
- LLM: [Q4 Gemma 3 27b abliterated](https://huggingface.co/mlabonne/gemma-3-27b-it-abliterated-GGUF?show_file_info=gemma-3-27b-it-abliterated.q4_k_m.gguf)+ [Open WebUI](https://openwebui.com/) + [Dolphin system message](https://github.com/QuixiAI/dolphin-system-messages)
|
- LLM: [Q4 Gemma 3 27b abliterated](https://huggingface.co/mlabonne/gemma-3-27b-it-abliterated-GGUF?show_file_info=gemma-3-27b-it-abliterated.q4_k_m.gguf)+ [Open WebUI](https://openwebui.com/) + [Dolphin system message](https://github.com/QuixiAI/dolphin-system-messages)
|
||||||
|
|
||||||
## Known Issues
|
## Known Issues
|
||||||
|
|
||||||
- Windows ROCm support is... problematic and currently requires for the user to manually add the installed ROCm bin directory to the system PATH. In particular "hipInfo.exe" must be present, which is not always the case for older versions of ROCm.
|
- **Windows ROCm**: Requires manual PATH configuration. The ROCm bin directory (containing `hipInfo.exe`) must be added to your system PATH.
|
||||||
|
|
||||||
## License
|
## Support This Project
|
||||||
|
|
||||||
AGPL v3 License - see LICENSE file for details
|
If Gerbil makes your life easier, consider:
|
||||||
|
|
||||||
|
- **Starring this repo** - Helps others discover the project
|
||||||
|
- **Spreading the word** - Share with friends who want to try local AI
|
||||||
|
- **Reporting bugs** - Makes Gerbil better for everyone
|
||||||
|
|
|
||||||
|
|
@ -49,7 +49,7 @@
|
||||||
"@uiw/react-codemirror": "^4.25.3",
|
"@uiw/react-codemirror": "^4.25.3",
|
||||||
"electron-updater": "^6.6.2",
|
"electron-updater": "^6.6.2",
|
||||||
"execa": "^9.6.1",
|
"execa": "^9.6.1",
|
||||||
"lucide-react": "^0.556.0",
|
"lucide-react": "^0.559.0",
|
||||||
"mime-types": "^3.0.2",
|
"mime-types": "^3.0.2",
|
||||||
"react": "^19.2.1",
|
"react": "^19.2.1",
|
||||||
"react-dom": "^19.2.1",
|
"react-dom": "^19.2.1",
|
||||||
|
|
@ -67,7 +67,7 @@
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"@eslint/js": "^9.39.1",
|
"@eslint/js": "^9.39.1",
|
||||||
"@types/mime-types": "^3.0.1",
|
"@types/mime-types": "^3.0.1",
|
||||||
"@types/node": "^24.10.2",
|
"@types/node": "^25.0.0",
|
||||||
"@types/react": "^19.2.7",
|
"@types/react": "^19.2.7",
|
||||||
"@types/react-dom": "^19.2.3",
|
"@types/react-dom": "^19.2.3",
|
||||||
"@types/yauzl": "^2.10.3",
|
"@types/yauzl": "^2.10.3",
|
||||||
|
|
|
||||||
|
|
@ -39,7 +39,7 @@ export const Modal = ({
|
||||||
const content = tallContent ? (
|
const content = tallContent ? (
|
||||||
<div
|
<div
|
||||||
style={{
|
style={{
|
||||||
height: '66vh',
|
height: '75vh',
|
||||||
padding: 0,
|
padding: 0,
|
||||||
display: 'flex',
|
display: 'flex',
|
||||||
flexDirection: 'column',
|
flexDirection: 'column',
|
||||||
|
|
|
||||||
26
yarn.lock
26
yarn.lock
|
|
@ -1401,12 +1401,12 @@ __metadata:
|
||||||
languageName: node
|
languageName: node
|
||||||
linkType: hard
|
linkType: hard
|
||||||
|
|
||||||
"@types/node@npm:*, @types/node@npm:^24.10.2":
|
"@types/node@npm:*, @types/node@npm:^25.0.0":
|
||||||
version: 24.10.2
|
version: 25.0.0
|
||||||
resolution: "@types/node@npm:24.10.2"
|
resolution: "@types/node@npm:25.0.0"
|
||||||
dependencies:
|
dependencies:
|
||||||
undici-types: "npm:~7.16.0"
|
undici-types: "npm:~7.16.0"
|
||||||
checksum: 10c0/560c894e1a9bf7468718ceca8cd520361fd0d3fcc0b020c2f028fc722b28b5b56aecd16736a9b753d52a14837c066cf23480a8582ead59adc63a7e4333bc976c
|
checksum: 10c0/5a10d6f873ad326e46fac923ea4305b7fea3de0cf7e11c835d1fdc99d4b4d0f3f108035432ec1732f505c97d367b9abf618ecbdfb02a75a5306665ecce2a81a6
|
||||||
languageName: node
|
languageName: node
|
||||||
linkType: hard
|
linkType: hard
|
||||||
|
|
||||||
|
|
@ -2039,11 +2039,11 @@ __metadata:
|
||||||
linkType: hard
|
linkType: hard
|
||||||
|
|
||||||
"baseline-browser-mapping@npm:^2.8.9":
|
"baseline-browser-mapping@npm:^2.8.9":
|
||||||
version: 2.8.16
|
version: 2.9.6
|
||||||
resolution: "baseline-browser-mapping@npm:2.8.16"
|
resolution: "baseline-browser-mapping@npm:2.9.6"
|
||||||
bin:
|
bin:
|
||||||
baseline-browser-mapping: dist/cli.js
|
baseline-browser-mapping: dist/cli.js
|
||||||
checksum: 10c0/e38a861f44236cfc5ba8bf9d4a024d37682e73efc3df3377e170a8a44303f466c0fce7fdc641029df27e47adb270b2f7cf29550b9be7dcdbeec18bc62858c9e9
|
checksum: 10c0/9d9bae544342c02200e0a489d4caf3611a2a517d0d7a8c97e1dd7d640ac18d2784710bf08375b5951fb9034bd18d902b50797d90218a33b88e6244ae917f881f
|
||||||
languageName: node
|
languageName: node
|
||||||
linkType: hard
|
linkType: hard
|
||||||
|
|
||||||
|
|
@ -3860,7 +3860,7 @@ __metadata:
|
||||||
"@mantine/core": "npm:^8.3.10"
|
"@mantine/core": "npm:^8.3.10"
|
||||||
"@mantine/hooks": "npm:^8.3.10"
|
"@mantine/hooks": "npm:^8.3.10"
|
||||||
"@types/mime-types": "npm:^3.0.1"
|
"@types/mime-types": "npm:^3.0.1"
|
||||||
"@types/node": "npm:^24.10.2"
|
"@types/node": "npm:^25.0.0"
|
||||||
"@types/react": "npm:^19.2.7"
|
"@types/react": "npm:^19.2.7"
|
||||||
"@types/react-dom": "npm:^19.2.3"
|
"@types/react-dom": "npm:^19.2.3"
|
||||||
"@types/yauzl": "npm:^2.10.3"
|
"@types/yauzl": "npm:^2.10.3"
|
||||||
|
|
@ -3883,7 +3883,7 @@ __metadata:
|
||||||
execa: "npm:^9.6.1"
|
execa: "npm:^9.6.1"
|
||||||
globals: "npm:^16.5.0"
|
globals: "npm:^16.5.0"
|
||||||
jiti: "npm:^2.6.1"
|
jiti: "npm:^2.6.1"
|
||||||
lucide-react: "npm:^0.556.0"
|
lucide-react: "npm:^0.559.0"
|
||||||
mime-types: "npm:^3.0.2"
|
mime-types: "npm:^3.0.2"
|
||||||
prettier: "npm:^3.7.4"
|
prettier: "npm:^3.7.4"
|
||||||
react: "npm:^19.2.1"
|
react: "npm:^19.2.1"
|
||||||
|
|
@ -5116,12 +5116,12 @@ __metadata:
|
||||||
languageName: node
|
languageName: node
|
||||||
linkType: hard
|
linkType: hard
|
||||||
|
|
||||||
"lucide-react@npm:^0.556.0":
|
"lucide-react@npm:^0.559.0":
|
||||||
version: 0.556.0
|
version: 0.559.0
|
||||||
resolution: "lucide-react@npm:0.556.0"
|
resolution: "lucide-react@npm:0.559.0"
|
||||||
peerDependencies:
|
peerDependencies:
|
||||||
react: ^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0
|
react: ^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0
|
||||||
checksum: 10c0/5c73bd317b2cd7d220082d1cc6f78c864819652be87a84c976f2b08aae0bf257b0f066f86f5d84c2cb838ba8fbd2fb1d2b9d84fbe025d9c6cb97dbf1cdbf6bbc
|
checksum: 10c0/0fcab0197949cda2873770c29a8374ca237133555b9ae8cb3987892681b10511728f2ee558eeb09a05cfbe9bef824e2382be7c7d68facacb243e082ad2d6643d
|
||||||
languageName: node
|
languageName: node
|
||||||
linkType: hard
|
linkType: hard
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue