151 lines
4.1 KiB
Markdown
151 lines
4.1 KiB
Markdown
# Vector Explorer
|
|
|
|
`vector_explorer` is a Rust + GTK desktop viewer for SQLite-backed vector stores. It is designed around OpenClaw's memory layout and can fall back to a generic adapter for similar databases such as NullClaw- or ZeroClaw-style stores that keep chunk text and embeddings in SQLite tables.
|
|
|
|
## Features
|
|
|
|
- Store overview with detected adapter, file count, chunk count, embedding dimensions, models, and inferred backend features such as FTS5 or `sqlite-vec`
|
|
- Filterable file list with a top-level reset entry to return to the full graph
|
|
- Rotatable 3D semantic map based on a lightweight PCA-style projection of embeddings
|
|
- Detail panel for the selected chunk or file
|
|
- OpenClaw-first schema detection with generic SQLite heuristics for other stores
|
|
|
|
## Supported store layouts
|
|
|
|
### OpenClaw
|
|
|
|
The first-class adapter targets the documented OpenClaw memory structure:
|
|
|
|
- `files` for indexed documents
|
|
- `chunks` for chunk text and stored embeddings
|
|
- `meta` for memory index metadata
|
|
- `chunks_fts` and `chunks_vec` as optional hybrid-search acceleration tables
|
|
|
|
The sample `main.sqlite` in this repository matches that layout.
|
|
|
|
### Generic SQLite vector stores
|
|
|
|
If the database is not recognized as OpenClaw, the app falls back to schema heuristics:
|
|
|
|
- Find a table with text/content columns
|
|
- Prefer tables that also expose `embedding` or `vector`
|
|
- Reconstruct file groupings from a path-like column or from the chunk table itself
|
|
|
|
That makes the viewer usable for adjacent SQLite-backed vector stores such as NullClaw- or ZeroClaw-style databases, as long as they expose chunk text and embeddings in a reasonably conventional schema.
|
|
|
|
## Rust dependencies
|
|
|
|
These are declared in [Cargo.toml](C:/Users/Trude/Desktop/vector_explorer/Cargo.toml):
|
|
|
|
- `anyhow`
|
|
- `chrono`
|
|
- `gtk4`
|
|
- `rfd`
|
|
- `rusqlite` with `bundled`
|
|
- `serde`
|
|
- `serde_json`
|
|
|
|
## System dependencies
|
|
|
|
### Required
|
|
|
|
- Rust toolchain with `cargo`
|
|
- GTK 4 development files and runtime
|
|
- `pkg-config`
|
|
- C/C++ build tooling appropriate for your platform
|
|
|
|
### WSL / Ubuntu packages
|
|
|
|
If you are building in WSL on an Ubuntu-like distro, install:
|
|
|
|
```bash
|
|
sudo apt update
|
|
sudo apt install -y build-essential pkg-config libgtk-4-dev
|
|
```
|
|
|
|
### Windows native build
|
|
|
|
If you are building natively on Windows instead of WSL, you need:
|
|
|
|
- Visual Studio Build Tools with the MSVC C++ toolchain
|
|
- GTK 4 SDK/runtime
|
|
- `pkg-config` configured to find the GTK installation
|
|
|
|
The project compiles more reliably in WSL than in a partially configured native Windows environment.
|
|
|
|
## How to compile
|
|
|
|
### WSL
|
|
|
|
```bash
|
|
cd /mnt/c/Users/Trude/Desktop/vector_explorer
|
|
cargo build
|
|
```
|
|
|
|
For an optimized build:
|
|
|
|
```bash
|
|
cd /mnt/c/Users/Trude/Desktop/vector_explorer
|
|
cargo build --release
|
|
```
|
|
|
|
### Windows native
|
|
|
|
```powershell
|
|
cd C:\Users\Trude\Desktop\vector_explorer
|
|
cargo build
|
|
```
|
|
|
|
For an optimized build:
|
|
|
|
```powershell
|
|
cd C:\Users\Trude\Desktop\vector_explorer
|
|
cargo build --release
|
|
```
|
|
|
|
## How to run
|
|
|
|
### WSL
|
|
|
|
```bash
|
|
cd /mnt/c/Users/Trude/Desktop/vector_explorer
|
|
cargo run -- main.sqlite
|
|
```
|
|
|
|
To open a different database:
|
|
|
|
```bash
|
|
cd /mnt/c/Users/Trude/Desktop/vector_explorer
|
|
cargo run -- /path/to/other.sqlite
|
|
```
|
|
|
|
### Windows native
|
|
|
|
```powershell
|
|
cd C:\Users\Trude\Desktop\vector_explorer
|
|
cargo run -- .\main.sqlite
|
|
```
|
|
|
|
## Notes for WSL graphics
|
|
|
|
- The app is GTK-based and needs a working GUI/display path in WSL.
|
|
- If WSLg or X/Wayland forwarding is not available, the binary can compile successfully but fail to open a display at runtime.
|
|
- The app forces `GSK_RENDERER=cairo` by default to avoid flaky EGL/Zink GPU paths commonly seen on WSL.
|
|
|
|
## Development workflow
|
|
|
|
Useful commands:
|
|
|
|
```bash
|
|
cargo fmt
|
|
cargo check
|
|
cargo build
|
|
```
|
|
|
|
## Repository contents
|
|
|
|
- [src/main.rs](C:/Users/Trude/Desktop/vector_explorer/src/main.rs): GTK application and UI behavior
|
|
- [src/store.rs](C:/Users/Trude/Desktop/vector_explorer/src/store.rs): SQLite schema detection and adapter loading
|
|
- [src/projection.rs](C:/Users/Trude/Desktop/vector_explorer/src/projection.rs): embedding projection for the semantic map
|
|
- [main.sqlite](C:/Users/Trude/Desktop/vector_explorer/main.sqlite): sample OpenClaw-style memory database
|