feat(plugins): Stage 2.1 — plugin framework and AI tool-use
Introduces a standalone plugin system where every integration lives as an independent Python script in ~/.pyra/plugins/, not hardcoded in core. Plugin framework (src/pyra/plugins/): - base.py: Tool dataclass, PyraPlugin Protocol, BasePlugin helper - loader.py: importlib-based discovery; one bad plugin never crashes pyra - registry.py: singleton aggregating tools, slash commands, system prompts - executor.py: approval gate — scans args, prompts y/N, scans result, logs - install.py: copies bundled_plugins/ to ~/.pyra/plugins/ on install Chat integration: - AI tool-use loop (litellm function calling, up to 10 iterations) - Plugin system prompt additions injected per session - Plugin slash commands merged with static commands CLI additions: - pyra plugin list/install/enable/disable/setup - pyra daemon start/stop/status/restart/install/uninstall (stubs for 2.4) Config: PluginConfig + DaemonConfig added to PyraConfig (backwards-compatible) Bootstrap: ~/.pyra/plugins/ and ~/.pyra/logs/ created on startup Security: tool args and results always injection-scanned; plugin dirs validated with assert_safe_path() before loading (symlink protection) Tests: 37 new tests (loader, registry, executor, plugin isolation security) 161 total, all passing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
curo1305
@@ -3,18 +3,61 @@
|
||||
## What Is This
|
||||
|
||||
Pyra is a personal AI assistant CLI combining a multi-provider AI chat interface with
|
||||
an automation/skills system (Stage 2+) and an encrypted vault (Stage 3+).
|
||||
a plugin/integration system (Stage 2+) and an encrypted vault (Stage 3+).
|
||||
|
||||
## Current Status
|
||||
|
||||
**Stage 2.1 — Plugin Framework: complete** (2026-05-17)
|
||||
Next: Stage 2.2 (Nextcloud plugin) + Stage 2.3 (Email plugin)
|
||||
|
||||
## Project Roadmap
|
||||
|
||||
### Stage 1 — Core CLI (current)
|
||||
### Stage 1 — Core CLI ✅ COMPLETE
|
||||
Working `pyra` executable with provider setup wizard, streaming chat REPL, .md-based
|
||||
memory in `~/.pyra/memory/`, and hard security boundaries around the vault.
|
||||
|
||||
### Stage 2 — Skills / Automations
|
||||
Shell (.sh), PowerShell (.ps1), and Python (.py) scripts in `~/.pyra/skills/`. The AI
|
||||
can suggest running a skill, but execution requires explicit user approval (y/n prompt).
|
||||
No skill can access the vault. Skills are discovered by the pyra CLI, not by the AI.
|
||||
### Stage 2 — Plugin System & Integrations (IN PROGRESS)
|
||||
|
||||
Pyra runs as a system daemon so messaging bots are always-on. All integrations are
|
||||
standalone Python plugin scripts in `~/.pyra/plugins/` — not hardcoded in `src/pyra/`.
|
||||
The AI uses tool-use (function calling) to invoke plugins; every execution requires
|
||||
explicit user approval (y/N prompt). Plugin credentials are stored in the vault under
|
||||
namespaced keys (`plugin:{name}:{key}`).
|
||||
|
||||
#### Stage 2.1 — Plugin Framework ✅ COMPLETE
|
||||
- `src/pyra/plugins/` package: `base.py`, `loader.py`, `registry.py`, `executor.py`, `install.py`
|
||||
- `src/pyra/bundled_plugins/` — ships bundled plugin scripts with pyra
|
||||
- `src/pyra/daemon/` stub (CLI surface only)
|
||||
- Config: `PluginConfig` + `DaemonConfig` added to `PyraConfig`
|
||||
- Bootstrap: `~/.pyra/plugins/` and `~/.pyra/logs/` created on startup
|
||||
- Chat session: AI tool-use loop (up to 10 iterations), approval gate, plugin slash commands
|
||||
- CLI: `pyra plugin list/install/enable/disable/setup`, `pyra daemon *` stubs
|
||||
|
||||
#### Stage 2.2 — Nextcloud Plugin (next)
|
||||
Bundled plugin: `src/pyra/bundled_plugins/nextcloud/plugin.py`
|
||||
CalDAV calendar, CardDAV contacts, WebDAV files.
|
||||
Install deps: `uv pip install -e ".[nextcloud]"`
|
||||
|
||||
#### Stage 2.3 — Email Plugin (next, parallel with 2.2)
|
||||
Bundled plugin: `src/pyra/bundled_plugins/email/plugin.py`
|
||||
IMAP (Hotmail.de/Outlook), folder browsing, smart event extraction to calendar.
|
||||
No new deps (uses stdlib imaplib).
|
||||
|
||||
#### Stage 2.4 — Daemon + Messaging Bots
|
||||
- `src/pyra/daemon/server.py` — asyncio event loop, plugin tasks, IPC socket
|
||||
- `src/pyra/daemon/ipc.py` — Unix socket (chmod 600), line-delimited JSON protocol
|
||||
- `src/pyra/daemon/service.py` — launchd plist (macOS) / systemd unit (Linux)
|
||||
- Bundled plugins: `matrix_bot`, `telegram_bot`, `signal_bot`
|
||||
- Security: sender allowlist, bcrypt passphrase challenge, rate limiting (20 msg/hr),
|
||||
injection scanning on all incoming messages, tool approval over messaging (2-min timeout)
|
||||
|
||||
#### Stage 2.5 — Infrastructure Plugins
|
||||
Bundled plugins: `ssh_tool` (paramiko), `docker_tool` (docker SDK),
|
||||
`kubernetes_tool` (kubectl subprocess), `service_manager` (systemctl/launchctl),
|
||||
`smb_mount` (mount subprocess)
|
||||
|
||||
#### Stage 2.6 — Cloud Storage Plugins
|
||||
Bundled plugins: `gdrive` (Google OAuth2), `onedrive` (MSAL device flow), `dropbox_tool`
|
||||
|
||||
### Stage 3 — Vault Encryption
|
||||
Encrypt `~/.pyra/vault/secrets/` using `age` (or GPG fallback). Pyra decrypts in memory
|
||||
@@ -29,7 +72,7 @@ Report written to `~/.pyra/security_audit.md` (not AI-readable during normal cha
|
||||
|
||||
### Stage 5 — Web UI / Advanced Features
|
||||
Optional local web interface (FastAPI + HTMX or similar). Embedding-based memory search
|
||||
(ChromaDB or sqlite-vec). Scheduled automations via cron-style skill scheduling.
|
||||
(ChromaDB or sqlite-vec). Scheduled automations via cron-style plugin scheduling.
|
||||
Multi-profile support (work vs personal).
|
||||
|
||||
---
|
||||
@@ -40,45 +83,107 @@ Multi-profile support (work vs personal).
|
||||
|
||||
| Module | Purpose |
|
||||
|--------|---------|
|
||||
| `cli.py` | Click entrypoint. Subcommands: `setup`, `chat`, `memory` |
|
||||
| `cli.py` | Click entrypoint. Subcommands: `setup`, `chat`, `memory`, `plugin`, `daemon` |
|
||||
| `setup/providers.py` | Provider registry — pure data, no I/O |
|
||||
| `setup/wizard.py` | questionary-based interactive setup wizard |
|
||||
| `config/schema.py` | Pydantic v2 models — no API keys, only `provider_id/model/base_url` |
|
||||
| `config/schema.py` | Pydantic v2 models — `PyraConfig`, `PluginConfig`, `DaemonConfig` |
|
||||
| `config/manager.py` | ruamel.yaml round-trip config read/write, chmod 600 enforced |
|
||||
| `config/dirs.py` | `bootstrap()` — creates `~/.pyra/` tree, checks vault sentinel every startup |
|
||||
| `chat/session.py` | prompt_toolkit REPL loop, slash commands, calls vault reader inline |
|
||||
| `chat/renderer.py` | Live streaming markdown via rich, injection warning panel, key redaction |
|
||||
| `chat/history.py` | Conversation list, token budget trimming, system prompt construction |
|
||||
| `chat/session.py` | prompt_toolkit REPL loop, AI tool-use loop, plugin slash commands |
|
||||
| `chat/renderer.py` | Streaming + non-streaming markdown via rich, injection warning panel |
|
||||
| `chat/history.py` | Conversation list, token budget trimming, tool message support |
|
||||
| `memory/reader.py` | `list_memories()`, `read_memory()`, `load_context_for_session()` |
|
||||
| `memory/writer.py` | `write_memory()`, `append_memory()` — relative names only, no traversal |
|
||||
| `memory/index.py` | Auto-regenerate `MEMORY_INDEX.md` on every write |
|
||||
| `vault/reader.py` | `get_key(provider_id)` — sole accessor of `vault/secrets/api_keys.json` |
|
||||
| `vault/writer.py` | `set_key()`, `delete_key()` — only called from setup wizard |
|
||||
| `vault/reader.py` | `get_key(key)` — sole accessor of `vault/secrets/api_keys.json` |
|
||||
| `vault/writer.py` | `set_key()`, `delete_key()` — only called from setup wizard + plugin setup |
|
||||
| `security/boundaries.py` | `assert_safe_path()`, `check_vault_lock()`, `BLOCKED_PREFIXES` |
|
||||
| `security/injection.py` | `scan_response()` — 15 regex patterns, 4 categories, logs to `security.log` |
|
||||
| `utils/paths.py` | `pyra_home()`, `ensure_dir()`, `safe_chmod()`, `expand()` |
|
||||
| `plugins/base.py` | `Tool` dataclass, `PyraPlugin` Protocol, `BasePlugin` helper class |
|
||||
| `plugins/loader.py` | Discovers + loads plugins via importlib; failures isolated per plugin |
|
||||
| `plugins/registry.py` | Singleton: aggregates tools, slash commands, system prompt additions |
|
||||
| `plugins/executor.py` | Approval gate: scan args → prompt → execute → scan result → log |
|
||||
| `plugins/install.py` | Copies bundled plugins to `~/.pyra/plugins/` |
|
||||
| `bundled_plugins/` | Standalone plugin scripts shipped with pyra (installed on demand) |
|
||||
| `daemon/__init__.py` | Daemon package stub (implementation in Stage 2.4) |
|
||||
|
||||
### Runtime: `~/.pyra/`
|
||||
|
||||
```
|
||||
~/.pyra/
|
||||
├── config.yaml chmod 600 ← provider_id, model, base_url ONLY
|
||||
├── config.yaml chmod 600 ← provider_id, model, base_url, enabled plugins
|
||||
├── security.log chmod 600 ← injection event log
|
||||
├── memory/ chmod 700
|
||||
│ ├── user/profile.md
|
||||
│ ├── context/
|
||||
│ ├── knowledge/
|
||||
│ └── MEMORY_INDEX.md
|
||||
├── skills/ chmod 700 ← Stage 2
|
||||
│ ├── bash/
|
||||
│ ├── powershell/
|
||||
│ └── python/
|
||||
├── plugins/ chmod 700 ← active plugins (each is a dir with manifest.json + plugin.py)
|
||||
│ └── <name>/
|
||||
│ ├── manifest.json
|
||||
│ └── plugin.py
|
||||
├── logs/ chmod 700
|
||||
│ ├── tool_executions.log chmod 600 ← every tool call: approved/declined, args, result preview
|
||||
│ └── plugin_errors.log chmod 600 ← plugin load failures
|
||||
└── vault/ chmod 700 ← AI CANNOT ACCESS
|
||||
├── .vault_lock chmod 400 ← sentinel; missing = refuse to start
|
||||
└── secrets/
|
||||
└── api_keys.json chmod 400 ← ALL API keys
|
||||
└── api_keys.json chmod 400 ← ALL secrets (AI keys + plugin credentials)
|
||||
```
|
||||
|
||||
### Plugin Credential Naming Convention
|
||||
|
||||
Plugin credentials live in the vault under namespaced keys:
|
||||
```
|
||||
plugin:{plugin_name}:{key_name}
|
||||
```
|
||||
Examples: `plugin:nextcloud:password`, `plugin:matrix_bot:access_token`
|
||||
|
||||
The vault's `get_key()` / `set_key()` accept any string — the namespace is enforced
|
||||
by convention in each plugin's `setup()` method.
|
||||
|
||||
### Writing a Plugin
|
||||
|
||||
1. Create `~/.pyra/plugins/<name>/manifest.json`:
|
||||
```json
|
||||
{"name": "<name>", "version": "1.0.0", "description": "...", "author": "you"}
|
||||
```
|
||||
2. Create `~/.pyra/plugins/<name>/plugin.py` exporting `get_plugin() -> BasePlugin`:
|
||||
```python
|
||||
from pyra.plugins.base import BasePlugin, Tool
|
||||
|
||||
class MyPlugin(BasePlugin):
|
||||
name = "<name>"
|
||||
description = "..."
|
||||
version = "1.0.0"
|
||||
|
||||
def on_load(self, vault_reader):
|
||||
self._secret = vault_reader("plugin:<name>:secret")
|
||||
|
||||
def tools(self):
|
||||
return [
|
||||
Tool("my_tool", "Does X", {"type": "object", "properties": {}},
|
||||
self._do_x, requires_approval=True)
|
||||
]
|
||||
|
||||
def _do_x(self):
|
||||
return "result"
|
||||
|
||||
def setup(self, console, vault_writer):
|
||||
secret = console.input("Enter secret: ")
|
||||
vault_writer("plugin:<name>:secret", secret)
|
||||
|
||||
def get_plugin():
|
||||
return MyPlugin()
|
||||
```
|
||||
3. `pyra plugin enable <name>`
|
||||
|
||||
**Plugin rules:**
|
||||
- Never import from `pyra.vault` directly — use the `vault_reader`/`vault_writer` callables
|
||||
- All write/destructive tools must set `requires_approval=True`
|
||||
- Return strings from tool handlers (truncated to 4000 chars by executor)
|
||||
|
||||
---
|
||||
|
||||
## Security Rules (never break these)
|
||||
@@ -86,9 +191,12 @@ Multi-profile support (work vs personal).
|
||||
1. **Never pass config file contents into a system prompt** — config may reveal provider/model
|
||||
2. **Never bypass `assert_safe_path()`** — not even in tests (use `tmp_pyra_home` fixture instead)
|
||||
3. **Always `chmod 600/400`** after writing any file in `~/.pyra/`
|
||||
4. **No shell execution from AI-generated text** — ever (Stage 2 uses explicit approval gates)
|
||||
5. **`vault/reader.py` and `vault/writer.py` are the only modules that import from `pyra.vault`**
|
||||
4. **No shell execution from AI-generated text** — plugins use explicit approval gates
|
||||
5. **`vault/reader.py` and `vault/writer.py` are the only modules that may open `api_keys.json`**
|
||||
6. **API key retrieved inline at call time** — never stored as an instance variable or logged
|
||||
7. **Tool arguments and results are always injection-scanned** before being used or returned to AI
|
||||
8. **Plugin directories are validated with `assert_safe_path()`** before loading (symlink protection)
|
||||
9. **Messaging bot security**: sender allowlist + bcrypt passphrase + rate limiting (Stage 2.4)
|
||||
|
||||
## Adding a New Provider
|
||||
|
||||
@@ -102,19 +210,17 @@ Add a test in `tests/unit/test_providers.py` to verify the new entry.
|
||||
uv venv && source .venv/bin/activate
|
||||
uv pip install -e ".[dev]"
|
||||
pyra setup
|
||||
```
|
||||
|
||||
Or with pip:
|
||||
```bash
|
||||
python -m venv .venv && source .venv/bin/activate
|
||||
pip install -e ".[dev]"
|
||||
pyra setup
|
||||
# Install optional plugin dependencies:
|
||||
uv pip install -e ".[nextcloud]" # Nextcloud plugin
|
||||
uv pip install -e ".[ssh]" # SSH plugin
|
||||
uv pip install -e ".[all-plugins]" # Everything
|
||||
```
|
||||
|
||||
## Running Tests
|
||||
|
||||
```bash
|
||||
pytest tests/ -v # all unit + security tests
|
||||
pytest tests/ -v # all unit + security tests (161 tests)
|
||||
pytest tests/integration/test_lmstudio.py # requires LM Studio at localhost:1234
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user