x/platform
Watch
1
0
Fork
You've already forked platform
 0
Code Issues Pull requests Projects Releases 24 Packages Wiki Activity Actions
24 releases 24 tags
  • v0.12.0 615bc52393

    v0.12.0 Stable

    x released this 2026-10-22 00:30:07 +00:00 | 4 commits to main since this release

    VSKI v0.12.0

    Plugins: supervised Python beside the server (experimental)

    Every real deployment grows a system around VSKI: an embeddings indexer,
    a record-to-graph sync, a crawler, a chat bot. Until now those lived as
    undocumented sidecars - cron entries, hand-rolled daemons, HTTP triggers
    pointing at localhost - with no shared lifecycle, no configuration story
    and no observability. v0.12.0 makes that pattern first-class: plugins are
    Python processes supervised by the server, configured and observed through
    the admin API and dashboard, running in the same container with zero
    network hops.

    The design stance matters as much as the feature:

    • Plugins are reusable source, not packages. Think shadcn components:
      you copy one into your distribution, verify its dependencies against
      the rest of your environment, tune its config schema, and build one
      image that just works. The server never installs or manages Python
      dependencies - the official images ship no Python at all, and
      distributions that want plugins bring their own locked interpreter
      (the wiki has the Dockerfile snippet).
    • An instance is data. Plugin + database + JSON config + enabled
      flag. Changing an API key or cloning a bot for another chat group is a
      PATCH or a POST that restarts exactly one process - never the server,
      never the container.
    • Not a sandbox. Plugin code ships inside your image and is trusted,
      exactly like workflow workers. There is no upload endpoint; code
      arrives via the filesystem only.

    Plugins are experimental and disabled by default
    (PLUGINS_ENABLED=true to switch on; when off, every plugin endpoint
    answers 503 with setup instructions).

    How it works

    A plugin is a directory under PLUGINS_DIR (default
    DATA_DIR/plugins) containing a vski-plugin.json manifest (name,
    runtime entry, config schema with secret fields, declared deps as
    metadata) and a Python package. The supervisor spawns
    $PLUGIN_PYTHON -m vski.plugin.runner inside it - PLUGIN_PYTHON must
    have vski-sdk>=0.5.0 (and the plugin's dependencies) installed by the
    distribution.

    • Control plane: protocol frames on the process's stdin/stdout
      (initialize / ping / invoke / shutdown), logs on stderr. A warm
      invocation is a pipe write - no socket, no cold start.
    • Health: pinged every 10s; an unresponsive process is killed and
      restarted. Crashes restart with exponential backoff (1s doubling to
      60s, reset after 5 minutes of uptime); the restarts counter tracks
      crash restarts. kill -9 recovery is covered by the e2e suite.
    • Credentials: each process start mints a fresh API key with the
      plugin role and deletes it when the process exits - key material
      exists only in server memory and the child environment. The key is a
      normal authenticated principal: collection and graph access rules
      apply to it.
    • Events: plugin handlers subscribe through the existing realtime
      websocket using a per-instance consumer group - at-least-once delivery
      with ACK-committed group offsets, so events that occur while a plugin
      is down are redelivered when it comes back (see the realtime section
      below).
    • Replicas never supervise plugins (like webhooks and realtime);
      instance status stays readable through the synced registry.

    API

    All admin-only:

    • GET /api/plugins, POST /api/plugins/scan - catalog from manifests
    • GET/POST /api/plugin-instances (instances are unique per
      (dbName, name)), GET/PATCH/DELETE /api/plugin-instances/:name?db=
      • a config change bumps configVersion and restarts only that
        instance's process
    • POST .../:name/start|stop|restart
    • POST .../:name/invoke - {function, params, timeoutSeconds?}
      (default 30s, max 120s) over the warm pipe
    • GET .../:name/logs?level=&limit= - captured stderr (newest first;
      kept 7 days in _plugin_logs in stats.db)

    Collection triggers and HTTP cron jobs accept
    plugin://<instance>/<function> as their URL: the webhook engine
    dispatches it in-process with the same retry policy and the same
    _webhook_logs pipeline - a zero-hop alternative to pointing a trigger
    at localhost. Cron jobs resolve their own database for the instance
    lookup.

    Dashboard

    Settings → Plugins: catalog cards from the manifests, the instances
    table with live status chips / restart counts / uptime, a config editor
    generated from each manifest's schema (saving restarts just that
    process), an invoke console (function picker + params + result), a
    captured-log tail, start/stop/restart actions, and a disabled-state
    banner pointing at the env flags. The dashboard now depends on
    @vski/sdk@^2.2.0.

    Writing a plugin

    from vski.plugin import Plugin

app = Plugin()

@app.on_event("documents")      # realtime consumer group, at-least-once
async def on_document(ctx, event): ...

@app.function("reindex")        # callable via /invoke and plugin:// triggers
async def reindex(ctx, params): ...

@app.cron("*/5 * * * *")        # dependency-free 5-field cron
async def sweep(ctx): ...

    ctx carries a db-scoped client, the instance config and a log
    whose output the server captures. Four working examples ship in
    examples/plugins/: hello (every handler type, doubles as the e2e
    fixture), note-bot (the wiki tutorial plugin), embeddings-indexer
    (event-driven chunking/embedding via an OpenAI-compatible endpoint -
    formalizes the Embeddings tutorial's "next steps") and graph-sync
    (mirrors a collection into a light graph with properties.ref links -
    formalizes the Graphs tutorial's sync pattern).

    Realtime: durable consumer groups and fixes

    Work on plugin event delivery hardened the realtime gateway for
    everyone:

    • API keys are now accepted on /api/realtime (?auth=<key>,
      any enabled key) - service processes can subscribe like any client.
    • Consumer groups are durable. ACKs now advance a per-group
      committed offset, and consumers joining later (or plugin processes
      restarting) resume after it instead of tailing from "now" - events
      that fired while nobody was connected are no longer skipped. Group
      deliveries now carry the ackId field both SDKs already expected
      (neither ever ACKed before, because the server never sent it).
    • Fixed: a fresh group consumer received one stale event from the
      head of the untrimmed event store (delivery position initialized
      from zero instead of the subscription position).
    • Fixed: event frames carried the id of the first stored event
      with a matching action rather than their own, so ACKs could commit
      the wrong position.
    • Event ids are zero-padded (<millis>-<seq>), making the id ordering
      used throughout the gateway numerically correct.

    Non-group subscriptions are unchanged. Delivery remains at-least-once
    for groups.

    Supervisor hardening

    Start/stop transitions are serialized per instance (a restart can no
    longer race the reconciler into double-spawning an orphaned process),
    shutdown waits are bounded, and no process is spawned once shutdown has
    begun.

    SDKs

    • TypeScript/JS (@vski/sdk v2.2.0): new client.plugins
      namespace - list, scan, listInstances, getInstance,
      createInstance, updateInstance, deleteInstance,
      startInstance/stopInstance/restartInstance, invoke, logs -
      with full types (PluginCatalogItem, PluginInstance,
      PluginRuntimeInfo, PluginLogEntry).
    • Python (vski-sdk v0.5.0): the vski.plugin module described
      above, a matching client.plugins management namespace, and three
      transport fixes: request paths with an embedded query string were
      silently dropped (httpx replaces a URL's query when given params,
      even empty), websocket auth tokens are now URL-quoted (raw-base64
      API keys containing + previously failed as spaces), and group
      subscriptions track lastId so a reconnecting consumer resumes
      after the last seen event.

    Documentation & tests

    Wiki: Plugins reference
    (architecture, manifest spec, env contract, API, security model,
    distribution Dockerfile) and a hands-on
    Plugins Tutorial
    whose flows are covered end-to-end by the new suites:
    48_plugins_test.ts (15 steps: catalog, lifecycle, warm invoke, event
    delivery, captured logs, per-instance restart on config change,
    plugin:// trigger delivery, clone-into-second-db, crash recovery,
    admin gating, disabled-503) and test_48_plugins.py (instance
    lifecycle, event delivery, plugin:// triggers, admin gating,
    including downtime redelivery). The e2e harnesses now start their
    servers with plugins enabled and the examples directory mounted.

    Compatibility

    • No breaking API or storage changes. Plugins are off by default; the
      plugins / plugin_instances tables (default.db) and _plugin_logs
      (stats.db) are created idempotently on boot.
    • Realtime group consumers start ACKing and resume from committed
      offsets - strictly fewer lost events; clients that manually ACKed are
      unaffected. The padded event-id format only matters within a single
      server process (the event store is in-memory).
    • Official docker images are unchanged (no Python); plugin-enabled
      distributions build FROM them per the wiki.
    Downloads