x/platform
1
0
Fork 0
Code Issues Pull requests Projects Releases 16 Packages Wiki Activity Actions
16 releases 16 tags
  • v0.7.1 e7c332a318

    v0.7.1 Stable

    x released this 2026-08-23 18:31:41 +00:00 | 0 commits to main since this release

    Signed by x
    GPG key ID: A14ACA8AB45A9C27

    VSKI v0.7.1

    Dashboard: database-scoped collections (web UI)

    The standalone dashboard now works with the per-database collection
    registry introduced in v0.7.0. Browsing a collection that lives outside
    the default database previously failed with
    {"error":"Collection not found"} — every dashboard request resolved in
    default, where the collection does not exist.

    • Sidebar collection links carry their database
      (/collections/<name>?db=<dbName>), so the same bare name in two
      databases is addressable and never ambiguous.
    • Collection pages resolve the database from the link (falling back
      to the default db when the URL is unqualified) and issue every
      records operation — list, create, update, delete, bulk — through the
      db-scoped SDK namespace.
    • Collection settings (schema/indexes/views/rules editor) and the
      API Rules page PATCH the collection through its own database;
      relation pickers load their options from the same db.
    • System collections (_api_keys, _sql_snippets, …) keep resolving
      from any namespace, unchanged.

    SDK (TS/JS) v2.0.4

    Published to the package registry. client.db(name) now also exposes the
    settings namespace (client.db(name).settings.collections.*) so
    registry operations resolve within the named database; records namespaces
    were already scoped. This is the version the dashboard ships against.

    Compatibility

    No API or storage changes — v0.7.1 is the dashboard + SDK companion
    release for the v0.7.0 per-database registry.

    Downloads
  • v0.7.0 74ab95c23e

    v0.7.0 Stable

    x released this 2026-08-23 17:17:00 +00:00 | 2 commits to main since this release

    Signed by x
    GPG key ID: A14ACA8AB45A9C27

    VSKI v0.7.0

    Per-Database Collection Names

    Collection names are now scoped per database instead of globally unique.
    Two databases may each have an orders collection — they are fully
    independent namespaces with independent schemas, records, views, indexes,
    realtime topics, and tombstones. Multi-tenant consumers no longer need to
    prefix collection names to avoid collisions.

    Breaking Changes

    • Bare-name cross-database access no longer falls back to a global
      lookup.       Collection names and IDs resolve strictly within the database
      addressed by the x-dbname header (default default); a wrong or
      missing header is a 404. Cross-database access must qualify the
      database (client.db(name).collection(...), collection(name, db), or
      the x-dbname header).
    • Registry uniqueness is now (dbName, name) instead of name
      alone. Creating a collection that duplicates a live name in the same
      database still fails with Collection name already exists; the same
      name in another database is allowed.
    • Realtime topics are keyed (db, name) and the connection's ?db=
      parameter is authoritative: SUBSCRIBE resolves the collection in that
      database and a miss is a loud ERROR frame. EVENT/SUBSCRIBED frames
      carry a new additive db field.
    • Duplicate relation targets must be qualified. Relation fields may
      declare an optional options.db for cross-database targets; relations
      without it resolve in the source collection's own database (then a
      unique global match). A name present in several databases with no
      qualifier is a 400 naming the candidates.

    Compatibility

    • System collections (_users, _stats_*, workflow queues) are
      platform-global and keep resolving from any namespace — stats and
      workflow surfaces are unchanged.
    • Legacy database files migrate automatically on boot: the inline
      UNIQUE(name) constraint is replaced by the uq_collections_db_name
      composite index via an idempotent table rebuild; rows are preserved.
    • Create-over-tombstone now revives the tombstoned row under its
      original id instead of a blind name-keyed resurrection followed by an
      error.
    • Options-based views in non-default databases now resolve (the options
      lookup previously ran against the target file, which never holds
      registry rows).

    SDKs

    • client (TS/JS): registry ops accept a database scope
      (client.db(name).settings), realtime subscriptions re-key to the
      db:name topic and dispatch on the frame's db.
    • client-python: every settings.collections method takes an
      optional db; realtime dispatch filters subscription entries by the
      frame's db. client.collection(name, db) was already scoped.
    Downloads
  • v0.6.9 29967aac64

    v0.6.9 Stable

    x released this 2026-08-14 10:36:13 +00:00 | 5 commits to main since this release

    Signed by x
    GPG key ID: A14ACA8AB45A9C27

    VSKI v0.6.9

    Maintenance Release

    Python SDK Release

    Breaking Changes

    None

    Downloads
  • v0.6.8 fcbf5fd20c

    v0.6.8 Stable

    x released this 2026-03-02 17:44:24 +00:00 | 8 commits to main since this release

    VSKI v0.6.8

    Maintenance Release

    Mostly fixing links in documentation and references in package.

    Migrate Domain Name

    Previously packages and containers were hosted on git.vski.sh/x now it's just vski.sh/x

    Breaking Changes

    None

    Downloads
  • v0.6.7 fcbf5fd20c

    v0.6.7 Stable

    x released this 2026-02-27 10:47:04 +00:00 | 8 commits to main since this release

    Signed by x
    GPG key ID: A14ACA8AB45A9C27

    VSKI v0.6.7

    Features

    Cron Job Handling on Replicas

    Cron jobs are now properly disabled on replicas to prevent duplicate execution and data conflicts.

    Behavior on Replicas

    • HTTP cron jobs: Always skipped on replicas (would cause duplicate external calls)
    • SQL cron jobs: Skipped for replicated databases (all databases except stats and workflows)
    • SQL cron jobs on non-replicated databases: Execute normally (e.g., cleanup jobs on stats or workflows)

    This ensures that:

    1. Replicas don't trigger duplicate HTTP webhooks
    2. SQL jobs don't conflict with master's data (replicated DBs are read-only)
    3. System maintenance jobs on non-replicated databases still run

    Realtime and Webhooks Disabled on Replicas

    On replica servers, the following are now disabled to prevent conflicts:

    • Realtime WebSocket: Connections are rejected with 503 status
    • Collection Webhooks: Trigger webhooks are not emitted on replicas

    This ensures all events originate from the master server, preventing duplicate webhooks and maintaining data consistency.

    For distributed realtime across multiple regions, see the Distributed Realtime Service documentation for implementing a Redis-backed realtime service.

    Technical Details

    Files Changed

    • internal/app/bootstrap.go - Configure cron replica mode, disable realtime/webhooks on replicas
    • internal/services/cron.go - Added replica mode support, skip jobs on replicated databases
    • internal/realtime/gateway.go - Added SetDisabled() method, reject WS connections when disabled
    • internal/services/webhooks.go - Added SetDisabled() method, skip triggers when disabled

    Breaking Changes

    None

    Migration

    No migration required.

    Downloads
  • v0.6.6 48794c0dee

    v0.6.6 Stable

    x released this 2026-02-27 08:51:22 +00:00 | 9 commits to main since this release

    Signed by x
    GPG key ID: A14ACA8AB45A9C27

    VSKI v0.6.6

    Features

    Replica Identification and Routing

    Replicas can now identify themselves with a unique ID and public URL, enabling proper routing by load balancer and workflow executors.

    How It Works

    1. Replica Identification: Replicas send X-Replica-Id and X-Replica-Public-Url headers to master during sync
    2. Master Tracking: Master stores known replicas in _known_replicas table with their public URLs
    3. Response Headers: Replicas include X-Replica-Id header in all response
    4. Client SDK: Client automatically captures X-Replica-Id from responses and includes it in subsequent request
    5. Workflow Routing: Workers can specify replicaId when subscribing to workflows to route job to specific replicas

    Incremental Sync

    Replicas now use SQLite changesets for incremental synchronization instead of full database downloads, dramatically reducing sync time and bandwidth.

    How it Works

    1. Initial Sync: Fresh replicas download the full database from master
    2. Session Reset: After full download, replica resets master's session to ensure clean changeset tracking
    3. Schema Comparison: Replicas compute and compare schema hashes with master before attempting incremental sync
    4. Schema Migration: If schemas differ, replica fetches master schema and applies migrations (CREATE TABLE, ALTER TABLE, indexes, triggers, views)
    5. Changeset Application: After schema sync, replicas apply changesets for data changes (INSERT, UPDATE, DELETE)
    6. Automatic Fallback: If replica has no replication state, it downloads full database

    Schema Synchronization

    When schema changes are detected (new collections, field changes, indexes), replicas automatically migrate their schema:

    1. Hash-Based Detection: Schema hashes are computed from sqlite_master (tables, indexes, triggers, views)
    2. Schema Fetch: Replica fetches full schema from master via /api/replica/schema-sync
    3. Diff Computation: Replica computes schema diff (new tables, altered tables, new/dropped indexes, triggers, views)
    4. Migration Application: Replica applies schema migrations:
      • CREATE TABLE for new collections
      • ALTER TABLE ADD COLUMN for new fields (columns are never deleted or renamed)
      • CREATE INDEX, CREATE TRIGGER, CREATE VIEW for new objects
      • DROP INDEX, DROP TRIGGER, DROP VIEW for removed objects
      • Tables are never dropped on replica

    Configuration

    # Replica server configuration
REPLICA_MODE=replica
REPLICA_ID=my-replica-001        # Unique identifier for this replica
PUBLIC_URL=https://replica.example.com  # Public URL for routing
MASTER_URL=https://master.example.com
SYNC_INTERVAL=60

    New Admin Endpoints

    Endpoint Description
    GET /api/admin/known-replicas List all known replicas that have synced with master
    GET /api/admin/known-replicas?with_public_url=true Filter replicas by presence of public URL

    Client SDK

    // Health check endpoint
const isHealthy = await client.health();

// Get list of known replicas from master
const replicas = await client.replicas.listKnown();

// Filter by those with public URLs
const publicReplicas = await client.replicas.listKnown({ withPublicUrl: true });

// Client automatically captures replicaId from responses
const records = await client.collection('posts').getList(1, 10);
console.log(client.replicaId); // Set from X-Replica-Id header

// Route workflow to specific replica
await client.workflow.start('my-workflow', { arg1: 'value' }, { replicaId: 'replica-001' });

    Web Dashboard

    • Connection modal displays known replicas with their IDs for allowing quick switching between master and replica servers
    • Health check endpoint used for connectivity verification

    Cron Job Handling on Replicas

    Cron jobs are now properly disabled on replicas to prevent duplicate execution and data conflicts.

    Behavior on Replicas

    • HTTP cron jobs: Always skipped on replicas (would cause duplicate external calls)
    • SQL cron jobs: Skipped for replicated databases (all databases except stats and workflows)
    • SQL cron jobs on non-replicated databases: Execute normally (e.g., cleanup jobs on stats or workflows)

    This ensures that:

    1. Replicas don't trigger duplicate HTTP webhooks
    2. SQL jobs don't conflict with master's data (replicated DBs are read-only)
    3. System maintenance jobs on non-replicated databases still run

    New Environment Variables

    Variable Default Description
    REPLICA_ID (none) Unique identifier for this replica instance
    PUBLIC_URL (none) Public URL for routing requests to this replica

    Technical Details

    New Replica API Endpoints

    Endpoint Method Description
    /api/replica/schema-sync GET Get full schema from master for migration (internal)
    /api/replica/reset-session POST Reset master's SQLite session after full download (internal)

    Files Changed

    • internal/replica/syncer.go - Incremental sync logic, schema comparison and migration, hash-based change detection
    • internal/replica/session_manager.go - Session management, ResetSession() method, schema hash computation, GetFullSchema()
    • internal/api/replica.go - ResetSession endpoint, SchemaSync endpoint, changeset generation
    • internal/config/config.go - Added ReplicaID and PublicURL config fields
    • internal/replica/middleware.go - Added X-Replica-Id to ReplicaStatusMiddleware
    • internal/replica/types.go - Added KnownReplica struct
    • internal/replica/registry.go - Added UpsertKnownReplica, ListKnownReplicas with filter
    • internal/db/db.go - Added _known_replicas table schema
    • internal/app/bootstrap.go - Pass cfg.ReplicaID to syncer and middleware, configure cron replica mode
    • internal/services/cron.go - Added replica mode support, skip jobs on replicated databases
    • client/src/client.ts - Added replicaId property, health() method, capture from response headers
    • client/src/types.ts - Added replicaId to WorkflowOptions
    • client/src/api/replicas.ts - New namespace for known replicas management
    • web/lib/sdk.tsx - Use /health endpoint for connectivity check
    • web/components/layout/ConnectionBadge.tsx - Display known replicas in server list

    Breaking Changes

    None

    Migration

    No migration required. The _known_replicas table is created automatically on startup.

    Downloads
  • v0.6.5 0b67d2ead8

    v0.6.5 Stable

    x released this 2026-02-26 13:45:57 +00:00 | 10 commits to main since this release

    Signed by x
    GPG key ID: A14ACA8AB45A9C27

    VSKI v0.6.5

    Features

    Workflow Execution on Replicas

    Replicas can now execute workflows locally using their own workflows.db database. Since workflows.db and stats.db are excluded from replication, each replica maintains independent workflow state while still having access to synced user data from default.db.

    This enables:

    • Distributed workflow processing - Workers can run on any replica
    • Load distribution - Offload workflow execution from master
    • Local stats collection - Each replica tracks its own statistics

    Configuration

    No additional configuration required. Workflows automatically work on replicas when:

    • Replica has its own workflows.db (created automatically)
    • Workflow endpoints are excluded from read-only middleware
    # Replica configuration
REPLICA_MODE=replica
MASTER_URL=https://master.example.com
SYNC_INTERVAL=60

# Workflows run locally on replica
# Workers connect to replica's /api/workflow/ws endpoint

    File Replication for Replicas

    When using local storage (not S3), replicas can now automatically sync uploaded files from the master server. This ensures file attachments are available on replica servers alongside the database data.

    How It Works

    1. File Journal: Master server tracks file operations (add/delete) in a _file_journal table
    2. Automatic Sync: After each database sync, replicas fetch and apply file changes
    3. Dual Cleanup Strategy:
      • Ack-based: Journal entries are removed after replicas confirm sync
      • Time-based: Entries older than the retention period are automatically cleaned up

    Configuration

    # Master server - file journal is created automatically when:
# - Not in replica mode
# - Not using S3 storage
FILE_SYNC_RETENTION_DAYS=7  # Default: 7 days

# Replica server - file sync is enabled automatically when:
# - REPLICA_MODE=replica
# - Not using S3 storage
REPLICA_MODE=replica
MASTER_URL=https://master.example.com
SYNC_INTERVAL=60

    New Replica Endpoints

    Endpoint Description
    GET /api/replica/files?since=<id> List file journal entries since the given ID
    GET /api/replica/file/*path Download a file from the master server
    POST /api/replica/files/ack Acknowledge synced files (triggers cleanup)

    Technical Details

    • File journal is stored in default.db and replicates with the database
    • Sync happens immediately after each database sync cycle
    • Files are verified with SHA256 checksum during transfer
    • Journal cleanup runs every 24 hours on the master

    New Environment Variables

    Variable Default Description
    FILE_SYNC_RETENTION_DAYS 7 Days to retain file journal entries (time-based cleanup)

    Technical Details

    Files Changed

    • internal/config/config.go - Added FileSyncRetentionDays config option
    • internal/db/db.go - Added _file_journal table and index
    • internal/replica/file_journal.go - New file journal service with cleanup scheduler
    • internal/replica/syncer.go - Added file sync methods
    • internal/replica/types.go - Added file sync request/response types
    • internal/api/replica.go - Added file sync endpoints
    • internal/services/storage.go - Added journal recording for file operations
    • internal/api/records.go - Record deletion now triggers file cleanup
    • internal/app/bootstrap.go - Wired up file journal and cleanup scheduler

    Breaking Changes

    None. File replication is automatic when using local storage in replica mode. S3 storage continues to handle file replication independently.

    Migration

    No migration required. The _file_journal table is created automatically on startup for master servers using local storage.

    Downloads
  • v0.6.0 0251ae8bea

    v0.6.0 Stable

    x released this 2026-02-25 21:02:55 +00:00 | 12 commits to main since this release

    Signed by x
    GPG key ID: A14ACA8AB45A9C27

    VSKI v0.6.0

    Features

    Cron Jobs: Target Database Selection

    SQL-based cron jobs can now specify a target database to execute queries against:

    await client.cron.create({
  name: "cleanup-tenant-logs",
  schedule: "0 2 * * *",
  type: "sql",
  database: "tenant1",  // Execute against specific database
  sql: "DELETE FROM logs WHERE created < datetime('now', '-30 days')",
});

    Previously, all SQL cron jobs executed against the default database. Now you can select any database from the dropdown in the dashboard or specify it via the API.

    Workflows System Database

    Workflow data is now stored in a dedicated workflows.db system database:

    • Isolation: Workflow runs, events, jobs, hooks, and waits are separated from user data
    • Performance: Workflow operations don't impact user database performance
    • Replication-safe: The workflows database is automatically omitted from replication

    This follows the same pattern as the existing stats database for system data.

    Replication (Beta)

    Note

    : Replication is currently in beta. The feature is production-ready but may undergo changes based on feedback.

    • Master/Replica Modes: Run VSKI in master (default) or replica (read-only) mode
    • Pull-Based Sync: Replicas pull data from master via HTTP endpoints
    • Incremental Sync: Uses SQLite session extension for efficient changeset replication
    • Auto-Discovery: Replicas automatically discover and sync all databases
    • Simple Auth: Master and replica share the same JWT_SECRET - no additional keys needed

    Configuration

    # Master server (default)
JWT_SECRET=your-shared-secret
REPLICA_MODE=master  # optional, master is default

# Replica server
REPLICA_MODE=replica
MASTER_URL=https://master.example.com
JWT_SECRET=your-shared-secret  # MUST match master
SYNC_INTERVAL=60  # seconds, 0 = manual only

    Replica Endpoints

    Replicas authenticate using JWT tokens generated from the shared JWT_SECRET:

    # Get master status (requires replica JWT)
curl -H "Authorization: Bearer <replica-jwt>" https://master.example.com/api/replica/status

# Download a database
curl -H "Authorization: Bearer <replica-jwt>" https://master.example.com/api/replica/db/tenant1

    Technical Details

    • Added SQLite session extension support in vski-sqlite driver
    • New system tables: _replicas, _replication_state
    • New system database: workflows.db for all workflow-related data
    • Read-only middleware blocks all mutations when in replica mode
    • Schema versioning for detecting when full sync is required
    • Replica generates its own JWT token using shared JWT_SECRET

    Breaking Changes

    None. Replication is opt-in via environment variables. The workflows.db is created automatically on startup.

    Migration

    No migration required. New system tables and the workflows.db are created automatically on startup. Existing workflow data in default.db will need to be manually migrated if needed.

    Downloads
  • v0.5.6 0251ae8bea

    v0.5.6 Stable

    x released this 2026-02-23 18:02:48 +00:00 | 12 commits to main since this release

    Signed by x
    GPG key ID: A14ACA8AB45A9C27

    VSKI v0.5.6

    Security Release

    This release addresses multiple security vulnerabilities identified in a security audit. All users are strongly encouraged to upgrade.

    Security Fixes

    Critical

    • Path Traversal Prevention - Database names from x-dbname header are now sanitized to only allow alphanumeric characters, underscores, and hyphens. This prevents path traversal attacks that could access arbitrary files.

    High

    • SQL Identifier Escaping - SqlEscapeIdentifier() now properly escapes double quotes by doubling them, preventing potential SQL injection via identifier names.

    • CORS Origin Validation - WebSocket connections (realtime and workflow) now validate the Origin header against an allowlist. Configure via ALLOWED_ORIGINS environment variable.

    • JWT Secret Warning - In production mode (when DEV is not set to true), a red warning is now printed if JWT_SECRET is not configured.

    • Rate Limiting on Auth Endpoints - Authentication endpoints now have configurable rate limiting to prevent brute force attacks:

      • Default: 5 attempts per hour per email
      • Default: 24 attempts per 24 hours per email
      • Returns HTTP 429 with retry_after when exceeded

    Medium

    • User Enumeration Fix - Authentication error messages are now generic ("Unauthorized") to prevent user enumeration attacks.

    New Environment Variables

    Variable Default Description
    ALLOWED_ORIGINS * Comma-separated list of allowed CORS origins. Use * for all origins (not recommended for production)
    AUTH_RATE_LIMIT_PER_HOUR 5 Max auth attempts per email per hour
    AUTH_RATE_LIMIT_PER_DAY 24 Max auth attempts per email per 24 hours

    Technical Details

    Files Changed

    • internal/config/config.go - Added AllowedOrigins, AuthRateLimitPerHour, AuthRateLimitPerDay config options and production warnings
    • internal/db/db.go - Added sanitizeDBName() function to prevent path traversal
    • internal/middleware/middleware.go - Added RateLimiter, RateLimitMiddleware, updated CORSMiddleware to accept allowed origins, fixed user enumeration
    • internal/utils/query.go - Improved SqlEscapeIdentifier() to escape double quotes
    • internal/realtime/gateway.go - Added origin validation for WebSocket connections
    • internal/workflow/gateway.go - Added origin validation for WebSocket connections
    • internal/app/bootstrap.go - Wired up rate limiting and CORS with config

    Usage Examples

    # Production configuration
JWT_SECRET=your-secure-random-string-at-least-32-chars
ALLOWED_ORIGINS=https://app.example.com,https://admin.example.com
AUTH_RATE_LIMIT_PER_HOUR=5
AUTH_RATE_LIMIT_PER_DAY=24

# Development (defaults)
DEV=true
# ALLOWED_ORIGINS defaults to * (all origins)
# JWT_SECRET defaults to dev-secret (with warning)

    Changelog

    v0.5.6 (2026-02-23)

    Security

    • Fixed path traversal vulnerability in database name handling
    • Fixed SQL injection via unescaped double quotes in identifiers
    • Added WebSocket origin validation (CSWSH prevention)
    • Added rate limiting on authentication endpoints
    • Fixed user enumeration via authentication error messages

    Added

    • ALLOWED_ORIGINS environment variable for CORS configuration
    • AUTH_RATE_LIMIT_PER_HOUR environment variable (default: 5)
    • AUTH_RATE_LIMIT_PER_DAY environment variable (default: 24)
    • Production warning for missing JWT_SECRET
    • Production warning for ALLOWED_ORIGINS=*

    Changed

    • SqlEscapeIdentifier() now properly escapes double quotes
    • CORSMiddleware() now accepts allowed origins parameter
    • WebSocket gateways now validate Origin header
    • Generic "Unauthorized" error message for auth failures
    Downloads
  • v0.5.5 08379c51c7

    v0.5.5 Stable

    x released this 2026-02-23 09:19:11 +00:00 | 13 commits to main since this release

    Signed by x
    GPG key ID: A14ACA8AB45A9C27

    VSKI v0.5.5

    Improvements

    • Better filter operator parsing - Reordered regex alternation to match longer operators first (>=, <=, !=) before shorter ones (>, <, =).
    • Parentheses handling - Added stripParens() function to properly remove parentheses from grouped filter conditions.

    Technical Details

    Files Changed

    • internal/services/records.go - Fixed filter being ignored in field rules bypass path (line 131)
    • internal/utils/query.go - Fixed regex operator order and added parentheses stripping

    Filter Examples Now Working

    // Comparison operators
client.collection("posts").getList(1, 30, { filter: "views >= 100" });
client.collection("posts").getList(1, 30, { filter: "views <= 500" });

// Logical operators
client.collection("posts").getList(1, 30, { 
  filter: "published = true AND views > 100" 
});
client.collection("posts").getList(1, 30, { 
  filter: "category = 'drafts' OR views >= 1000" 
});

// Parentheses grouping
client.collection("posts").getList(1, 30, { 
  filter: "(category = 'tech' OR category = 'news') AND published = true" 
});

// LIKE operator
client.collection("posts").getList(1, 30, { 
  filter: "title ~ 'Post'" 
});

    Testing

    New e2e test added: 52_filters_test.ts - Comprehensive test coverage for:

    • Equality operators (=, !=)
    • Comparison operators (>, <, >=, <=)
    • Logical operators (AND, OR)
    • LIKE operator (~)
    • Parentheses grouping
    • Combined filters with sorting
    • Combined filters with pagination

    Changelog

    v0.5.5 (2026-02-23)

    Fixed

    • Filters now work correctly when field rules are active (admin or defined field rules)
    • Filter parser correctly handles >= and <= operators
    • Parentheses in filter expressions are properly handled

    Added

    • stripParens() helper function in internal/utils/query.go
    • E2E test suite for filter functionality (52_filters_test.ts)

    Changed

    • Regex operator alternation reordered to match longer operators first
    Downloads