Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added
- **Environment-per-database multi-tenancy (`service-tenant` v4.1)** — Refactored the multi-tenant architecture from "per-organization database" to **per-environment database** high-isolation, with a hard split between Control Plane (environment registry / addressing / credentials / RBAC) and Data Plane (one physical database per environment). See [`docs/adr/0002-environment-database-isolation.md`](docs/adr/0002-environment-database-isolation.md) for the full rationale and trade-offs.
- **Zod protocol schemas** (`packages/spec/src/cloud/environment.zod.ts`): `EnvironmentSchema`, `EnvironmentDatabaseSchema`, `DatabaseCredentialSchema`, `EnvironmentMemberSchema`, `EnvironmentTypeSchema`, `EnvironmentStatusSchema`, `EnvironmentRoleSchema`, `DatabaseCredentialStatusSchema`, `ProvisionEnvironmentRequest/ResponseSchema`, `ProvisionOrganizationRequest/ResponseSchema`. `TenantDatabaseSchema` is now marked `@deprecated`.
- **Control-plane objects** (`packages/services/service-tenant/src/objects/`): `sys_environment` (UNIQUE `(organization_id, slug)`), `sys_environment_database` (UNIQUE `environment_id` — exactly one DB per environment), `sys_database_credential` (rotatable, encrypted, with `active` / `rotating` / `revoked` lifecycle), `sys_environment_member` (UNIQUE `(environment_id, user_id)`, owner / admin / maker / reader / guest). Every field carries `.describe()` metadata and every uniqueness constraint is explicit.
- **`EnvironmentProvisioningService`** (`packages/services/service-tenant/src/environment-provisioning.ts`): `provisionOrganization()` bootstraps a new org with a default environment and DB in one call; `provisionEnvironment()` allocates any subsequent dev / test / sandbox / preview environment; `rotateCredential()` mints a new `active` credential and revokes the previous one. Pluggable `EnvironmentDatabaseAdapter` (initial `turso`; `libsql` / `sqlite` / `postgres` drop in without core changes) and `SecretEncryptor` hooks.
- **Tenant plugin wiring**: `createTenantPlugin()` now registers all four new control-plane objects out of the box, plus `sys_tenant_database` as a v4.x shim (opt out via `registerLegacyTenantDatabase: false`).
- **v4 → v5 migration skeleton** (`packages/services/service-tenant/migrations/v4-to-v5-env-migration.ts`): idempotent, non-destructive, re-encrypts credentials with the current KMS key, reuses existing physical DBs as each org's new `prod` environment DB — no data movement required.
- **Tests**: 22 new schema round-trip tests in `packages/spec/src/cloud/environment.test.ts`, 10 new provisioning tests in `packages/services/service-tenant/src/environment-provisioning.test.ts` covering organization bootstrap, environment creation, default-environment invariants, adapter routing, credential rotation, and encryption hooks.

### Deprecated
- **`TenantDatabaseSchema` / `sys_tenant_database`** — Superseded by the environment-per-database model above. The schema and object remain registered in v4.x as a deprecation shim; both will be removed in **v5.0**. Consumers should migrate by running `migrateV4ToV5Environments()` before upgrading to v5.0.

### Changed
- **Polished `examples/app-crm` dashboards** — Rewrote `executive`, `sales`, and `service` dashboards and added a new unified `crm` overview dashboard, modeled after the reference implementation at [objectstack-ai/objectui/examples/crm](https://github.com/objectstack-ai/objectui/tree/main/examples/crm/src/dashboards). The dashboards now use the framework's first-class metadata fields instead of ad-hoc hex strings stuffed into `options.color`:
- Semantic `colorVariant` tokens (`success`/`warning`/`danger`/`blue`/`teal`/`purple`/`orange`) replace raw hex codes
Expand Down
10 changes: 10 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -628,6 +628,16 @@ Objects now declare `namespace: 'sys'` and a short `name` (e.g., `name: 'user'`)
- [ ] Tenant usage tracking and quota enforcement
- [ ] Cross-tenant data sharing policies
- [ ] Tenant-specific RBAC and permissions
- [x] **Phase 4: Environment-Per-Database Isolation (v4.1)** — ✅ Protocol & service foundation landed (2026-04-19) — see [`docs/adr/0002-environment-database-isolation.md`](docs/adr/0002-environment-database-isolation.md)
- [x] Protocol schemas (`packages/spec/src/cloud/environment.zod.ts`): `EnvironmentSchema`, `EnvironmentDatabaseSchema`, `DatabaseCredentialSchema`, `EnvironmentMemberSchema` + provisioning requests/responses; `TenantDatabaseSchema` marked `@deprecated`
- [x] Control-plane objects: `sys_environment`, `sys_environment_database`, `sys_database_credential`, `sys_environment_member` (all with `.describe()` coverage and explicit UNIQUE constraints)
- [x] `EnvironmentProvisioningService` with `provisionOrganization()` / `provisionEnvironment()` / `rotateCredential()` and pluggable `EnvironmentDatabaseAdapter` (turso/libsql/sqlite/postgres-ready)
- [x] v4.x deprecation shim for `sys_tenant_database` (opt-out via `registerLegacyTenantDatabase: false`)
- [x] v4→v5 migration skeleton (`migrations/v4-to-v5-env-migration.ts`) — idempotent, non-destructive, re-encrypts credentials
- [ ] better-auth session `active_environment_id` integration (v4.2)
- [ ] Per-environment quota enforcement via `sys_quota` (v4.2)
- [ ] Solution publishing on environment DBs via `sys_solution_history` (v4.2)
- [ ] v5.0: remove `sys_tenant_database` and legacy provisioning code; run migration on production tenants

### 6.3 Observability

Expand Down
125 changes: 125 additions & 0 deletions docs/adr/0002-environment-database-isolation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# ADR-0002: Environment-Per-Database Isolation

**Status**: Accepted
**Date**: 2026-04-19
**Deciders**: ObjectStack Protocol Architects
**Supersedes**: The v3.4/v4.0 "per-organization database" tenant model
**Consumers**: `@objectstack/service-tenant`, `@objectstack/spec/cloud`, future `service-subscription`, `service-quota`, `service-audit-log`, `service-dlp-policy`, `service-solution-history`

---

## Context

The v3.4 / v4.0 multi-tenant model in `@objectstack/service-tenant` provisions **one physical database per organization**, registered in `sys_tenant_database`. Logical separation between *environments* (dev / test / prod / sandbox) is achieved by an `env_id` column carried on every row in every data-plane table.

Operating this model in production surfaced five classes of recurring problems:

1. **Leaky logical isolation.** Every query must carry `WHERE env_id = ?`. A single missing predicate in a hand-written query, a migration, a background job, or a badly-written skill can corrupt production from a developer shell.
2. **Coupled schema evolution.** A Solution can't upgrade its schema in `dev` without affecting `prod` — the tables are the same physical tables. This blocks blue/green schema rollouts, destructive migrations, and safe rollback.
3. **Complex backup / DR.** Backing up or restoring just `prod` requires per-row filtering during dump/restore. Point-in-time recovery of one environment leaks into others.
4. **Difficult Solution publishing.** "Promote Solution X from dev to prod" degenerates into row-level copy jobs with `env_id` rewriting — slow, fragile, and nearly impossible to make atomic.
5. **No physical boundary for security / compliance.** Per-environment encryption keys, IP allow-lists, retention policies, and audit isolation all require a per-environment DB to be credible.

Meanwhile, the ecosystem has moved on:

- **Turso / libSQL**, **Neon**, **Supabase branches**, **PlanetScale branches**, and **Cloudflare D1** all make "a database per environment" a near-free operation (milliseconds to provision, cents per month to idle).
- **Power Platform**, **Salesforce**, and **ServiceNow** all expose environments as first-class primitives backed by isolated storage.
- **Kubernetes namespaces** are the pattern developers reach for; the data layer should match.

## Decision

We upgrade the multi-tenant architecture from **per-organization database** to **per-environment database**, with a hard split between Control Plane and Data Plane:

### Control Plane (shared, single database)

Registers environments and how to reach them — **never** stores business data:

| Table | Purpose |
|---------------------------|------------------------------------------------------------|
| `sys_environment` | One row per environment — `(organization_id, slug)` UNIQUE |
| `sys_environment_database`| Physical DB addressing (1:1 with `sys_environment`) |
| `sys_database_credential` | Rotatable encrypted secrets (N:1 with `sys_environment_database`) |
| `sys_environment_member` | Per-environment RBAC (`(environment_id, user_id)` UNIQUE) |

### Data Plane (one database per environment)

Each environment owns its own physical database containing:

- All `sys_` data-plane objects — `sys_package_installation`, `sys_solution_history`, …
- All business objects — `account`, `contact`, user tables, …
- **Zero** `environment_id` columns. The environment is **implicit** in the connection.

### Session → Routing

`better-auth` sessions carry a single `active_environment_id`. The tenant router resolves:

```
session.active_environment_id
→ sys_environment (→ organization_id)
→ sys_environment_database (url, driver, region)
→ sys_database_credential (active secret, decrypted)
→ data-plane driver
```

Switching environments ⇒ swapping DB connections. There is no in-process filter that can be forgotten.

### Provisioning API

`EnvironmentProvisioningService` (new) exposes:

- `provisionOrganization(req)` — atomically creates the org's **default** environment and its physical DB (replaces `provisionTenant`).
- `provisionEnvironment(req)` — allocates any subsequent `dev` / `test` / `sandbox` / `preview` environment, each with its own DB and credential row.
- `rotateCredential(envDbId, plaintext)` — issues a new `active` credential and revokes the previous one.

Physical-DB allocation is delegated to pluggable `EnvironmentDatabaseAdapter` implementations (initially `turso`; `libsql` / `sqlite` / `postgres` drop in without core changes).

### Deprecation & Migration

- **v4.x** keeps `sys_tenant_database` registered as a deprecation shim (TSDoc `@deprecated`, runtime log warning). The new control-plane objects ship alongside it, additive, non-breaking.
- `migrations/v4-to-v5-env-migration.ts` ships in v4.x as a **skeleton** (stable public API) and is executed during the v5.0 upgrade.
- **v5.0** removes `sys_tenant_database` and its reader code entirely.

The migration is **non-destructive** and **idempotent**: each legacy org's database is reused as its new `prod` environment DB — no data movement, no cutover window.

## Consequences

### Positive

- **Hard isolation.** Prod and dev are separate databases; no `WHERE` clause can be forgotten.
- **Independent schema evolution.** Solutions upgrade their schema in `dev`, validate, then promote via a single DB-level backup/restore into `prod`.
- **Trivial backup / DR.** Per-environment backup = native DB backup. PITR stays within one environment.
- **First-class Solution publishing.** "Publish" becomes a schema + metadata export from `dev` and an idempotent apply into `prod`, operating on cleanly-scoped DBs.
- **Per-environment security posture.** Each environment owns its own credential, its own network ACL, its own quotas, its own retention.
- **Pluggable backend.** Driver-agnostic — new backends register an `EnvironmentDatabaseAdapter` without core changes.
- **Future-proof.** Naturally slots in quotas (`sys_quota`), subscriptions (`sys_subscription`), audit (`sys_audit_log`), DLP (`sys_dlp_policy`), and solution history (`sys_solution_history`) as subsequent PRs.

### Negative / Trade-offs

- **More databases to operate.** Every org now has ≥1 DB; heavy users of `sandbox` / `preview` environments may have 5–20. Mitigated by Turso/libSQL free-tier economics and lazy provisioning.
- **Cross-environment reporting** (e.g. "how many leads across all of Acme's envs?") becomes an explicit federation query. Acceptable — such queries are rare and better expressed at the BI layer.
- **Cold starts.** A dormant environment may need to be resumed on first access. Mitigated by the router's TTL cache and the adapter's warm-up hook.
- **Connection sprawl.** A node handling many environments holds N connections. Mitigated by an LRU connection pool with per-env TTL (already present in the v3.4 router).
- **Irrevocable breaking change at v5.0.** v4.x ships the shim and migration; v5.0 removes legacy code. Customers must run the migration before upgrading.

### Neutral

- No change to Zod-first, `.describe()` on every field, `sys_` prefix invariants.
- No change to the public `ObjectKernel` / plugin lifecycle.
- No change to `better-auth` session shape beyond renaming `active_organization_id` → `active_environment_id` (v5.0).

## Alternatives Considered

1. **Stay with per-org DB + `env_id` column.** Rejected — the failure modes above are structural, not implementation bugs.
2. **Schema-per-environment inside one DB.** Works for Postgres but not Turso/libSQL/SQLite, and defeats the backup/DR argument. Rejected.
3. **Row-level security via Postgres RLS.** Strengthens the `env_id` approach but still leaves schema evolution coupled and DR complex. Rejected.
4. **One global DB + tenant column.** Was never on the table — already discarded in v3.4's ADR-0001.

## References

- `packages/spec/src/cloud/environment.zod.ts` — protocol schemas
- `packages/services/service-tenant/src/objects/sys-environment*.object.ts` — control-plane objects
- `packages/services/service-tenant/src/environment-provisioning.ts` — provisioning service
- `packages/services/service-tenant/migrations/v4-to-v5-env-migration.ts` — migration skeleton
- Power Platform environments: <https://learn.microsoft.com/power-platform/admin/environments-overview>
- Salesforce sandboxes: <https://help.salesforce.com/s/articleView?id=data.sandboxes.htm>
- Turso multi-DB pricing: <https://turso.tech/pricing>
24 changes: 24 additions & 0 deletions packages/services/service-tenant/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,35 @@

Multi-tenant context management and routing service for ObjectStack.

> **⚠️ Architectural upgrade (v4.1): Environment-per-Database**
>
> Starting in v4.1 this package ships the **environment-per-database**
> isolation model described in [ADR-0002](../../../docs/adr/0002-environment-database-isolation.md).
> Each `environment` (prod / sandbox / dev / test / preview / …) gets its
> **own physical database**, registered in the new control-plane objects
> `sys_environment`, `sys_environment_database`, `sys_database_credential`,
> and `sys_environment_member`.
>
> The legacy `sys_tenant_database` (per-organization DB) registry is
> **deprecated** and kept as a v4.x shim only. It will be **removed in
> v5.0** together with `TenantDatabaseSchema`. Run
> `migrateV4ToV5Environments()` (from `migrations/v4-to-v5-env-migration.ts`)
> before upgrading to v5.0. The migration is idempotent, non-destructive,
> and reuses your existing physical databases as each org's new `prod`
> environment DB — no data movement required.
>
> New integrations should use `EnvironmentProvisioningService`
> (`provisionOrganization()` + `provisionEnvironment()`) instead of
> `TenantProvisioningService`.

## Overview

This service provides comprehensive multi-tenant infrastructure for ObjectStack deployments, including:

- Tenant identification and context resolution
- **Environment registry + per-environment database provisioning** *(v4.1+)*
- **Rotatable, encrypted per-environment database credentials** *(v4.1+)*
- **Per-environment RBAC** *(v4.1+)*
- Turso Platform API integration for automated database provisioning
- Tenant database schema initialization
- Global control plane management
Expand Down
Loading
Loading