JSON storage and JSONB
Silt stores documents as JSON TEXT in SQLite. SQLite gathers, filters, joins, aggregates, and updates that data; JavaScript serializes inputs and decodes returned documents. No BSON encoding or JavaScript query evaluator is installed.
SQLite JSONB and MongoDB BSON are different formats. The optional binary JSON representation in SQLite could avoid some parsing work, but it would not add MongoDB types such as Date or ObjectId. Silt's current domain is pure JSON, which fits ShareDB's data contract.
Document and value representation
Each core collection has a physical table with:
The compiler also uses JSON text for intermediate values. SQL NULL means a missing value; the text null means explicit JSON null. Keeping those distinct is necessary for projections, matching, grouping, and array behavior. Existing values are copied from their original stored JSON tokens where possible; computed values go through explicit numeric rendering.
JSON arrays and objects remain structured values. Object field order matters for Mongo-style embedded-object equality, and numeric semantic equality cannot be replaced with raw text equality. Refer to Compatibility and Numeric precision before assuming SQLite's ordinary casts or text comparisons are equivalent.
IDs and storage migration
A missing _id becomes a UUID string. Scalar and valid embedded-object IDs are supported; arrays and dollar-prefixed object keys are rejected. String IDs use the SQLite unique-key fast path and are preferable for large collections.
For non-string IDs, Silt checks semantic equality inside a write transaction before insertion and keeps semantic SQL predicates when reading. Those checks can scan a collection. This avoids relying exclusively on the spelling of a numeric key, which can vary across SQLite engines. It does not repair inaccurate numeric parsing in an engine.
Core collection storage has a key-format marker. On first writable access, a missing or older marker triggers an atomic SQL-only key migration; documents are not rewritten, and a failed migration restores the previous keys and marker. ShareDB's separate tables are unaffected. Read-only legacy collections remain readable through conservative predicates and are not migrated. The marker describes key format rather than SQLite release, so compatible engines do not rekey a shared file on every open.
The known arbitrary-double boundary in Expo SQLite 3.50.3 still applies to equality and computed values, including very close numeric IDs. Numeric precision records examples and measured scope.
JSONB experiment
The current decision is to retain JSON TEXT. A controlled storage-format experiment found improvements for scanning larger documents, but regressions for complete-document reads and committed insertion. There is no runtime JSONB mode or automatic format conversion option.
The experiment used Node 24.19.0 and SQLite 3.53.3 with 3,000 documents per format, either a 1 KiB or 8 KiB document-specific string payload, plus nested data and a 100-document lookup source. Seven measured rounds followed two warmups. Statements were prepared before timing; TEXT/JSONB order alternated. Actual compiler output was used for both formats and results were verified against independent expected documents.
The general compiler repeatedly produces JSON text through ->, constructors, json_set, and aggregation. Changing only the stored format therefore removes some parsing, while JSONB needs a final json(doc) conversion for the result decoder. Both formats still require JSON.parse when returning JS objects. Raw JSON extraction was faster with JSONB, but the complete compiler workloads showed smaller or mixed gains.
No secondary indexes were installed in this comparison. The measured updates timed the UPDATE statement inside an existing transaction, excluding durable commit and full public API overhead; they did not test jsonb_set partial-write optimization. Committed insert timing was a separate measurement. These are local exploratory results, not guarantees for another server or for Expo.
The detailed report and every sample, SQL statement, plan, format probe, and verification hash are retained in internal/benchmarks.
Why switching needs a format design
The executable probes identified concrete boundaries:
The same format probes ran on the actual Expo web WASM engine, SQLite 3.50.3. Basic JSONB exists there, but jsonb_each and jsonb_tree require SQLite 3.51.0 and cannot be assumed for that target. These browser probes confirm format behavior, not performance or native-device compatibility. See SQLite's JSON documentation and 3.51.0 release notes.
The string-heavy fixtures showed small storage differences: JSONB values were roughly 2.47% smaller with a 1 KiB payload and 0.34% smaller with 8 KiB; whole database files differed by only one 4,096-byte page. Heap/RSS snapshots retained both fixtures and both databases, so they cannot rank memory consumption by storage format. SQLite allocator/cache usage was unavailable in that experiment.
Revisit JSONB for a concrete workload only after covering migration, result decoding, write boundaries, and the full cross-driver correctness surface. A binary intermediate representation needs a separate design for missing/null values, booleans, structural equality, and aggregate outputs.
Reproduce the experiment
From the repository root:
Optional arguments set document count (1,000–10,000; default 3,000) and measured rounds (3–15; default 7). The runner uses temporary SQLite files and removes them after verification. It requires no SQLite extension or JavaScript SQL functions. For application-level improvements first, see Performance.