For AI agents: the complete documentation index is available at https://silt-db.dev/llms.txt, the full documentation bundle is available at https://silt-db.dev/llms-full.txt, and this page is available as Markdown at https://silt-db.dev/reference/pipeline.md.

    Pipeline stages and accumulators

    For your first pipeline, read Aggregations. This reference lists accepted forms and limits so you can check a pipeline before moving it from MongoDB or Mingo.

    The compiler currently implements 20 stage names and 18 group accumulator names, within the finite JSON data profile. This is an implemented subset, not full Mingo or MongoDB parity. The initial research examined MongoDB 8.3.8; executable differential checks in the captured validation run use MongoDB 8.0.29. No MongoDB 8.3 execution is claimed.

    Implemented stages

    Stage namesSupported forms and relevant limits
    $matchDelegates to the query compiler, including $expr; see its operator inventory.
    $projectInclusion/exclusion with Mongo _id exception; expression fields; dotted paths; nested array traversal and reconstruction. Missing inclusion fields are omitted; non-object array elements are omitted for nested inclusion, and preserved/replaced as appropriate for computed projection. Conflicting paths and mixed modes fail. Positional find projection is not implemented.
    $set, $addFieldsAll assigned expressions see the original input document. Dotted assignment traverses arbitrarily nested arrays and replaces scalar ancestors with objects, as aggregation does. This is distinct from update-operator $set semantics.
    $unsetOne field name or a nonempty list; dotted exclusions traverse nested arrays.
    $replaceRoot, $replaceWithReplacement must evaluate to an object; missing/scalar/array replacements fail in SQLite.
    $sortNumeric directions 1 and -1; mixed JSON type ordering, ordered objects, lexicographic nested arrays, ascending array minimum and descending maximum, empty-array ordering. Sorting on more than one array-valued path in one document fails explicitly: correlated compound multikey sort keys are not yet supported. $meta and collation are unsupported.
    $limit, $skipLiteral safe integers: positive limit and nonnegative skip. Order is explicitly carried between stages.
    $countA nonempty simple output field; empty input emits no document, matching MongoDB rather than Mingo 7.2.4.
    $unwindString or object form; preserveNullAndEmptyArrays, includeArrayIndex; scalar input as singleton; distinction between missing, null, and empty arrays.
    $groupExpression group key, including objects/arrays and missing-to-null keys. Deep numeric equality and field-order-sensitive object identity use the common SQL comparator/key representation. Accumulators listed below.
    $sortByCountLowers into grouping and descending count sort.
    $lookupEquality local/foreign fields; scalar-array and array-array membership; missing/null matches; duplicate join-key suppression; correlated/uncorrelated pipelines; let/$expr; nested lexical variables; equality followed by optional foreign pipeline; per-local-document foreign sort/limit/group/project; dotted as overwrites conflicting ancestors. A from-less pipeline can begin with literal $documents. Only stages/expressions implemented by this compiler are available in its pipeline.
    $facetMultiple subpipelines over materialized shared input, including empty-input result arrays. Rejects nested $facet and prohibited administrative/write stages. Does not enforce MongoDB's exact facet/result byte limits.
    $unionWithString collection or {coll,pipeline}; a collection-less pipeline may begin with literal $documents. Uses UNION ALL.
    $documentsLiteral array of JSON objects; first stage only. Dynamic expression form is not implemented.
    $sampleLiteral nonnegative safe integer size; SQL random order without replacement. Does not emulate MongoDB's exact random distribution/algorithm or seed behavior.
    $bucketIncreasing homogeneous number or string boundaries, optional default outside the boundary interval, optional supported group accumulators. Lower boundary inclusive, upper exclusive. Other boundary types fail explicitly.

    These count aliases separately because they are distinct accepted Mongo stage names: $set/$addFields and $replaceRoot/$replaceWith.

    Implemented group accumulators

    AccumulatorsSemantics and limitations
    $countEmpty object operand only.
    $sum, $avgIgnore nonnumeric/missing/null inputs. Sum returns zero if no numeric values; average returns null. Inputs are evaluated as IEEE-754 doubles to avoid SQLite's signed-integer aggregate overflow. BSON int32/int64/Decimal128 promotion and exact decimal arithmetic are outside the data profile. Nonfinite numeric output fails explicitly.
    $min, $maxIgnore missing/null; return null when no remaining values. Compare complete JSON values using BSON-style ordered object and array comparison.
    $pushPreserves input order and omits missing values; explicit null retained.
    $addToSetDeep numeric equality with ordered object identity; missing omitted. Output order is unspecified.
    $first, $lastRespect carried pipeline order. Missing selected value becomes null.
    $firstN, $lastN, $minN, $maxN{input,n} with a literal positive safe integer n. Per-group/dynamic n expressions are unsupported and rejected; unknown fields fail.
    $top, $bottom{sortBy,output}; output may be a supported expression. Returns scalar output as scalar, unlike Mingo 7.2.4's observed single-element-array result.
    $topN, $bottomN{sortBy,output,n} with literal positive safe integer n; same sorting restrictions as $sort.
    $mergeObjectsIgnores missing/null, requires objects otherwise; last value wins, null values are retained, nested object values replace rather than recursively merge. Keys retain first-seen order.

    Explicitly unimplemented

    Stages include $setWindowFields, $graphLookup, $densify, $fill, $bucketAuto, $redact, $out, $merge, $geoNear, server/cluster metadata stages, Atlas search/vector/fusion stages, and other unlisted stages. No JS fallback exists.

    Accumulators include $accumulator, $stdDevPop, $stdDevSamp, $median, $percentile, window-specific accumulators, and other unlisted accumulator names. Operator names shared with array expressions do not imply group-accumulator support; for example, an expression $concatArrays does not make the group accumulator $concatArrays available.

    Also outside the current profile: BSON-only types, Decimal128, locale collation, timezone database semantics, MongoDB's full error codes/messages and resource limits, and exact query planner/index behavior. Optional native PCRE2 expressions are available when the extension is explicitly enabled. Primitive unsupported syntax is rejected before execution, including on an empty input. Some document-dependent semantic failures are deliberately raised inside SQLite and surface as its JSON validation error, rather than MongoDB's corresponding error code.

    SQL strategy and performance

    Every stage emits a SQL relation carrying JSON text doc and an ordering key ord. SQLite performs matching, joins, grouping, sorting, projection, and output assembly. JavaScript only compiles SQL and serializes/deserializes data. No JavaScript SQLite UDFs are registered.

    Nested-array projection uses json_tree, recursive traversal, and iterative json_set reconstruction, not a fixed JS recursion limit on document array depth. Generated SQL size grows linearly with dotted path depth after factoring intermediate expressions; a 20-component $set path generates roughly 27 KB, rather than exponential text expansion. SQLite's own parser/expression/resource limits still apply.

    For scalar sort keys, ordinary SQLite type rank plus scalar ORDER BY applies. If compound values occur, exact structural comparisons use an O(n²) SQL ranking fallback. This prioritizes correct JSON ordering over speed. It is expensive: in one local run, sorting 17 mixed scalar/object/array documents took roughly 250–315 ms; compound minN/maxN took about 1.5 seconds. Those are diagnostic observations, not performance guarantees or a benchmark claim. Compiled SQL for general semantic paths is considerably larger than hand-written scalar SQL. Optimizing typed scalar indexes, comparator normalization, and sort-key encoding is future work.

    $lookup uses correlated SQL aggregation; no foreign collection is loaded into JS. Without an index-compatible foreign predicate it can still scan the foreign collection for each local document. Group arrays, facets, and lookup output arrays are constructed within SQLite, so persistent storage does not imply bounded size for one returned document. Streaming the final cursor avoids materializing the whole result set in JS; .toArray() intentionally collects results.

    Validation evidence

    • test/pipeline.test.js: 104 passing cases, including ordinary SQLite execution, Mingo differential examples, explicit Mongo expectations for known Mingo differences, join and failure regressions, and 23 captured real-Mongo expected results.
    • test/mongo-pipeline.test.js: real MongoDB comparison for 23 edge cases (24 tests including the parent), including nested projection/set arrays, confirmed Mingo discrepancies, projection field order, lookup literal sources, and compound sort/min/max/top/bottom. Requires MONGODB_URI; absent-server tests are marked skipped, not counted as passed conformance.
    • test/fixtures/mongo-pipeline-reference.json: the actual MongoDB 8.0.29 responses for those 23 cases, with version and capture time. The ordinary offline suite replays them against SQLite. To recapture with a real server, run SILT_CAPTURE_MONGO_PIPELINE=1 node --test test/mongo-pipeline.test.js with MONGODB_URI set.
    • The broader Mongo corpus and independent review tests are documented separately. These tests validate the listed examples; they do not establish exhaustive conformance for every operand combination.

    Remaining known compatibility caveats

    Projection retains source order for included fields, puts _id first, and appends computed fields in specification order. Dedicated tests compare serialized field order against MongoDB; ordinary JavaScript deep equality alone would not catch these differences. More combinations of nested mixed inclusion/computation remain useful conformance work.

    Sorting on multiple array-valued paths is deliberately rejected, even where MongoDB can derive a correlated compound key. Scalar/object compound sorts and a single multikey path with other scalar keys are tested.

    Very large/deep pipelines can hit SQLite parser, expression depth, compound SELECT, bind-variable, or JSON nesting limits before MongoDB's corresponding limits. Error translation and explicit configurable compiler budgets should be developed alongside performance work.