dualfroz/kestrel
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(sql): add DiskExecutor and nextValue to DiskExecutor

Browse Source
This commit is contained in:
dualfroz committed 2026-09-06 19:18:12 +02:00
1 parent 1334b9331c
commit f0dee2aee0
1 file changed
+159
Download Patch File Download Diff File Expand all files Collapse all files
src/main/kotlin/kestrel/store/sql/DiskExecutor.kt
+159
View File
@@ -155,3 +155,162 @@ import kestrel.storage.typeName
class DiskExecutorException(message: String) : KestrelException(message)
/**
* Executes a deliberately narrow subset of the SQL AST directly against
* [DiskCatalog]/[DiskTable]: CREATE/DROP TABLE, INSERT, SELECT (WHERE/ORDER
* BY/LIMIT/GROUP BY/HAVING/aggregates, window functions (ROW_NUMBER/RANK/
* DENSE_RANK/aggregate/LAG/LEAD/FIRST_VALUE/LAST_VALUE/NTILE OVER
* PARTITION BY/ORDER BY, including an explicit ROWS/RANGE/GROUPS frame with
* EXCLUDE), INNER/LEFT/RIGHT/FULL/CROSS JOIN across any number of table, derived-table
* (`FROM (SELECT ...) AS alias`) or LATERAL derived-table sources,
* scalar/EXISTS/IN subqueries anywhere an expression is allowed - including
* correlated to an outer row - UNION/UNION ALL/INTERSECT/INTERSECT ALL/
* EXCEPT/EXCEPT ALL and WITH/WITH RECURSIVE (CTE, fixed-point with an
* iteration limit)), UPDATE and DELETE, each top-level statement running inside its own disk
* transaction (commit on success, abort on any exception so a failure never
* leaves a partially applied statement); a subquery or nested query (set
* operation branch, CTE definition) runs inside the same transaction as the
* statement that contains it rather than opening one of its own.
*
* This is additive, not a replacement for [kestrel.engine.Executor]: it
* reuses the same AST ([kestrel.sql.ast]), the same [Evaluator] and its
* [SubqueryRunner] hook, the same [GroupAggregation]/[AggregateFunctions]
* (both operate purely on `List<Value>` rows, so they need no disk-specific
* counterpart), the same [StatementResult] types and the same
* [Column]/[TableSchema] the in-memory engine already uses (both
* [DiskCatalog] and [DiskTable] were built against those exact types for
* this reason), so no translation layer sits between the parser and this
* executor - a query that resolves against the disk backend evaluates
* expressions with exactly the same semantics as the in-memory one. Window
* function evaluation itself (frame resolution, peer groups, RANGE distance
* search) is deliberately duplicated from [kestrel.engine.SelectRunner]
* rather than shared, for the same free-standing-isolation reason as every
* other validation this executor duplicates (see [validateUniqueDefinitions]).
* `CREATE [OR REPLACE] VIEW`/`DROP VIEW [CASCADE]` are supported (materialized fresh on every
* reference - as a plain FROM/JOIN source, inside a CTE body, or on either side of a set operation,
* since all three already resolve a name through the same [resolveFromSource] - view-over-view,
* recursive-definition and RESTRICT/CASCADE dependent-view detection, mirroring
* [kestrel.engine.Executor]/[kestrel.catalog.Catalog]'s own view handling; see
* [DiskCatalog.createView]/[DiskCatalog.dropView]). A derived table (`FROM (SELECT ...) AS alias`,
* see [resolveFromItem]) runs eagerly, once, since it cannot correlate to anything; a LATERAL derived
* table (`JOIN LATERAL (SELECT ...) AS alias ON ...`) is instead re-run once per row already
* accumulated from earlier FROM/JOIN sources, correlated to that row exactly like a WHERE-clause
* correlated subquery (see [executeLateralJoin]) - mirroring [kestrel.engine.SelectRunner]'s own
* handling of both.
* Table-level composite PRIMARY KEY, UNIQUE (inline/table-level `UNIQUE`
* constraints, any number of columns), `CREATE [UNIQUE] INDEX`
* (single-column or composite), column-level/table-level CHECK constraints (validated column
* references/no subquery-aggregate-window at CREATE TABLE time via [validateCheckDefinitions],
* enforced on every INSERT/UPDATE via [enforceChecks], persisted as rendered expression text - see
* [CheckConstraintEntry]), and column-level/table-level FOREIGN KEY constraints - including
* composite keys, self-reference, and `ON DELETE`/`ON UPDATE` `CASCADE`/`SET NULL`/`RESTRICT` (default
* `NO ACTION` behaves like `RESTRICT`), enforced via [DiskForeignKeyEnforcer] and blocking `DROP
* TABLE` (RESTRICT by default, `CASCADE` strips the constraint from every referencing table without
* dropping it - see [DiskCatalog.dropTable]) - are supported, mirroring
* [kestrel.engine.Executor]'s own handling of them. `ALTER TABLE ... ADD/DROP/RENAME COLUMN/RENAME
* TO` (see [executeAlterTable]/[DiskCatalog.addColumn]/[DiskCatalog.dropColumn]/
* [DiskCatalog.renameColumn]/[DiskCatalog.renameTable]) is supported with the same RESTRICT/CASCADE
* and FOREIGN KEY/UNIQUE/CHECK/index propagation rules as [kestrel.engine.Executor]/
* [kestrel.catalog.Catalog]. `ALTER TABLE ... ADD CONSTRAINT name CHECK/UNIQUE/FOREIGN KEY` and
* `DROP CONSTRAINT name` (see [executeAddConstraint]/[DiskCatalog.dropConstraint]) validate every
* existing row before installing a new constraint (via a probe transaction, mirroring
* [executeAddColumn]'s own `probeTxn`) and give a UNIQUE constraint the same RESTRICT/CASCADE
* treatment against a dependent FOREIGN KEY that [DiskCatalog.dropTable] already gives a whole table,
* and so is `DROP TABLE ... CASCADE` itself (see [executeDropTable]),
* which additionally drops every transitively dependent view - no DDL mode remains unsupported.
* `BEGIN`/`COMMIT`/`ROLLBACK` (see [executeBegin]/[executeCommit]/
* [executeRollback]) group any number of INSERT/SELECT/UPDATE/DELETE statements into one disk
* transaction with one commit/abort boundary, backed directly by the storage engine's own WAL/2PL
* transaction rather than a copy of [kestrel.engine.Executor]'s whole-catalog snapshot/restore -
* DDL is deliberately rejected while a transaction is open (see [isDdl] in [execute]) because there
* is no equivalent way to undo a [DiskCatalog] schema change by aborting a storage-engine
* transaction. `CREATE`/`DROP`/`ALTER SEQUENCE` and `NEXTVAL`/`CURRVAL` (see [executeCreateSequence]/
* [executeDropSequence]/[executeAlterSequence]/[nextValue]/[currentValue]) are supported with the
* same Postgres-mirroring semantics as [kestrel.engine.Executor] - NEXTVAL alone survives a ROLLBACK
* that undoes everything else the rolled-back statement did (see [DiskCatalog.nextValue]), CURRVAL is
* scoped to this [DiskExecutor] instance the same way it is scoped to one [kestrel.engine.Executor],
* and both are restricted to exactly the same three positions (INSERT VALUES/DEFAULT, UPDATE SET/
* DEFAULT) for the same volatile-function reason - CHECK constraints and `WHERE` reject them outright.
* `SAVEPOINT`/`RELEASE [SAVEPOINT]`/`ROLLBACK TO SAVEPOINT` (see [executeSavepoint]/
* [executeReleaseSavepoint]/[executeRollbackToSavepoint]) are supported inside an explicit transaction,
* with the same stack semantics (reusing a name shadows the older savepoint of that name; releasing or
* rolling back to one forgets every savepoint established after it, but rolling back keeps the target
* itself usable again) as [kestrel.engine.Executor]. Unlike that whole-catalog-snapshot
* implementation, this one partially rolls back the storage engine's own WAL transaction - see
* [kestrel.store.wal.LogRecord.CompensationUpdate] and [kestrel.store.wal.RecoveryManager.rollbackTo]
* for the ARIES-style compensation log records that make undoing only a suffix of a transaction safe
* to repeat and safe to crash through, and [kestrel.store.engine.StorageEngine.savepoint]/
* [kestrel.store.engine.StorageEngine.rollbackToSavepoint] for how a B+Tree's root pointer (plain
* in-memory state no log record tracks) and the buffer pool's cache of pages the undo just overwrote
* on disk are kept in sync with it.
* `CREATE`/`DROP TRIGGER` `BEFORE`/`AFTER INSERT`/`UPDATE`/`DELETE FOR EACH ROW` (see
* [executeCreateTrigger]/[executeDropTrigger]/[fireRowTriggers]/[runTriggerBody]) mirror
* [kestrel.engine.Executor]'s own trigger semantics 1:1 (optional `WHEN`, `NEW`/`OLD` resolved via
* the shared [kestrel.engine.TriggerRowContext], cascading fires across tables up to
* [MAX_TRIGGER_DEPTH]) with one deliberate difference in how atomicity is achieved: instead of a
* whole-catalog snapshot/restore, a whole statement plus every trigger it cascades into is undone by
* [withTriggerAtomicity] taking one internal storage-engine savepoint (see [DiskCatalog.savepoint])
* at the outermost call and rolling back to it on any failure - reusing the exact same primitive
* `SAVEPOINT` is built on, just invisibly. `INSERT ... ON CONFLICT` on a table with a trigger defined
* fires that table's own INSERT triggers for a row that lands fresh and its own UPDATE triggers for a
* row folded into `DO UPDATE` (never both for the same row, and never anything for `DO NOTHING`) - see
* [executeInsertWithConflictHandling] - mirroring the in-memory engine's own resolution of the same gap.
* `TRUNCATE TABLE t1 [, t2, ...] [RESTART IDENTITY | CONTINUE IDENTITY] [CASCADE | RESTRICT]` (see
* [executeTruncate]/[DiskCatalog.truncateTables]) empties every listed table - and, under CASCADE,
* every table transitively reaching it via a FOREIGN KEY - without firing a single row-level trigger
* or consulting any FOREIGN KEY's `onDelete` action, sharing the calling statement's transaction
* exactly like a WHERE-less DELETE; `RESTART IDENTITY` additionally restarts every sequence backing a
* `DEFAULT NEXTVAL(...)` column of a truncated table, the one part of this statement that is NOT
* covered by that same transaction - see [executeTruncate]'s own doc for why, mirroring
* [DiskCatalog.nextValue]'s already-established non-transactional sequence position.
* `MERGE INTO target USING source ON onCondition WHEN [NOT] MATCHED [AND cond] THEN ...` (see
* [executeMerge]/[executeMergeInner]) mirrors [kestrel.engine.Executor]'s own MERGE 1:1 - `source` is
* a table or derived-table subquery resolved once, up front, into a fixed snapshot via
* [resolveMergeSource]/[resolveFromItem], `ON` is matched against a snapshot of the target table taken
* before any mutation (not the live, mutating [DiskTable]), and a target row already affected earlier
* in the same statement throws rather than silently applying a second action - the one difference from
* the in-memory engine is that "already affected"/"this pair's target row" is tracked by
* [kestrel.store.heap.RecordId] rather than [Row] object identity, since a disk row is decoded fresh
* on every read and so is never `===` to an earlier read of "the same" row.
* `PREPARE name AS <SELECT|INSERT|UPDATE|DELETE|MERGE>`/`EXECUTE name(args...)`/`DEALLOCATE [PREPARE]
* name|ALL` (see [executePrepare]/[resolveExecute]/[executeDeallocate]) are ported 1:1 from
* [kestrel.engine.Executor]'s own registry - session-scoped, kept outside [DiskCatalog] for the same
* reason ([explicitTxn] `COMMIT`/`ROLLBACK` never touches it), sharing the same [ParameterSubstitution]
* walkers so `$1`, `$2`, ... resolve identically on both engines.
*/
class DiskExecutor(private val catalog: DiskCatalog) : SequenceAccess {
private val accessPlanner = DiskIndexAccessPlanner()
private val activeCtes = mutableListOf<CteScope>()
private val expandingViews = mutableSetOf<String>()
/** Guards against a trigger whose body (directly or transitively) fires itself forever - mirrors [kestrel.engine.Executor.triggerDepth]. */
private var triggerDepth = 0
/**
* Postgres-session-scoped CURRVAL cache, ported 1:1 from [kestrel.engine.Executor.
* lastSequenceValues] - see that field's own doc for why it is keyed by [Sequence] identity
* rather than by name.
*/
private val lastSequenceValues = HashMap<Sequence, Long>()
override fun nextValue(sequenceName: String): Long {
val sequence = catalog.sequence(sequenceName)
return catalog.nextValue(sequenceName).also { lastSequenceValues[sequence] = it }
}
override fun currentValue(sequenceName: String): Long {
val sequence = catalog.sequence(sequenceName)
return lastSequenceValues[sequence]
?: throw DiskExecutorException("currval of sequence '$sequenceName' is not yet defined in this session - call NEXTVAL first")
}
/**
* Session-scoped, ported 1:1 from [kestrel.engine.Executor.preparedStatements] - deliberately
* outside [DiskCatalog], since a prepared statement carries no data of its own to undo: it is
* unaffected by `COMMIT`/`ROLLBACK` on [explicitTxn] the same way it is unaffected by
* catalog-snapshot rollback on the in-memory engine.
*/
private val preparedStatements = linkedMapOf<String, PreparableStatement>()
val preparedStatementNames: List<String> get() = preparedStatements.keys.toList()