New
1.81.3
Merged PRs
dolt
- 10383: Fix SQL engine panic when database is locked • Prevent NewSqlEngine from panicking when a repo fails to open due to lock contention. • Load-check the underlying DoltDB in engine DB collection and return the recorded load error (e.g. nbs.ErrDatabaseLocked) instead of nil-dereferencing. • Enables embedded/driver callers to detect “database locked” and safely retry engine initialization.
- 10376: /go/libraries/doltcore/sqle/dsess/autoincrement_tracker.go: remove print statement
- 10375: Fix embedded DB lock contention by closing nocache DoltDBs and plumbing DBLoadParams through CREATE DATABASE / clone / stats Summary This change fixes an embedded / driver reopen failure where a two-phase flow (CREATE DATABASE then later Ping) could still hit the database is locked by another dolt process even with disable_singleton_cache + fail-fast lock behavior enabled. Key updates: • Close underlying DoltDBs in nocache mode: sqle.DoltDatabaseProvider.Close() now closes all underlying *doltdb.DoltDB instances when DisableSingletonCacheParam is in effect, ensuring .dolt/noms/LOCK is released on engine shutdown. • Plumb DBLoadParams into provider-managed DB creation paths: • CREATE DATABASE / UNDROP DATABASE now use env.LoadWithoutDB so DBLoadParams can be applied before any DB is opened. • dolt_clone applies provider DBLoadParams to the clone env before registration. • registerNewDatabase defensively applies provider DBLoadParams to the env. • Propagate DBLoadParams into stats backing store: statspro now uses LoadWithoutDB for new stats storage and copies DBLoadParams from the host env so embedded-mode settings apply consistently. • Regression coverage: adds a test that creates a DB via the SQL engine, closes the engine, and asserts the LOCK is released. Why Embedded callers rely on deterministic reopen semantics under contention. Previously, the SQL provider close path didn’t close the underlying *doltdb.DoltDB for newly created DBs in nocache mode, leaving the journal manifest LOCK held and causing retries / reopens to fail.
- 10374: cache
strictLookupIter - 10371: go.mod,.github: Bump Go version to 1.25.6. Fixes potential impact in some configurations from CVE-2025-61729.
- 10366: Bug fix: don't set branch control admin privs when rebasing Fixes https://github.com/dolthub/dolt/issues/10352
- 10363: Add embedded-mode options to bypass DB cache and fail fast on journal lock contention This PR adds opt-in controls for embedded/driver use-cases that need deterministic reopen semantics under contention: • Disable local singleton DB caching for file-backed databases via a new dbfactory param, so each open constructs a fresh store. • Fail fast on journal manifest lock timeout via a new dbfactory param, returning a sentinel error instead of falling back to read-only. • Plumb DB load parameters through the SQL engine / env so an embedded driver can thread these options into Dolt database creation. Key changes • dbfactory: • Adds DisableSingletonCacheParam (disable_singleton_cache) • Adds FailOnJournalLockTimeoutParam (fail_on_journal_lock_timeout) • Bypasses the singleton cache in FileFactory.CreateDB when disable_singleton_cache is set. • nbs: • Adds ErrDatabaseLocked • Adds JournalingStoreOptions + NewLocalJournalingStoreWithOptions • Updates newJournalManifest(..., failOnTimeout) to return ErrDatabaseLocked on lock timeout when enabled. • SQL engine / env plumbing: • Adds DBLoadParams to engine.SqlEngineConfig and threads them into env.DoltEnv prior to DB loading. • Adds DBLoadParams to env.DoltEnv and merges them into both DB load paths (LoadDoltDB and LoadDoltDBWithParams). • Updates env.MultiEnvForDirectory to respect a caller-provided *DoltEnv to avoid preloading before params are set.
- 10314: Minimize binlog deletions in branch control I didn't find a way to trigger multiple deletions through SQL on non-existent rows, but just in case we're somehow able to get to that point through some other bug, we put it in the binlog everytime. This way, we only add to the binlog if we actually had something to delete.
- 9748: Bump gopkg.in/yaml.v3 from 3.0.0 to 3.0.1 in /integration-tests/transactions Bumps gopkg.in/yaml.v3 from 3.0.0 to 3.0.1.
go-mysql-server
- 3406: Set table names to lower when creating table map fixes dolthub/dolt#10385
- 3404: Bump google.golang.org/protobuf from 1.28.1 to 1.33.0 Bumps google.golang.org/protobuf from 1.28.1 to 1.33.0.
- 3403: /go.{mod,sum}: bump go version
- 3399: GMS warning on functional indices
Makes it so
create indexqueries with expression argument will produce a warning instead of an error. - 3396: avoid converting
float64tofloat64We save on conversion costs by avoiding a call to convert float64 to float64. Unfortunately this has little to no impact on any of our benchmarks becausegroupbyIteruses concurrency. benchmarks: https://github.com/dolthub/dolt/pull/10359#issuecomment-3787735021 - 3395: fewer
strings.ToLowercalls ingatherTableAliasbenchmarks: https://github.com/dolthub/dolt/pull/10355#issuecomment-3787079045 - 3388: Apply filter simplifications to Join condition part of dolthub/dolt#10284 part of dolthub/dolt#10335
- 3386: Push filters that contain references to outer/lateral scopes.
We attempt to push filter expressions deeper into the tree so that they can reduce the size of intermediate iterators. Ideally, we want to push filters to directly above the data source that they reference.
Previously, we only pushed filters if they only referenced a single table, since pushing a filter that referenced multiple tables could potentially move the filter to a location where one of the referenced tables is no longer in scope. However, if the extra table references refer to a table in an outer scope or lateral scope, pushing the filter is completely safe. GetFields that reference an outer or lateral scope can be effectively treated as literals for the purpose of this optimization.
This PR changes
getFiltersByTable, a function that maps tables onto the filters that reference those tables. Previously it would ignore filters that reference multiple tables. Now, it allows those filters provided that the extra references are to outer/lateral scopes. This improves many of the plan tests:- The changed test in tpch_plans.go pushes a filter into the leftmost table lookup
- The second changed test in query_plans.go replaces a naive InnerJoin with a CrossHashjoin
- integration_plans.go shows many queries that now have an IndexedTableAccess instead of a table scan, or where we push a filter deeper into a join. A small number of neutral / slightly negative changes:
- One of the changes in integration_plans.go introduces a redundant filter that was previously being removed. In practice this is pretty benign because filters rarely impact the runtime unless they require type conversions.
- The first changed test in query_plans.go replaces a LookupJoin with a LateralCrossJoin on an IndexedTableAccess. These two plans are effectively equivalent, but the LateralCrossJoin is harder to analyze, has a larger estimated cost and larger row estimate, and could in theory inhibit subsequent optimizations. I imagine we could create a new analysis pass that converts this kind of LateralCrossJoin into a LookupJoin.
- 3379: Create scope mapping for views This is a partial fix for dolthub/dolt/issues/10297 When parsing a subquery alias, we create a new column id for each column in the SQA schema. The scope mapping is a dictionary on the SQA node that maps those column ids onto the expressions within the subquery that determine their values, and is used in some optimizations. For example, in order to push a filter into a subquery, we need to use the scope mapping to replace any GetFields that were pointing to the SQA with the expressions those fields map to. If for whatever reason the SQA doesn't have a scope mapping, we can't perform that optimization. We parse views by recursively calling the parser on the view definition. This works but it means that the original parser doesn't have any references to the expressions inside the view, which prevents us from creating the scope mapping. This PR attempts to fix this. Instead of defining the SQA columns in the original parser (where we no longer have access to the view's scope), we now create the columns while parsing the view, and attach them to the scope object for the view definition. Then we store that scope in a field on the Builder, so that the original parser can copy them into its own scope. This feels hacky, but was the best way I could think of to generate the scope mappings and ensure they're visible outside the view.
- 2103: Bump google.golang.org/grpc from 1.53.0 to 1.56.3 Bumps google.golang.org/grpc from 1.53.0 to 1.56.3.
vitess
- 452: Bump google.golang.org/grpc from 1.24.0 to 1.56.3 Bumps google.golang.org/grpc from 1.24.0 to 1.56.3.
- 451: Bump google.golang.org/protobuf from 1.27.1 to 1.33.0 Bumps google.golang.org/protobuf from 1.27.1 to 1.33.0.
- 450: /go.mod: bump go
- 449: Nathan/functional index
Adds a limited method for parsing indexes on functions.
Can now parse queries like
create index idx on tbl ((col1 + col2)) - 448: Move
SERIALtype out of numeric type definitions so thatunsignedvalue is not overwritten fixes dolthub/dolt#10345 MySQL docs