ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
Databases & messaging · #duckdb
Learn how DuckDB pushes WHERE predicates and column projections into Parquet scans, reducing I/O without any ETL step.
Discover how DuckDB lets you query remote Parquet files directly, reducing data egress and speeding up analytics with zero‑copy, predicate pushdown, and an in‑process columnar engine.
In a data ingestion pipeline, I need to insert rows into a DuckDB table using a simple INSERT statement. Occasionally, the operation fails due to a transient issue such as a locked file or a network glitch, and I want to retry the INSERT automatically. However, if the first attempt succeeded but the failure occurred after the commit, a naive retry would inse
Goal Implement reliable pagination for large result sets using DuckDB’s LIMIT and OFFSET clauses. Constraints DuckDB’s optional query cache can store the results of a SELECT to accelerate repeated executions. The cache is not documented to automatically invalidate entries when the underlying table is modified, which may lead to stale pages being returned aft
Teams that share a single DuckDB database file across Python, R, and CLI environments face a storage-format compatibility policy decision. Because DuckDB is embedded, each environment bundles its own engine version. The v1.0.0 release is documented to introduce a stable storage format commitment for files created by that version and later, but opening a file
I'm designing a schema migration for a DuckDB database (assuming a recent stable release, e.g. 1.x) and need to add a non-nullable-style column to a large existing table. My understanding from the documentation is that ALTER TABLE t ADD COLUMN c INTEGER DEFAULT 0 records the default as schema metadata, but existing rows still read as NULL until an explicit U
DuckDB’s ConnectionManager enforces a ceiling on concurrent Connection objects. When a new connection is requested while active connections equal the configured max_connections value, the error Connection limit exceeded is raised. The default is commonly 1, though it can be changed with a SET statement. What remains unclear is whether this limit is advisory
Setup I am running DuckDB (assume v0.10.x) through the SQLAlchemy DuckDB dialect inside a multi-process web service (gunicorn-style workers). Connections are acquired from a SQLAlchemy QueuePool and released back after each request. Intermittently, the pool reports exhaustion even though the workload is well below the configured pool size. Constraints and un
VARCHAR(n) Length Compatibility Boundary When Migrating to DuckDB Goal: Preserve declared VARCHAR(n) maximums as database-enforced limits when porting a schema from PostgreSQL or MySQL into DuckDB. DuckDB treats VARCHAR(n), VARCHAR, and TEXT as synonymous unbounded string types. The optional n is not enforced during INSERT or CAST, and PRIMARY KEY, UNIQUE, o
sqlalchemy