ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
Databases & messaging · #duckdb
DuckDB's built-in Parquet extension lets you read and write columnar files directly from SQL with automatic predicate push-down and partitioning—no Spark cluster required. Here's the practical map for engineers moving data in pipelines.
When DuckDB queries fail with out-of-memory errors or sudden slowdowns from external sorting, this guide walks you through recognizing the condition, diagnosing with EXPLAIN ANALYZE, and applying targeted fixes.
Choose between DuckDB persistent files and in-memory mode based on durability, dataset size, and concurrency constraints — with a runnable script to validate the behavior.
Learn how DuckDB lets you read, transform, and write Parquet files directly from SQL without extra services or complex setup.
Learn how to use DuckDB's zero-copy Arrow integration to query Pandas DataFrames using SQL without the memory overhead of data duplication.
Learn how DuckDB can query Parquet files directly, pushing down predicates and pruning columns for fast analytics without ETL.
Learn how DuckDB's ATTACH command lets you query multiple .duckdb files as if they were schemas, with a step‑by‑step example, limits, and common mistakes to avoid.
Learn how DuckDB automatically prunes unneeded columns from Parquet files, see a concrete query example, verify the plan with EXPLAIN, and understand the limits of this optimization.
DuckDB’s zero‑copy Parquet reader lets you query terabyte‑scale datasets in Python with minimal memory, using column pruning and vectorised execution. This post walks through a practical example, shows how to verify the benefit, and discusses trade‑offs.
pandas