Skip to main content

DuckDB integration

DuckDB is an open-source, in-process SQL OLAP database management system designed for fast analytical query workloads.

Windmill supports seamless integration with DuckDB, allowing you to manipulate data from S3 (csv, parquet, json), Azure Blob Storage, BigQuery, PostgreSQL, and MySQL.

DuckDB in Windmill supports automatic column detection on S3 objects. You can query S3 paths directly without wrapping them in read_parquet() — for example SELECT col1, col2 FROM 's3:///file.parquet' — and the SQL parser will infer the referenced columns. The standard read_parquet(), read_csv(), and read_json() table functions also support column detection when used with S3 paths.


Integration between DuckDB and Windmill

Azure Blob Storage support​

DuckDB scripts in Windmill can read from and write to Azure Blob Storage. When Azure Blob is configured as a workspace storage, DuckDB can use the same storage paths to query and write data in Parquet, CSV, or JSON format.

This works with the same S3-compatible path syntax, and requires an Azure Blob storage resource to be configured in the workspace.

To get started, check out the SQL Getting Started section.