DuckDB
DuckDB is an open-source column-oriented Relational Database Management System. It is designed to provide high performance on complex queries against large databases in embedded configuration, such as combining tables with hundreds of columns and billions of rows. Unlike other embedded databases DuckDB is not focusing on transactional applications and instead is specialized for online analytical processing workloads. The project has over 6 million downloads per month.
History
DuckDB was originally developed by Mark Raasveldt and at the Centrum Wiskunde & Informatica in the Netherlands. The project co-founders designed DuckDB to address the need for an in-process OLAP database solution. DuckDB was first released in 2019. DuckDB version 1.0.0 was released on June 3, 2024, under the codename SnowDuck.Features
DuckDB uses a vectorized query processing engine. DuckDB is special amongst database management systems because it does not have any external dependencies and can be built with just a C++11 compiler. DuckDB also deviates from the traditional client–server model by running inside a host process. DuckDB's SQL parser is derived from the pg_query library developed by Lukas Fittl, which is itself derived from PostgreSQL's SQL parser that has been stripped down as much as possible. DuckDB uses a single-file storage format to store data ondisk, designed to support efficient scans
and bulk updates, appends and deletes. DuckDB is also compiled to WebAssembly using emscripten which enables DuckDB to run SQL in browser-based analytics tools.
Comparison
DuckDB in its OLAP niche does not compete with the traditional DBMS like MSSQL, PostgreSQL and Oracle database. While using SQL for queries, DuckDB targets serverless applications and provides extremely fast responses using either Apache Parquet files or its own format for storage. These attributes make it a popular choice for large dataset analysis in interactive mode.Commercial use
DuckDB is used at Facebook, Google, and Airbnb.DuckDB co-author Mühleisen also runs a support and consultancy firm for the software, DuckDB Labs. The company has chosen not to take venture capital funding, stating "We feel investment would force the project direction towards monetization, and we would much prefer keeping DuckDB open and available for as many people as possible". Another company, MotherDuck, has received $100 million funding for its data platform based on DuckDB, with investors including Andreessen Horowitz.
DuckDB Foundation
The independent non-profit DuckDB Foundation safeguards the long-term maintenance and development of DuckDB. The foundation holds much of the intellectual property of the project and is funded by charitable donations. The DuckDB Foundation's statutes ensure DuckDB remains open-source under the MIT license in perpetuity.Language support
In addition to the native C and C++ APIs, DuckDB supports a range of programming languages.| Language | Notes | Reference |
| Java | The is implemented using JNI. Integration with the Apache Arrow format is provided. | |
| Python | The implements support for the Pandas, Apache Arrow and Polars data analysis packages. | |
| Rust | The is distributed as a that exposes an elegant wrapper over the native C API. | |
| Node.JS | ||
| R | ||
| Julia | ||
| Swift | ||
| WebAssembly | ||
| Go |