What is DuckDB?

DuckDB is a relational (table-oriented) DBMS that supports the Structured Query Language (SQL).

DuckDB is designed to support analytical query workloads, also known as Online analytical processing (OLAP). These workloads are characterized by complex, relatively long-running queries that process significant portions of the stored dataset, for example aggregations over entire tables or joins between several large tables. Changes to the data are expected to be rather large-scale as well, with several rows being appended, or large portions of tables being changed or added at the same time.

DuckDB has no external dependencies, neither for compilation nor during run-time. For releases, the entire source tree of DuckDB is compiled into two files, a header and an implementation file, a so-called “amalgamation”. This greatly simplifies deployment and integration in other build processes. For building, all that is required to build DuckDB is a working C++11 compiler.

For DuckDB, there is no DBMS server software to install, update and maintain. DuckDB does not run as a separate process, but completely embedded within a host process. For the analytical use cases that DuckDB targets, this has the additional advantage of high-speed data transfer to and from the database. In some cases, DuckDB can process foreign data without copying. For example, the DuckDB Python package can run queries directly on Pandas data without ever importing or copying any data.

Features

  1. Open-source and free
  2. Developer-friendly documentation
  3. Transactions, persistence
  4. Extensive SQL support
  5. Direct Parquet & CSV querying
  6. In-process, serverless
  7. C++11, no dependencies, single file build
  8. Fast and extremely lightweight
  9. Optimized for analytics
  10. Processing and storing tabular datasets, e.g. from CSV or Parquet files
  11. Interactive data analysis, e.g. Joining & aggregate multiple large tables
  12. Concurrent large changes, to multiple large tables, e.g. appending rows, adding/removing/updating columns
  13. Large result set transfer to client
  14. Parallel query processing
  15. Built-in API
  16. Several native clients for Python, Java, R, C++, C, Node.js, WASM, and a CLI app
  17. Built-in bulk-optimized Multi-Version Concurrency Control (MVCC).
  18. CPU optimized
  19. Supports OLAP queries

When to not use DuckDB?

  • High-volume transactional use cases (e.g., tracking orders in a webshop)
  • Large client/server installations for centralized enterprise data warehousing
  • Writing to a single database from multiple concurrent processes

License

The project is released under the MIT License.

Resources