Results of 1 - 10 of about 12 for Bestpractice (0.001 sec.)
- Schema 13955
- Types It is important to use the correct type for encoding columns (e.g., BIGINT , DATE , DATETIME ). While it is always possible to use string types ( VARCHAR , etc.) to encode more specific values, this is not recommended. Strings use mor
...Join column schema type Example value Query time BIGINT BIGINT 70368755640078 1.2 s BIGINT VARCHAR '70368755640078' 2.1 s Bestpractice Avoid representing numeric values as strings, especially if you intend to perform e.g., join operations on them. Encoding...e NATURAL JOIN dim_table GROUP BY ALL ) SELECT rollup.* EXCLUDE (field_pk), field_name FROM rollup NATURAL JOIN dim_table; Bestpractice DuckDB already applies dictionary encoding to string columns in storage, so this technique mainly pays off for query-heavy...ce of queries as DuckDB does not rely on these indexes for join and aggregation operators (see indexing for more details). Bestpractice Do not define constraints unless your goal is to ensure data integrity. Microbenchmark: The Effect of Primary Keys We illu... - https://man.mtool.work/duckdb/2.0/guides/performance/schema.html
- File Formats 9816
- Handling Parquet Files DuckDB has advanced support for Parquet files, which includes directly querying Parquet files . When deciding on whether to query these files directly or to first load them to the database, you need to consider severa
.... Parquet The queries on the TPC-H benchmark run approximately 1.1-5.0× slower on Parquet files than on a DuckDB database. Bestpractice If you have the storage space available, and have a join-heavy workload and/or plan to run many queries on the same datase...10 row groups will achieve full parallelism. It is also beneficial to keep the size of individual Parquet files moderate. Bestpractice The ideal range is between 100 MB and 10 GB per individual Parquet file. Hive Partitioning for Filter Pushdown When queryi... - https://man.mtool.work/duckdb/2.0/guides/performance/file_formats.html
- Data Import 9816
- Recommended Import Methods When importing data from other systems to DuckDB, there are several considerations to take into account. We recommend importing using the following order: For systems which are supported by a DuckDB scanner extens
...rming row-by-row inserts (even with prepared statements) is detrimental to performance and will result in slow load times. Bestpractice Unless your data is small (<100k rows), avoid using inserts in loops.... - https://man.mtool.work/duckdb/2.0/guides/performance/import.html
- INSERT Statements 9551
- INSERT statements are the standard way of loading data into a relational database. When using INSERT statements, the values are supplied row-by-row. While simple, there is significant overhead involved in parsing and processing individual I
...NSERT statements. This makes lots of individual row-by-row insertions very inefficient for bulk insertion. Bestpractice As a rule-of-thumb, avoid using lots of individual row-by-row INSERT statements when inserting more than a few rows (i.e.,... - https://man.mtool.work/duckdb/2.0/data/insert.html
- Partitioned Writes 8171
- Examples Write a table to a Hive partitioned dataset of Parquet files: COPY orders TO 'orders' (FORMAT parquet, PARTITION_BY (year, month)); Write a table to a Hive partitioned dataset of CSV files, allowing overwrites: COPY orders TO 'orde
..., use the partitioned_write_max_open_files configuration option (default: 100): SET partitioned_write_max_open_files = 10; Bestpractice Writing data into many small partitions is expensive. It is generally recommended to have at least 100 MB of data per part... - https://man.mtool.work/duckdb/2.0/data/partitioning/partitioned_writes.html
- Environment 8171
- The environment where DuckDB is run has an obvious impact on performance. This page focuses on the effects of the hardware configuration and the operating system used. Hardware Configuration CPU DuckDB's officially supported architectures a
...other architecture such as LoongArch and RISC-V . However, there are no performance guarantees for these platforms. Memory Bestpractice Aim for 1-4 GB memory per thread. Minimum Required Memory As a rule of thumb, DuckDB requires a minimum of 125 MB of memor... - https://man.mtool.work/duckdb/2.0/guides/performance/environment.html
- Indexing 8171
- DuckDB has two types of indexes: zonemaps and ART indexes. Zonemaps DuckDB automatically creates zonemaps (also known as min-max indexes) for the columns of all general-purpose data types . Operations like predicate pushdown into scan opera
...he required parts of the index. Therefore, having an index will not cause any slowdowns when opening an existing database. Bestpractice We recommend following these guidelines: Only use primary keys, foreign keys, or unique constraints, if these are necessar... - https://man.mtool.work/duckdb/2.0/guides/performance/indexing.html
- SELECT Statement 8171
- The SELECT statement retrieves rows from the database. Examples Select all columns from the table tbl : SELECT * FROM tbl; Select the rows from tbl : SELECT j FROM tbl WHERE i = 3; Perform an aggregate grouped by the column i : SELECT i, su
...i)); DELETE FROM t WHERE i % 2 = 0; SELECT rowid FROM t; rowid 1 3 5 7 9 The rowid values are stable within a transaction. Bestpractice It is strongly advised to avoid using rowids as identifiers. If there is a user-defined column named rowid , it shadows th... - https://man.mtool.work/duckdb/2.0/sql/statements/select.html
- Samples 7888
- Samples are used to randomly select a subset of a dataset. Examples Select a sample of exactly 5 rows from tbl using reservoir sampling: SELECT * FROM tbl USING SAMPLE 5; Select a sample of approximately 10% of the table using system sampli
...nbiased sampling. This is not a big problem when the reservoir is very small, but becomes costly when the sample is large. Bestpractice Avoid using reservoir sampling with large sample sizes if possible. Reservoir sampling requires the entire sample to be ma... - https://man.mtool.work/duckdb/2.0/sql/samples.html
- Migrating to DuckDB 2.0 7623
- DuckDB 2.0.0 is packed with new features and performance improvements, including query optimization and asynchronous I/O. It also ships a small set of breaking changes, most notably a new default storage format, which DuckDB versions before
...extended support for DuckDB 1.5 until March 2027 . Note that DuckDB 1.4 LTS reaches its end of life on November 17, 2026. Bestpractice In all cases, we recommend setting up a shadow system that tracks your existing workload. This lets you test DuckDB 2.0.0... - https://man.mtool.work/duckdb/2.0/operations_manual/migration.html
DuckDB 2.0 Documentation / man.mtool.work