Results of 1 - 9 of about 9 for algorithm (0.002 sec.)
- Join Operations 13858
- How to Force a Join Order DuckDB has a cost-based query optimizer, which uses statistics in the base tables (stored in a DuckDB database or Parquet files) to estimate the cardinality of operations. Turn off the Join Order Optimizer To turn
...the Join Order Optimizer DuckDB's join order optimizer computes the optimal join order exactly using a dynamic programming algorithm. As the number of tables in a join grows, exact enumeration becomes expensive, so for large joins the optimizer switches t...o an approximate (greedy) algorithm. The approximate_join_order_threshold setting controls when this happens: joins with at least this many tables are ordered...approximately, while joins with fewer tables are ordered exactly. The default value is 12. For example, to use the exact algorithm for joins of up to 15 tables, run: SET approximate_join_order_threshold = 16; Raising the threshold can lead to better joi... - https://man.mtool.work/duckdb/2.0/guides/performance/join_operations.html
- Querying Parquet Metadata 9185
- Parquet Metadata The parquet_metadata function can be used to query the metadata contained within a Parquet file, which reveals various internal details of the Parquet file such as the statistics of the different columns. This can be useful
...The parquet_file_metadata function can be used to query file-level metadata such as the format version and the encryption algorithm used: SELECT * FROM parquet_file_metadata('test.parquet'); Below is a table of the columns returned by parquet_file_metada...a . Field Type file_name VARCHAR created_by VARCHAR num_rows BIGINT num_row_groups BIGINT format_version BIGINT encryption_algorithm VARCHAR footer_signing_key_metadata VARCHAR file_size_bytes UBIGINT footer_size UBIGINT column_orders VARCHAR[] Parquet Ke... - https://man.mtool.work/duckdb/2.0/data/parquet/metadata.html
- File Formats 9023
- Handling Parquet Files DuckDB has advanced support for Parquet files, which includes directly querying Parquet files . When deciding on whether to query these files directly or to first load them to the database, you need to consider severa
...amortizes the initial load time. High decompression times: Some Parquet files are compressed using heavyweight compression algorithms such as gzip. In these cases, querying the Parquet files will necessitate an expensive decompression time every time the...ike Snappy, LZ4, and zstd, are faster to decompress. You may use the parquet_metadata function to find out the compression algorithm used. Microbenchmark: Running TPC-H on a DuckDB Database vs. Parquet The queries on the TPC-H benchmark run approximately...kload and/or plan to run many queries on the same dataset, load the Parquet files into the database first. The compression algorithm and the row group sizes in the Parquet files have a large effect on performance: study these using the parquet_metadata fu... - https://man.mtool.work/duckdb/2.0/guides/performance/file_formats.html
- R-Tree Indexes 8556
- The spatial extension provides support for spatial indexing through the R-tree extension index type. Why Should I Use an R-Tree Index? When working with geospatial datasets, it is very common that you want to filter rows based on their spat
...de up the tree. However, when the R-tree index is created on an already populated table, a special bottom up "bulk loading algorithm" (Sort-Tile-Recursive) is used, which divides all entries into an already balanced tree as the total number of required no...des can be computed from the beginning. Additionally, using the bulk loading algorithm tends to create a R-tree with a better structure (less overlap between bounding boxes), which usually leads to better quer... - https://man.mtool.work/duckdb/2.0/core_extensions/spatial/r-tree_indexes.html
- Overview of DuckDB Internals 8394
- Tip For a detailed explanation of DuckDB internals, see the Design and Implementation of DuckDB Internals ("DiDi") library entry. On this page is a brief description of the internals of the DuckDB engine. Parser The parser converts a query
...at statically evaluate to false). Join Order Optimizer : Reorders joins using dynamic programming. Specifically, the DPhyp algorithm from the paper Dynamic Programming Strikes Back is used. Common Sub Expressions : Extracts common subexpressions from proj... - https://man.mtool.work/duckdb/2.0/internals/overview.html
- Order Preservation 8394
- For many operations, DuckDB preserves the order of rows, similarly to data frame libraries such as Pandas. Example Take the following table for example: CREATE TABLE tbl AS SELECT * FROM (VALUES (1, 'a'), (2, 'b'), (3, 'c')) t(x, y); SELECT
...ate function) GROUP BY (neither in- nor output order are guaranteed) ORDER BY (specifically, ORDER BY may not use a stable algorithm ) Scalar subqueries Insertion Order By default, the following components preserve insertion order: CSV reader ( read_csv f... - https://man.mtool.work/duckdb/2.0/sql/dialect/order_preservation.html
- WITH Clause 8070
- The WITH clause allows you to specify common table expressions (CTEs). Regular (non-recursive) common-table-expressions are essentially views that are limited in scope to a particular query. CTEs can reference each other and can be nested.
...the union table built so far . References to recurring.T allow for the elegant and idiomatic translation of rather complex algorithms into readable SQL code. Example: USING KEY This is a recursive CTE where USING KEY has a key column ( a ) and a payload c...This allows you to use results from not just the previous iteration, but also earlier ones. This new feature makes certain algorithms easier to implement. One example is the connected components algorithm. For each node, the algorithm determines the node... - https://man.mtool.work/duckdb/2.0/sql/query_syntax/with.html
- Aggregate Functions 7747
- Examples Produce a single row containing the sum of the amount column: SELECT sum(amount) FROM sales; Produce one row per unique region, containing the sum of amount for each group: SELECT region, sum(amount) FROM sales GROUP BY region; Ret
...Downsamples the series of (x, y) points to at most n representative points using the Largest-Triangle-Three-Buckets (LTTB) algorithm. This function is affected by ordering . max(arg) Returns the maximum value present in arg . This function is unaffected b...Downsamples the series of (x, y) points to at most n representative points using the Largest-Triangle-Three-Buckets (LTTB) algorithm, which preserves the visual shape of the series. Returns a LIST of STRUCT s with fields x and y , always including the fir... - https://man.mtool.work/duckdb/2.0/sql/functions/aggregates.html
- Reading and Writing Parquet Files 7585
- Examples Read a single Parquet file: SELECT * FROM 'test.parquet'; Figure out which columns/types are in a Parquet file: DESCRIBE SELECT * FROM 'test.parquet'; Create a table from a Parquet file: CREATE TABLE test AS SELECT * FROM 'test.par
...he Parquet file scans. Parquet datasets differ based on the number of files, the size of individual files, the compression algorithm used, row group size, etc. These have a significant effect on performance. Please consult the Performance Guide for detail... - https://man.mtool.work/duckdb/2.0/data/parquet/overview.html
PREV
1
NEXT
DuckDB 2.0 Documentation / man.mtool.work