Results of 1 - 10 of about 16 for aggregation (0.003 sec.)
- Schema 14729
- Types It is important to use the correct type for encoding columns (e.g., BIGINT , DATE , DATETIME ). While it is always possible to use string types ( VARCHAR , etc.) to encode more specific values, this is not recommended. Strings use mor
...e space and are slower to process in operations such as filtering, join, and aggregation. When loading CSV files, you may leverage the CSV reader's auto-detection mechanism to determine the correct types for CSV...eating columns is necessary to reduce memory consumption. Microbenchmark: Using Timestamps We illustrate the difference in aggregation speed using the creationDate column of the LDBC Comment table on scale factor 300 . This table has approx. 554 million uno...rdered timestamp values. We run a simple aggregation query that returns the average day-of-the month from the timestamps in two configurations. First, we use a DATETIME to enc... - https://man.mtool.work/duckdb/2.0/guides/performance/schema.html
- File Formats 10431
- Handling Parquet Files DuckDB has advanced support for Parquet files, which includes directly querying Parquet files . When deciding on whether to query these files directly or to first load them to the database, you need to consider severa
...ions such as projection and filter pushdown on Parquet files. Therefore, workloads that combine projection, filtering, and aggregation tend to perform quite well when run on Parquet files. Storage considerations: Loading the data from Parquet files will req...how many row groups a Parquet file has. When writing Parquet files, use the row_group_size option. Microbenchmark: Running Aggregation Query at Different Row Group Sizes We run a simple aggregation query over Parquet files using different row group sizes, s... - https://man.mtool.work/duckdb/2.0/guides/performance/file_formats.html
- JSON Processing Functions 9010
- JSON Extraction Functions There are two extraction functions, which have their respective operators. The operators can only be used if the string is stored as the JSON logical type. These functions support the same two location notations as
...ree JSON aggregate functions. Function Description json_group_array(any) Return a JSON array with all values of any in the aggregation. json_group_object(key, value) Return a JSON object with all key , value pairs in the aggregation. json_group_structure(js...on) Return the combined json_structure of all json in the aggregation. Examples: CREATE TABLE example1 (k VARCHAR, v INTEGER); INSERT INTO example1 VALUES ('duck', 42), ('goose', 7); SELECT js... - https://man.mtool.work/duckdb/2.0/data/json/json_functions.html
- Relational API 8820
- The Relational API builds a query one step at a time from Python. A relation, a DuckDBPyRelation , is a piece of SQL text: the methods of the Transformation and Functions sections return a new relation whose text extends the one they were c
...her relation object in other_rel. filter Filter the relation object by the filter in filter_expr. group_by Begin a grouped aggregation; continue with .agg(...) . head The first n rows. intersect Create the set intersection of this relation object with anoth...────┴─────────┴──────────┴───────┴──────────┘ group_by group_by(self, *columns: object) -> GroupedRelation Begin a grouped aggregation; continue with .agg(...) . group_by() returns a grouped relation whose only method is agg(*aggregates) . agg() reads its a...of all bits present in a given expression. Parameters expression : str The column or expression to perform the bitwise AND aggregation on. * groups : str, default: '' Comma-separated list of columns to include in the GROUP BY ; cannot be combined with windo... - https://man.mtool.work/duckdb/2.0/clients/python/relational_api.html
- Order Preservation 8646
- For many operations, DuckDB preserves the order of rows, similarly to data frame libraries such as Pandas. Example Take the following table for example: CREATE TABLE tbl AS SELECT * FROM (VALUES (1, 'a'), (2, 'b'), (3, 'c')) t(x, y); SELECT
...guarantee that the row order is preserved: FROM with multiple tables and/or subqueries JOIN UNION USING SAMPLE Whole-table aggregation (the input order, that is, the order in which rows are fed into order-sensitive aggregate functions is not guaranteed unle... - https://man.mtool.work/duckdb/2.0/sql/dialect/order_preservation.html
- Unofficial and Unsupported Platforms 8473
- Warning The platforms listed on this page are not officially supported. The build instructions are provided on a best-effort basis. Community contributions are very welcome. DuckDB is built and distributed for several platforms with differe
...message;” ┌───────────────────┐ │ message │ │ varchar │ ├───────────────────┤ │ Hello from RISC-V │ └───────────────────┘ Aggregation queries work as expected: ./duckdb -c “CREATE TABLE test AS SELECT range AS id, range * 3.14 AS value FROM range(1000); SE... - https://man.mtool.work/duckdb/2.0/dev/building/unofficial_and_unsupported_platfo...
- Environment 8283
- The environment where DuckDB is run has an obvious impact on performance. This page focuses on the effects of the hardware configuration and the operating system used. Hardware Configuration CPU DuckDB's officially supported architectures a
...intermediate datasets and thus require more memory for their evaluation to fully fit into the memory. As an approximation, aggregation-heavy workloads require 1-2 GB memory per thread and join-heavy workloads require 3-4 GB memory per thread. Larger-than-Me... - https://man.mtool.work/duckdb/2.0/guides/performance/environment.html
- Indexing 8283
- DuckDB has two types of indexes: zonemaps and ART indexes. Zonemaps DuckDB automatically creates zonemaps (also known as min-max indexes) for the columns of all general-purpose data types . Operations like predicate pushdown into scan opera
...tors and computing aggregations use zonemaps. If a filter criterion (like WHERE column1 = 123 ) is in use, DuckDB can skip any row group whose min-max ra...he performance of highly selective queries using the indexed column. An ART index does not affect the performance of join, aggregation, and sorting queries. ART Index Scans ART index scans probe a single-column ART index for the requested data instead of sc... - https://man.mtool.work/duckdb/2.0/guides/performance/indexing.html
- Friendly SQL 8109
- DuckDB offers several advanced SQL features and syntactic sugar to make SQL queries more concise. We refer to these colloquially as “friendly SQL”. Several of these features were first introduced by DuckDB, while some are inspired by other
...olumn aliases (also known as “lateral column aliases”), e.g.: SELECT i + 1 AS j, j + 2 AS k FROM range(0, 3) t(i) Advanced aggregation features for analytical (OLAP) queries: FILTER clause GROUPING SETS , GROUP BY CUBE , GROUP BY ROLLUP clauses count() shor... - https://man.mtool.work/duckdb/2.0/sql/dialect/friendly_sql.html
- Tuning Workloads 7919
- DuckDB aims to perform well out of the box, so most workloads do not require any tuning. When tuning is worthwhile, there is no single configuration or magic number that is optimal for every workload: the right settings depend on what you a
...o not support offloading to disk. Aggregate functions that use sorting are holistic, i.e., they need all inputs before the aggregation can start. As DuckDB cannot yet offload some complex intermediate aggregate states to disk, these functions can cause an o... - https://man.mtool.work/duckdb/2.0/guides/performance/how_to_tune_workloads.html
DuckDB 2.0 Documentation / man.mtool.work