Results of 1 - 10 of about 34 for Groups (0.005 sec.)
- Relational API 13453
- The Relational API builds a query one step at a time from Python. A relation, a DuckDBPyRelation , is a piece of SQL text: the methods of the Transformation and Functions sections return a new relation whose text extends the one they were c
...a new query from the relation and run nothing. Name Description aggregate Compute the aggregate aggr_expr by the optional groups group_expr on the relation. apply Compute the function of a single column or a list of columns by the optional groups on t...gate(self, aggr_expr: object, group_expr: object = '') -> DuckDBPyRelation Compute the aggregate aggr_expr by the optional groups group_expr on the relation. Parameters aggr_expr : str, list[Expression] The columns and aggregates to compute: SQL text,...name. * group_expr : str, Expression, None, default: '' The columns to group by, as SQL text or an Expression. '' or None groups by every column of aggr_expr that is not an aggregate ( GROUP BY ALL ). Example import duckdb duckdb_conn = duckdb.connect... - https://man.mtool.work/duckdb/2.0/clients/python/relational_api.html
- Parquet Tips 12702
- Below is a collection of tips to help when dealing with Parquet files. Tips for Reading Parquet Files Use union_by_name When Loading Files with Different Schemas The union_by_name option can be used to unify the schema of files that have di
...column chunk for each column in the dataset. Compression algorithms are only applied per row group, so the larger the row group size, the more opportunities to compress the data. On the other hand, larger row group sizes mean that each thread keeps mo...re data in memory before flushing when streaming results. Another argument for smaller row group sizes is that DuckDB can read Parquet row groups in parallel even within the same file and uses predicate pushdown to only...scan the row groups whose metadata ranges match the WHERE clause of the query. However, there is some overhead associated with reading the met... - https://man.mtool.work/duckdb/2.0/data/parquet/tips.html
- Metrics 11280
- DuckDB provides a set of metrics that can be used to monitor the performance and health of the database. Metrics are named ⟨group⟩.⟨metric⟩ , e.g., query.total_time or operator.timing . The set of collected metrics is controlled with the tr
...T metric_name, metric_type, unit FROM duckdb_available_metrics(); All Metrics The metrics are organized into the following groups: query system io storage operator optimizer parser planner physical_planner query Metrics Metrics measured over the entire...mediate_size_bytes uint64 bytes Cumulative size in bytes of all intermediate results produced by the query query.total_row_groups_scanned uint64 row_groups Cumulative number of row groups scanned by the query query.total_row_groups_to_scan uint64 row_g...roups Cumulative total number of row groups available to scan query.total_rows_scanned uint64 rows Cumulative number of rows scanned by the query query.total_time dou... - https://man.mtool.work/duckdb/2.0/dev/metrics.html
- Python Client API 10321
- Connection class duckdb. Appender Bases: object append_row ( self , * args ) → None Buffer one row, one value per column, each converted as an executemany parameter would be append_rows ( self , rows : object ) → None Buffer each row of an
...egate ( aggr_expr : object , group_expr : object = '' ) → DuckDBPyRelation Compute the aggregate aggr_expr by the optional groups group_expr on the relation. Detailed examples can be found at Relational API page . alias Get the name of the current alia...s. Detailed examples can be found at Relational API page . any_value ( expression : str , groups : str = '' , window_spec : str = '' , projected_columns : str = '' ) → DuckDBPyRelation Returns the first non-null value f...ected_columns : str = '' ) → DuckDBPyRelation Compute the function of a single column or a list of columns by the optional groups on the relation. Detailed examples can be found at Relational API page . arg_max ( arg_column : str , value_column : str ,... - https://man.mtool.work/duckdb/2.0/clients/python/reference/
- Regular Expressions 9522
- DuckDB offers pattern matching operators ( LIKE , SIMILAR TO , GLOB ), as well as support for regular expressions via functions. Regular Expression Syntax DuckDB uses the RE2 library as its regular expression engine. For the regular express
...tion regexp_extract(string, pattern[, group = 0][, options]) If string contains the regexp pattern , returns the capturing group specified by optional parameter group ; otherwise, returns the empty string. The group must be a constant value. If no grou...e set. regexp_extract(string, pattern, name_list[, options]) If string contains the regexp pattern , returns the capturing groups as a struct with corresponding names from name_list ; otherwise, returns a struct with the same keys and empty strings as...ct_all(string, regex, name_list[, options]) Finds non-overlapping occurrences of regex in string and returns the capturing groups as a list of structs with corresponding names from name_list . regexp_full_match(string, regex[, options]) Returns true if... - https://man.mtool.work/duckdb/2.0/sql/functions/regular_expressions.html
- File Formats 8611
- Handling Parquet Files DuckDB has advanced support for Parquet files, which includes directly querying Parquet files . When deciding on whether to query these files directly or to first load them to the database, you need to consider severa
...un many queries on the same dataset, load the Parquet files into the database first. The compression algorithm and the row group sizes in the Parquet files have a large effect on performance: study these using the parquet_metadata function . The Effect...of Row Group Sizes DuckDB works best on Parquet files with row groups of 100K-1M rows each. The reason for this is that DuckDB can only...parallelize over row groups – so if a Parquet file has a single giant row group it can only be processed by a single thread. You can use the parquet_m... - https://man.mtool.work/duckdb/2.0/guides/performance/file_formats.html
- EXPLAIN ANALYZE: Profile Queries 8212
- Prepending a query with EXPLAIN ANALYZE both pretty-prints the query plan, and executes it, providing run-time performance numbers and the actual cardinality (number of rows produced) for every operator. EXPLAIN ANALYZE SELECT * FROM tbl; N
...d, name │ │ Dynamic Filters: │ │ Filters: name >= 'Ma' │ │ optional: (sid IN (1, 3)) AND optional: sid IN PRF(sid) │ │ Row Groups Scanned: 1 / 1 │ │ Row Groups Scanned: 1 / 1 │ │ 2 rows 0µs │ │ 2 rows 0µs │ ╰─────────────────────────────╯ ╰────────────...─╯ Each operator shows the number of rows it produced and the time spent in it. Scan operators also show the number of row groups scanned and any dynamic filters that were pushed into them at runtime, e.g., by the hash join. See Also For more informati... - https://man.mtool.work/duckdb/2.0/guides/meta/explain_analyze.html
- Window Functions 8116
- DuckDB supports window functions , which can use multiple rows to calculate a value for each row. Window functions are blocking operators , i.e., they require their entire input to be buffered, making them one of the most memory-intensive o
...with current row) / total partition rows. dense_rank() The rank of the current row without gaps; this function counts peer groups. fill(expr [ ORDER BY ordering]) Fill in missing values using linear interpolation with ORDER BY as the X-axis. first_valu...| Example | cume_dist() | dense_rank() | Description | The rank of the current row without gaps; this function counts peer groups. | | Return type | BIGINT | | Example | dense_rank() | | Aliases | rank_dense() | fill(expr[ ORDER BY ordering]) | Descrip...pecified as a number of rows , as a range of values using the partition's ordering value and a distance, or as a number of groups (sets of rows with the same sort value). The full syntax is shown in the diagram at the top of the page, and this diagram... - https://man.mtool.work/duckdb/2.0/sql/functions/window_functions.html
- CHECKPOINT Statement 7813
- The CHECKPOINT statement synchronizes data in the write-ahead log (WAL) to the database data file. Examples Synchronize data in the default database: CHECKPOINT; Synchronize data in the specified database: CHECKPOINT file_db; Synchronize da
...ce occupied by deleted rows is partially reclaimed. Note that this does not remove all deleted rows, but rather merges row groups that have a significant amount of deletes together. In the current implementation this requires ~25% of rows to be deleted...in adjacent row groups. Since v1.4.0, checkpointing also applies to in-memory databases (see Checkpointing In-Memory Tables ), where it likewise... - https://man.mtool.work/duckdb/2.0/sql/statements/checkpoint.html
- Querying Parquet Metadata 7749
- Parquet Metadata The parquet_metadata function can be used to query the metadata contained within a Parquet file, which reveals various internal details of the Parquet file such as the statistics of the different columns. This can be useful
...of the columns returned by parquet_file_metadata . Field Type file_name VARCHAR created_by VARCHAR num_rows BIGINT num_row_groups BIGINT format_version BIGINT encryption_algorithm VARCHAR footer_signing_key_metadata VARCHAR file_size_bytes UBIGINT foot...the same columns as the corresponding standalone function. Bloom Filters DuckDB supports Bloom filters for pruning the row groups that need to be read to answer highly selective queries. Currently, Bloom filters are supported for the following types: I...ng point types: FLOAT , DOUBLE VARCHAR BLOB The parquet_bloom_probe(filename, column_name, value) function shows which row groups can be excluded when filtering for a given value of a given column using the Bloom filter. For example: FROM parquet_bloom... - https://man.mtool.work/duckdb/2.0/data/parquet/metadata.html
DuckDB 2.0 Documentation / man.mtool.work