Results of 1 - 10 of about 16 for efficiently (0.001 sec.)
- Importing Data 14989
- The first step to using a database system is to insert data into that system. DuckDB can directly connect to many popular data sources and offers several data ingestion methods that allow you to easily and efficiently fill up the database.
...earch_path is not set, the working directory is used as the basis for relative paths. File Formats CSV Loading Data can be efficiently loaded from CSV files using several methods. The simplest is to use the CSV file's name: SELECT * FROM 'test.csv'; Alterna...BLE test AS SELECT * FROM 'test.csv'; For more details, see the page on CSV loading . Parquet Loading Parquet files can be efficiently loaded and queried using their filename: SELECT * FROM 'test.parquet'; Alternatively, use the read_parquet function : SELE...atement : COPY tbl FROM 'test.parquet'; For more details, see the page on Parquet loading . JSON Loading JSON files can be efficiently loaded and queried using their filename: SELECT * FROM 'test.json'; Alternatively, use the read_json function : SELECT * F... - https://man.mtool.work/duckdb/2.0/data/overview.html
- Constraints 9530
- In SQL, constraints can be specified for tables. Constraints enforce certain properties over data that is inserted into a table. Constraints can be specified along with the schema of the table as part of the CREATE TABLE statement . In cert
...ent 1'); Constraint Error: Duplicate key "id: 1, name: Student 1" violates primary key constraint To enforce this property efficiently, an ART index is automatically created for every primary key or unique constraint that is defined in the table. Primary ke...Error: Violates foreign key constraint because key "id: 2" does not exist in the referenced table To enforce this property efficiently, an ART index is automatically created for every foreign key constraint that is defined in the table. Warning Indexes have... - https://man.mtool.work/duckdb/2.0/sql/constraints.html
- Execution Format 9340
- Vector is the container format used to store in-memory data during execution. DataChunk is a collection of Vectors, used for instance to represent a column list in a PhysicalProjection operator. Data Flow DuckDB uses a vectorized query exec
...quence Vectors Sequence vectors are physically stored as an offset and an increment value. Sequence vectors are useful for efficiently storing incremental sequences. They are generally emitted for row identifiers. Unified Vector Format These properties of t...ric view over the contents of the Vector. Every type of Vector can convert to this format. Complex Types String Vectors To efficiently store strings, we make use of our string_t class. struct string_t { union { struct { uint32_t length; char prefix[4]; char... - https://man.mtool.work/duckdb/2.0/internals/vector.html
- Friendly SQL 8976
- DuckDB offers several advanced SQL features and syntactic sugar to make SQL queries more concise. We refer to these colloquially as “friendly SQL”. Several of these features were first introduced by DuckDB, while some are inspired by other
...s max(arg, n) , min(arg, n) , arg_max(arg, val, n) , arg_min(arg, val, n) , max_by(arg, val, n) and min_by(arg, val, n) to efficiently return the "top" n rows in a group based on a specific column in either ascending or descending order. For example, let's...│ [3, 2, 1] │ a │ │ [6, 5, 4] │ b │ └───────────────────┴─────────┘ But in DuckDB, we can do this much more concisely (and efficiently!): SELECT max(val, 3) FROM t1 GROUP BY grp; ┌─────────────┐ │ max(val, 3) │ │ int32[] │ ├─────────────┤ │ [3, 2, 1] │ │ [6... - https://man.mtool.work/duckdb/2.0/sql/dialect/friendly_sql.html
- Loops 8404
- Loops can be used in sqllogictests when it is required to execute the same query many times but with slight modifications in constant values. For example, suppose we want to fire off 100 queries that check for the presence of the values 0..
...e two functions, together with clever use of cross products and other expressions, many different types of datasets can be efficiently generated. The random() function can also be used to generate random data. An alternative option is to read data from an e... - https://man.mtool.work/duckdb/2.0/dev/sqllogictest/loops.html
- Hugging Face Support 8213
- The httpfs extension introduces support for the hf:// protocol to access datasets hosted in Hugging Face repositories. See the announcement blog post for details. Usage Hugging Face repositories can be queried using the following URL patter
...le: SELECT count(*) AS count FROM 'hf://datasets/cais/mmlu/astronomy/*.parquet'; count 173 By using glob patterns, you can efficiently handle large datasets and perform comprehensive queries across multiple files, simplifying your data inspections and proce... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/hugging_face.html
- Guides 8213
- The guides section contains compact how-to guides that are focused on achieving a single goal. For API references and examples, see the rest of the documentation. Note that there are many tools using DuckDB, which are not covered in the off
...ide) How to design the schema for optimal performance What is the ideal hardware environment for DuckDB How to import data efficiently How to use indexes and zonemaps How to make join operations efficient What performance implications do Parquet files and (... - https://man.mtool.work/duckdb/2.0/guides/overview.html
- Variant Type 8213
- The VARIANT type stores typed, binary data where each row is self-contained with its own type information. This differs from the JSON type , which is physically stored as text. Because type metadata is embedded per-value, VARIANT provides b
...em, i.e., stores the fields that share a type as separate, typed columns. Shredded data compresses well and can be scanned efficiently. Shredding is only applied to row groups with at least variant_minimum_shredding_size rows (by default, 30,000 rows, rough... - https://man.mtool.work/duckdb/2.0/sql/data_types/variant.html
- DuckDB Wasm Client 8040
- Installation To use the DuckDB Wasm client, visit the duckdb-wasm GitHub repository . The latest stable version of the DuckDB WebAssembly client is 2.0.0. DuckDB-Wasm is a version of DuckDB that has been compiled to WebAssembly , the portab
...r, database-friendly format that the apache-arrow npm package implements in the browser, letting DuckDB-Wasm exchange data efficiently and interoperate with other JavaScript data tools without reimplementing SQL type logic in JavaScript. DuckDB-Wasm is buil... - https://man.mtool.work/duckdb/2.0/clients/wasm/overview.html
- Environment 8040
- The environment where DuckDB is run has an obvious impact on performance. This page focuses on the effects of the hardware configuration and the operating system used. Hardware Configuration CPU DuckDB's officially supported architectures a
...re AMD64 (x86_64) and ARM64 (AArch64) CPU architectures. DuckDB works efficiently on both of these architectures. DuckDB can be compiled to other architecture such as LoongArch and RISC-V . However, there... - https://man.mtool.work/duckdb/2.0/guides/performance/environment.html
DuckDB 2.0 Documentation / man.mtool.work