Results of 1 - 10 of about 22 for storing (0.001 sec.)
- Variant Type 13433
- The VARIANT type stores typed, binary data where each row is self-contained with its own type information. This differs from the JSON type , which is physically stored as text. Because type metadata is embedded per-value, VARIANT provides b
...uctured VARIANT data type . It is available in Parquet since 2025 and also supported by DuckDB's Parquet reader . Examples Storing Different Types in the Same Column A VARIANT column can hold values of different types across rows: CREATE TABLE events (i...──────┤ │ Alice │ └─────────┘ Storage New Shredding VARIANT columns in DuckDB's storage was introduced in DuckDB 2.0. When storing VARIANT columns in a DuckDB database file, DuckDB 2.0 automatically detects the common structure of the values and shreds... - https://man.mtool.work/duckdb/2.0/sql/data_types/variant.html
- Working with Huge Databases 11127
- This page contains information for working with huge DuckDB database files. While most DuckDB databases are well below 1 TB, in our 2024 user survey , 1% of respondents used DuckDB files of 2 TB or more (corresponding to roughly 10 TB of CS
...gnificant effect on performance when working with large files. On Linux, DuckDB performs best with XFS on large files. For storing large amounts of data, consider using the DuckLake lakehouse format . Because DuckLake spreads the data across many immuta... - https://man.mtool.work/duckdb/2.0/guides/performance/working_with_huge_databases...
- PostgreSQL Extension and the Secret Manager 10555
- User credentials and other PostgreSQL database connection details can be stored using the DuckDB Secrets Manager . The following syntax can be used to create a secret: CREATE SECRET ( TYPE postgres, HOST '127.0.0.1', PORT 5432, DATABASE pos
...nfigure AWS Credential Chain the same way as with the secret of type s3 , see details in the AWS extension documentation . Storing Secrets inside a PostgreSQL Database DuckDB Secrets Manager supports pluggable Storage Providers. The postgres extension i...mplements storing the secrets (of any type) as records in the PostgreSQL database table. The following example initializes the secrets stora... - https://man.mtool.work/duckdb/2.0/core_extensions/postgres/secrets.html
- Lakehouse Formats 9982
- Lakehouse formats, often referred to as open table formats, are specifications for storing data in object storage while maintaining some guarantees such as ACID transactions or keeping snapshot history. Over time, multiple lakehouse formats
... - https://man.mtool.work/duckdb/2.0/lakehouse_formats.html
- Reading Faulty CSV Files 9410
- CSV files can come in all shapes and forms, with some presenting many errors that make the process of cleanly reading them inherently difficult. To help users read these files, DuckDB supports detailed error messages, the ability to skip fa
...ulty lines and the possibility of storing faulty lines in a temporary table to assist users with a data cleaning step. Structural Errors DuckDB supports the detecti...ber of characters of a rejected line stored in the csv_line column. Longer lines are truncated, and 0 disables truncation, storing the full line. BIGINT 10000 New As of DuckDB 2.0, the rejects_line_size_limit parameter bounds how many characters of each... - https://man.mtool.work/duckdb/2.0/data/csv/reading_faulty_csv_files.html
- Excel Export 9410
- DuckDB supports exporting data to Excel .xlsx files via the excel extension. Please note that .xls files are not supported. To install and load the extension, run: INSTALL excel; LOAD excel; Exporting Excel Sheets To export a table to an Ex
...option: COPY tbl TO 'output.xlsx' WITH (FORMAT xlsx, SHEET 'Sheet1'); Type Conversions Because Excel only really supports storing numbers or strings – the equivalent of VARCHAR and DOUBLE , the following type conversions are automatically applied when... - https://man.mtool.work/duckdb/2.0/guides/file_formats/excel_export.html
- UPDATE EXTENSIONS 9410
- The UPDATE EXTENSIONS statement allows synchronizing the locally installed extension state with the repository that published a given extension. This statement is the recommended way to keep up to date with new features or bug fixes being r
...xtension names to be updated: UPDATE EXTENSIONS (name_a, name_b, name_c); How It Works UPDATE EXTENSIONS is implemented by storing, if available, the ETag information, and sending a GET request conditional on the fact that the remote extension is differ... - https://man.mtool.work/duckdb/2.0/sql/statements/update_extensions.html
- JSON Overview 8821
- DuckDB supports SQL functions that are useful for reading values from existing JSON and creating new JSON data. JSON is supported with the json extension which is shipped with most DuckDB distributions and is auto-loaded on first use. If yo
...ult of a query to a JSON file: COPY (SELECT * FROM todos) TO 'todos.json'; JSON Data Type Create a table with a column for storing JSON data and insert data into it: CREATE TABLE example (j JSON); INSERT INTO example VALUES ('{ "family": "anatidae", "sp... - https://man.mtool.work/duckdb/2.0/data/json/overview.html
- Legacy Authentication Scheme for S3 API 8248
- Prior to version 0.10.0, DuckDB did not have a Secrets manager . Hence, the configuration of and authentication to S3 endpoints was handled via variables. This page documents the legacy authentication scheme for the S3 API. Warning This pag
...reases the risk of accidentally leaking secrets (e.g., by printing their values). Therefore, avoid using these methods for storing secrets. The recommended way to configure and authenticate of S3 endpoints is to use secrets . Legacy Authentication Schem... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/s3api_legacy_authentica...
- List Type 8248
- A LIST column encodes lists of values. Fields in the column can have values with different lengths, but they must all have the same underlying type. LIST s are typically used to store arrays of numbers, but can contain any uniform data type
...ons are provided for PostgreSQL compatibility. See the data types overview for a comparison between nested data types. For storing fixed-length lists, DuckDB uses the ARRAY type . Creating Lists Lists can be created using the list_value(expr, ...) funct... - https://man.mtool.work/duckdb/2.0/sql/data_types/list.html
DuckDB 2.0 Documentation / man.mtool.work