Results of 1 - 10 of about 48 for Metadata (0.002 sec.)
- Querying Parquet Metadata 12995
- Parquet Metadata The parquet_metadata function can be used to query the metadata contained within a Parquet file, which reveals various internal details of the Parquet file such as the statistics of the different columns. This can be useful
...es, or even to obtain a quick overview of what the different columns contain. The function supports glob patterns to query metadata across multiple files in parallel: SELECT * FROM parquet_metadata('test.parquet'); SELECT * FROM parquet_metadata('data/*....parquet'); Below is a table of the columns returned by parquet_metadata . Field Type file_name VARCHAR row_group_id BIGINT row_group_num_rows BIGINT row_group_num_columns BIGINT row_group_bytes...ictionary_page_offset BIGINT data_page_offset BIGINT total_compressed_size BIGINT total_uncompressed_size BIGINT key_value_metadata MAP(BLOB, BLOB) bloom_filter_offset BIGINT bloom_filter_length BIGINT min_is_exact BOOLEAN max_is_exact BOOLEAN row_group_... - https://man.mtool.work/duckdb/2.0/data/parquet/metadata.html
- Iceberg Options 12465
- This page lists the options of the iceberg extension: the parameters accepted by the Iceberg functions , the options of the ATTACH and CREATE SECRET statements used to connect to a catalog , and the global settings. ATTACH Options To make a
...se Disables the multi-table transactions/commit endpoint. Enable for catalogs that reject this endpoint. SKIP_CREATE_TABLE_METADATA_UPDATES BOOLEAN false Skips follow-up metadata updates after non-staged CREATE TABLE . Enable for catalogs that fully init...ialize metadata during table creation and reject subsequent updates. REMOVE_FILES_ON_DELETE BOOLEAN true If disabled, files (data and meta...sted with a placeholder column of type UNKNOWN . With eager , schemas are resolved while tables are enumerated, which adds metadata-loading work and can increase catalog enumeration time for large catalogs. Eager resolution is needed when a Quack client... - https://man.mtool.work/duckdb/2.0/core_extensions/iceberg/iceberg_options.html
- Iceberg Functions 11723
- The iceberg extension provides the following functions: Function Description iceberg_scan(source) Reads an Iceberg table from a path. iceberg_metadata(source) Returns the manifest entries of an Iceberg table. iceberg_snapshots(source) Retur
...Iceberg table. iceberg_scan_plan(table) Returns the scan tasks (files to read) planned for an Iceberg table. iceberg_view_metadata(view) Returns the raw LoadView response of the REST Catalog. iceberg_table_properties(table) Returns all properties of the...le) Compacts the data files of an Iceberg table. iceberg_to_ducklake(iceberg_catalog, ducklake_catalog) Copies the Iceberg metadata of a catalog to a DuckLake catalog. iceberg_bucket(num_buckets, value) Computes the Iceberg bucket partition transform of..._scan('data/iceberg/lineitem_iceberg', allow_moved_paths = true); count_star() 51793 The path can also point directly to a metadata file, or to an object store such as S3 or Azure Blob Storage. See the Iceberg Options page for the parameters accepted by... - https://man.mtool.work/duckdb/2.0/core_extensions/iceberg/iceberg_functions.html
- DuckDB_% Metadata Functions 10541
- DuckDB offers a collection of table functions that provide metadata about the current database. These functions reside in the main schema and their names are prefixed with duckdb_ . The resultset returned by a duckdb_ table function may be
...metric, e.g., seconds or rows . Empty for unitless metrics. VARCHAR duckdb_columns The duckdb_columns() function provides metadata about the columns available in the DuckDB instance. Column Description Type database_name The name of the database that co...fter the decimal separator). INTEGER The information_schema.columns system view provides a more standardized way to obtain metadata about database columns, but the duckdb_columns function also returns metadata about DuckDB internal objects. (In fact, inf...scription Type count The number of open connections. UBIGINT duckdb_constraints The duckdb_constraints() function provides metadata about the constraints available in the DuckDB instance. Column Description Type database_name The name of the database tha... - https://man.mtool.work/duckdb/2.0/sql/meta/duckdb_table_functions.html
- Iceberg Extension 10299
- The iceberg extension implements support for the Apache Iceberg open table format . There are two ways to work with Iceberg in DuckDB: Individual tables are read directly from storage, by pointing at a table's metadata. This requires no cat
...n is a UUID. To do so, navigate to the data/iceberg directory and run: SELECT count(*) FROM iceberg_scan('lineitem_iceberg/metadata/v1.metadata.json'); count_star() 60175 For the full list of parameters accepted by iceberg_scan , see the Iceberg Options...les in object stores such as S3 or Azure Blob Storage. SELECT count(*) FROM iceberg_scan('s3://bucketname/lineitem_iceberg/metadata/v1.metadata.json'); Metadata and Snapshots To inspect the manifest entries and snapshots of a table, use the iceberg_metad...ata and iceberg_snapshots functions. To read tables without a version-hint.text file or with non-standard metadata file names, see Selecting Metadata Versions . Time Travel To read a historical snapshot of a table, pass either snapshot_f... - https://man.mtool.work/duckdb/2.0/core_extensions/iceberg/overview.html
- S3 Iceberg Import 9315
- Prerequisites Loading an Iceberg file from S3 requires both the httpfs and iceberg extensions. Install them using the INSTALL SQL command. You only need to install extensions once. INSTALL httpfs; INSTALL iceberg; To load the extensions, us
...g tables can be read from S3 using the following command: SELECT * FROM iceberg_scan('s3://⟨bucket⟩/⟨iceberg_table_folder⟩/metadata/⟨id⟩.metadata.json'); If you pass the table folder instead of the metadata file, DuckDB looks for a version-hint.text file...in the table's metadata directory. If there is no such file, you'll get an error like this: Invalid Configuration Error: Failed to read iceberg ta...ading uncommitted data. To enable this use 'SET unsafe_enable_version_guessing = true;' In this case, link directly to the metadata file or see Selecting Metadata Versions .... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/s3_iceberg_import...
- Extension Distribution 8194
- Platforms Extension binaries are distributed for several platforms (see below). For platforms where packages for certain extensions are not available, users can build them from source and install the resulting binaries manually . Most offic
...r platform, DuckDB will throw an error and refuse to load the extension. DuckDB stores this compatibility information in a metadata footer appended to the end of the extension binary. Warning Do not run strip on an extension binary. Stripping removes or...corrupts the appended metadata footer, after which DuckDB reports the metadata as invalid and refuses to load the extension. On some platforms, strip fai...ipping the Compatibility Check For debugging purposes, extension developers can skip this check using the allow_extensions_metadata_mismatch setting. This lets INSTALL and UPDATE EXTENSIONS accept extensions with mismatching or invalid metadata. For LOAD... - https://man.mtool.work/duckdb/2.0/extensions/extension_distribution.html
- Reading and Writing Parquet Files 8118
- Examples Read a single Parquet file: SELECT * FROM 'test.parquet'; Figure out which columns/types are in a Parquet file: DESCRIBE SELECT * FROM 'test.parquet'; Create a table from a Parquet file: CREATE TABLE test AS SELECT * FROM 'test.par
...quet', 'folder2/*.parquet']); Read over HTTPS: SELECT * FROM read_parquet('https://some.url/some_file.parquet'); Query the metadata of a Parquet file : SELECT * FROM parquet_metadata('test.parquet'); Query the file metadata of a Parquet file : SELECT * F...ROM parquet_file_metadata('test.parquet'); Query the key-value metadata of a Parquet file : SELECT * FROM parquet_kv_metadata('test.parquet'); Query...tbl TO 'result-zstd.parquet' (FORMAT parquet, COMPRESSION zstd, COMPRESSION_LEVEL 1); Write to Parquet file with key-value metadata : COPY ( SELECT 42 AS number, true AS is_even ) TO 'kv_metadata.parquet' ( FORMAT parquet, KV_METADATA { number: 'Answer t... - https://man.mtool.work/duckdb/2.0/data/parquet/overview.html
- Configuration 7951
- DuckDB has a number of configuration options that can be used to change the behavior of the system. The configuration options can be set using either the SET statement or the PRAGMA statement . They can be reset to their original values usi
...ly move from 'undecided' to 'allowed' or 'forbidden', or from 'allowed' to 'forbidden'. VARCHAR undecided allow_extensions_metadata_mismatch Allow to load extensions with no compatible metadata. BOOLEAN false allow_parser_override_extension Allow extensi...stom_extension_repository Overrides the custom endpoint for extension installation. VARCHAR custom_user_agent HTTP request metadata that can be set by callers/wrappers to tag their own traffic when DuckDB makes a request. VARCHAR default_block_size The d...ue enable_global_s3_configuration Automatically fetch AWS credentials from environment variables. BOOLEAN true enable_http_metadata_cache Whether or not the global http metadata is used to cache HTTP metadata. BOOLEAN false enable_logging Enables the log... - https://man.mtool.work/duckdb/2.0/configuration/overview.html
- Troubleshoot 7906
- Overview This page collects common issues encountered when using the DuckDB JDBC driver, together with their workarounds. If you run into a problem that is not covered here, search the driver's issue tracker on GitHub. Driver Class Not Foun
...JNI surface eagerly when its native library initializes, so a single class, method, or field missing from the reachability metadata fails the whole initialization. This indicates that the metadata compiled into the image is missing or older than the driv...er: upgrade to a driver version that ships its own metadata, or regenerate the metadata with the tracing agent against the exact driver version in use. The error is sometimes wrapped...a.⟨suffix⟩ This is what the library's JNI_OnLoad reports when an internal lookup failed, and the cause is the same missing metadata, not a JNI version problem. Native Image: UnsatisfiedLinkError: Can't load library In a Native Image executable, the first... - https://man.mtool.work/duckdb/2.0/clients/java/troubleshoot.html
DuckDB 2.0 Documentation / man.mtool.work