Results of 1 - 10 of about 20 for ⟩/⟨ (0.002 sec.)
- Azure Extension 14070
- The azure extension lets DuckDB read and write files in Azure Blob Storage and Azure Data Lake Storage (ADLS) Gen2. Installing and Loading The azure extension will be transparently autoloaded on first use from the official extension reposit
...uery Azure storage as follows: Azure Blob Storage Allowed URI schemes: az or azure SELECT count(*) FROM 'az://⟨my_container⟩/⟨path⟩/⟨my_file⟩.⟨parquet_or_csv⟩'; Globs are also supported: SELECT * FROM 'az://⟨my_container⟩/⟨path⟩/*.csv'; SELECT * FRO...M 'az://⟨my_container⟩/⟨path⟩/**'; Or with a fully qualified path syntax: SELECT count(*) FROM 'az://⟨my_storage_account⟩.blob.core.windows.net/⟨my..._container⟩/⟨path⟩/⟨my_file⟩.⟨parquet_or_csv⟩'; SELECT * FROM 'az://⟨my_storage_account⟩.blob.core.windows.net/⟨my_container⟩/⟨path⟩/*.c... - https://man.mtool.work/duckdb/2.0/core_extensions/azure.html
- Delta Extension 11742
- The delta extension adds support for the Delta Lake open-source storage format . It is built using the Delta Kernel . The extension reads Delta tables, local and remote, and writes them: it can create tables and append to them. For implemen
...my_table; Reading from an S3 Bucket To scan a Delta table in an S3 bucket , run: SELECT * FROM delta_scan('s3://⟨my_bucket⟩/⟨path⟩'); To authenticate, create a secret : CREATE SECRET ( TYPE s3, PROVIDER credential_chain ); SELECT * FROM delta_scan(...'s3://⟨my_bucket⟩/⟨path⟩'); For a public bucket, you may need to set its region in a secret: CREATE SECRET ( TYPE s3, REGION '⟨my_region⟩' );...SELECT * FROM delta_scan('s3://⟨my_bucket⟩/⟨path⟩'); Reading from Azure Blob Storage To scan a Delta table in an Azure Blob Storage container , run: SELECT * FROM delt... - https://man.mtool.work/duckdb/2.0/core_extensions/delta.html
- Hugging Face Support 9499
- The httpfs extension introduces support for the hf:// protocol to access datasets hosted in Hugging Face repositories. See the announcement blog post for details. Usage Hugging Face repositories can be queried using the following URL patter
...n: hf://datasets/⟨my_username⟩/⟨my_dataset⟩/⟨path_to_file⟩ For example, to read a CSV file, you can use the following query: SELECT * FROM 'hf://datasets/d...specific commit. You can query different dataset versions/revisions by using the following URL: hf://datasets/⟨my_username⟩/⟨my_dataset⟩@⟨my_branch⟩/⟨path_to_file⟩ For example: SELECT * FROM 'hf://datasets/datasets-examples/doc-formats-csv-1@~parqu... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/hugging_face.html
- S3 API Support 9329
- The httpfs extension supports reading/writing/ globbing files on object storage servers using the S3 API. S3 offers a standard API to read and write to remote files (while regular http servers, predating S3, do not offer a common write API)
...azonaws.com' . Now, to query using the above secret, simply query any s3:// prefixed file: SELECT * FROM 's3://⟨your-bucket⟩/⟨your_file⟩.parquet'; credential_chain Provider The credential_chain provider automatically fetches credentials using the AW...TYPE s3, PROVIDER credential_chain, CHAIN config, REGION '⟨eu-west-1⟩', KMS_KEY_ID 'arn:aws:kms:⟨region⟩:⟨account_id⟩:⟨key⟩/⟨key_id⟩', SCOPE 's3://⟨bucket-sub-path⟩' ); R2 Secrets While Cloudflare R2 uses the regular S3 API, DuckDB has a special Se...file/that/uses/a/gcs/secret⟩.parquet'); Reading Reading files from S3 is now as simple as: SELECT * FROM 's3://⟨your-bucket⟩/⟨filename⟩.⟨extension⟩'; Partial Reading The httpfs extension supports partial reading from S3 buckets. Pinning Object Versi... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/s3api.html
- Google Cloud Storage Import 9159
- Prerequisites The Google Cloud Storage (GCS) can be used via the httpfs extension . This can be installed with the INSTALL httpfs SQL command. This only needs to be run once. Credentials and Configuration You need to create HMAC keys and de
...erying After setting up the GCS credentials, you can query the GCS data using: SELECT * FROM read_parquet('gs://⟨gcs_bucket⟩/⟨file.parquet⟩'); Attaching to a Database You can attach to a database file in read-only mode: LOAD httpfs; ATTACH 'gs://⟨gc...s_bucket⟩/⟨file.duckdb⟩' AS ⟨duckdb_database⟩ (READ_ONLY); Databases in Google Cloud Storage can only be attached in read-only mode.... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/gcs_import.html
- Tigris Import 8751
- Prerequisites For Tigris , the S3-compatible API allows you to use DuckDB's S3 support to read and write from Tigris buckets. This requires the httpfs extension , which can be installed using the INSTALL SQL command. This only needs to be r
...query the data using DuckDB's built-in methods, such as read_csv or read_parquet : SELECT * FROM 's3://⟨tigris-bucket-name⟩/⟨file⟩.csv'; SELECT * FROM read_parquet('s3://⟨tigris-bucket-name⟩/⟨file⟩.parquet');... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/tigris_import.htm...
- Advanced Installation Methods 8666
- Downloading Extensions Directly from S3 Downloading an extension directly can be helpful when building a Lambda service or container that uses DuckDB. DuckDB extensions are stored in public S3 buckets, but the directory structure of those b
...st be used. To download an extension file directly, use the following format: http://extensions.duckdb.org/v⟨duckdb_version⟩/⟨platform_name⟩/⟨extension_name⟩.duckdb_extension.gz For example: http://extensions.duckdb.org/v2.0.0/windows_amd64/json.duc... - https://man.mtool.work/duckdb/2.0/extensions/advanced_installation_methods.html
- S3 Parquet Export 8666
- To write a Parquet file to S3, the httpfs extension is required. This can be installed using the INSTALL SQL command. This only needs to be run once. INSTALL httpfs; To load the httpfs extension for usage, use the LOAD SQL command: LOAD htt
...correctly configured, Parquet files can be written to S3 using the following command: COPY ⟨table_name⟩ TO 's3://⟨s3-bucket⟩/⟨filename⟩.parquet'; Similarly, Google Cloud Storage (GCS) is supported through the Interoperability API. You need to create.../bPxRfiCYEXAMPLEKEY⟩' ); After setting up the GCS credentials, you can export using: COPY ⟨table_name⟩ TO 'gs://⟨gcs_bucket⟩/⟨filename⟩.parquet';... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/s3_export.html
- SeaweedFS Import 8326
- Prerequisites For SeaweedFS , the S3-compatible gateway allows you to use DuckDB's S3 support to read and write from SeaweedFS buckets. This requires the httpfs extension , which can be installed using the INSTALL SQL command. This only nee
...you can query the data using DuckDB's built-in methods, such as read_csv or read_parquet : SELECT * FROM 's3://⟨your-bucket⟩/⟨file⟩.csv'; SELECT * FROM read_parquet('s3://⟨your-bucket⟩/⟨file⟩.parquet'); Writing works the same way, including glob rea... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/seaweedfs_import....
- Cloudflare R2 Import 7919
- Prerequisites For Cloudflare R2, the S3 Compatibility API allows you to use DuckDB's S3 support to read and write from R2 buckets. This requires the httpfs extension , which can be installed using the INSTALL SQL command. This only needs to
...2 data using DuckDB's built-in methods, such as read_csv or read_parquet : SELECT * FROM read_parquet('r2://⟨r2-bucket-name⟩/⟨file⟩');... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/cloudflare_r2_imp...
DuckDB 2.0 Documentation / man.mtool.work