Results of 1 - 10 of about 18 for provider (0.002 sec.)
- S3 and AWS Authentication 12903
- This page is the entry point for authenticating to S3 and other AWS services from DuckDB. Authentication is configured using secrets , and the detailed reference material lives on the S3 API Support and AWS Extension pages. The sections bel
...explain which method to use and link to the relevant reference. Choosing an Authentication Method There are two S3 secret providers. Use the config provider when you have static credentials, and the credential_chain provider when you want DuckDB to fetc...h credentials automatically through the AWS SDK. Situation Method Reference You have a static access key and secret config provider config provider Credentials come from ~/.aws/credentials or ~/.aws/config profiles credential_chain with a profile Selecti...You run on an EC2 instance and want to use its instance profile credential_chain with the instance chain credential_chain provider You connect to Amazon RDS or Aurora with IAM authentication rds secret Amazon RDS (IAM Authentication) The config provider... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/authentication.html
- AWS Extension 12216
- The aws extension adds functionality, e.g., authentication, on top of the httpfs extension's S3 capabilities , using the AWS SDK. Installing and Loading The aws extension will be transparently autoloaded on first use from the official exten
...tion page. The preferred way to configure and authenticate to AWS S3 endpoints is to use secrets . There are two S3 secret providers: The config provider, where you supply the access key and secret manually. It is part of the httpfs extension and is docu...mented in the S3 API page . Use it when you already have static credentials. The credential_chain provider, described below, which fetches credentials automatically using the AWS SDK. It is provided by the aws extension and suppo...SSO, assumed roles, web identities (IRSA), and instance metadata. The full list of S3 secret parameters that apply to both providers ( ENDPOINT , REGION , URL_STYLE , USE_SSL , KMS_KEY_ID , REQUESTER_PAYS , …) is documented in the S3 API page . credentia... - https://man.mtool.work/duckdb/2.0/core_extensions/aws.html
- Azure Extension 11444
- The azure extension lets DuckDB read and write files in Azure Blob Storage and Azure Data Lake Storage (ADLS) Gen2. Installing and Loading The azure extension will be transparently autoloaded on first use from the official extension reposit
...re; Usage Queries need a secret ; for a public container, a secret with the storage account name is enough (see the CONFIG provider ). Then you can query Azure storage as follows: Azure Blob Storage Allowed URI schemes: az or azure SELECT count(*) FROM '...he SET variables in Authentication with Variables still work but are deprecated. Authentication with Secret Several Secret Providers are available, described below. These rules apply to every provider: If you need to define different secrets for differen...default endpoint ( blob.core.windows.net for az:// and azure:// , dfs.core.windows.net for abfss:// and abfs:// ). CONFIG Provider The default provider, CONFIG (i.e., user-configured), allows access to the storage account using a connection string or an... - https://man.mtool.work/duckdb/2.0/core_extensions/azure.html
- Secrets Manager 9334
- The Secrets manager provides a unified user interface for secrets across all backends that use them. Secrets can be scoped, so different storage prefixes can have different secrets, allowing for example to join data across organizations in
...stgres quack Quack quack r2 Cloudflare R2 httpfs s3 AWS S3 httpfs aws AWS aws For each type, there are one or more “secret providers” that specify how the secret is created. Secrets can also have an optional scope, which is a file path prefix that the se..._secrets directory. On startup of DuckDB, persistent secrets are read from this directory and automatically loaded. Secret Providers To create a secret, a Secret Provider needs to be used. A Secret Provider is a mechanism through which a secret is genera...ted. To illustrate this, for the S3 , GCS , R2 , and AZURE secret types, DuckDB currently supports two providers: CONFIG and credential_chain . The CONFIG provider requires the user to pass all configuration information into the CREAT... - https://man.mtool.work/duckdb/2.0/configuration/secrets_manager.html
- S3 API Support 8922
- The httpfs extension supports reading/writing/ globbing files on object storage servers using the S3 API. S3 offers a standard API to read and write to remote files (while regular http servers, predating S3, do not offer a common write API)
.... DuckDB conforms to the S3 API, that is now common among industry storage providers. Platforms The httpfs filesystem is tested with AWS S3 , Minio , Google Cloud and lakeFS . Other services that implement...S Authentication page. The preferred way to configure and authenticate to S3 endpoints is to use secrets . Multiple secret providers are available. To migrate from the deprecated S3 API , use a defined secret with a profile. See Selecting a Profile . con...fig Provider The default provider, config (i.e., user-configured), allows access to the S3 bucket by manually providing a key. For exam... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/s3api.html
- Hugging Face Support 8150
- The httpfs extension introduces support for the hf:// protocol to access datasets hosted in Hugging Face repositories. See the announcement blog post for details. Usage Hugging Face repositories can be queried using the following URL patter
...ns to obtain your access token. Second, set it in your DuckDB session using DuckDB's Secrets Manager . DuckDB supports two providers for managing secrets: CONFIG The user must pass all configuration information into the CREATE SECRET statement . To creat...e a secret using the CONFIG provider, use the following command: CREATE SECRET hf_token ( TYPE huggingface, TOKEN 'your_hf_token' ); credential_chain Automatic...Hugging Face token, it will try to get it from ~/.cache/huggingface/token . To create a secret using the credential_chain provider, use the following command: CREATE SECRET hf_token ( TYPE huggingface, PROVIDER credential_chain );... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/hugging_face.html
- DuckDB_% Metadata Functions 8012
- DuckDB offers a collection of table functions that provide metadata about the current database. These functions reside in the main schema and their names are prefixed with duckdb_ . The resultset returned by a duckdb_ table function may be
...ported in the current DuckDB session. Column Description Type type The name of the secret type, e.g., s3 . VARCHAR default_provider The default secret provider, e.g., config . VARCHAR extension The extension that registered the secret type, e.g., aws . V...e duckdb_secret_type_parameters() function was introduced in DuckDB 2.0. It lists the parameters that each secret type and provider accepts. Column Description Type type The name of the secret type, e.g., s3 . VARCHAR provider The secret provider, e.g.,...n Description Type name The name of the secret. VARCHAR type The type of the secret, e.g., S3 , GCS , R2 , AZURE . VARCHAR provider The provider of the secret. VARCHAR persistent Denotes whether the secret is persistent. BOOLEAN storage The backend for s... - https://man.mtool.work/duckdb/2.0/sql/meta/duckdb_table_functions.html
- Reference 7927
- This page lists the functions, settings, and log types exposed by the quack extension . For a tour of the protocol, start with the Overview . Function Reference Server Management Function Description quack_serve(⟨listen_uri⟩, ...) Start a s
...ack ; existing key files are not overwritten. Returns the file paths and the certificate fingerprint. quack_identify(name, provider, hostname, region, meta) Set this node's whoami identity fields. Any subset can be supplied. whoami() Table macro returnin...is sugar that updates them. Setting Type Default Description whoami_name VARCHAR (empty) Human-readable node name. whoami_provider VARCHAR (empty) Deployment provider ( ec2 , docker , local , ...). whoami_hostname VARCHAR (empty) Network hostname / publ...nt region. whoami_started_at VARCHAR (empty) Node start time (ISO 8601 timestamp). Anchors uptime . whoami_meta VARCHAR {} Provider-specific metadata as JSON. quack_loaded_at_us BIGINT Epoch microseconds at extension load Fallback uptime anchor when whoa... - https://man.mtool.work/duckdb/2.0/quack/reference.html
- PostgreSQL Extension and the Secret Manager 7824
- User credentials and other PostgreSQL database connection details can be stored using the DuckDB Secrets Manager . The following syntax can be used to create a secret: CREATE SECRET ( TYPE postgres, HOST '127.0.0.1', PORT 5432, DATABASE pos
...rts IAM authentication, when the password is not specified in the secret, but instead one of the configured AWS Credential Providers is used to generate the password, that is refreshed by the postgres extension automatically. Connecting with IAM Authenti...rameters as the aws rds generate-db-auth-token command in the example above: sql CREATE SECRET aws_rds_secret1 ( TYPE rds, PROVIDER credential_chain, CHAIN 'env;sso;', REGION 'eu-west-1', RDS_USER 'postgres', RDS_HOST 'database-1-instance-1.xxxxxxxxxxxx....S extension documentation . Storing Secrets inside a PostgreSQL Database DuckDB Secrets Manager supports pluggable Storage Providers. The postgres extension implements storing the secrets (of any type) as records in the PostgreSQL database table. The fol... - https://man.mtool.work/duckdb/2.0/core_extensions/postgres/secrets.html
- Lance Extension 7686
- The lance extension adds support for reading and writing Lance tables. Lance is a modern lakehouse format optimized for ML/AI workloads, with native cloud storage support. Installing and Loading You can install the lance extension from Duck
...object store URIs (e.g., s3://... ), configure a TYPE lance secret using the Secrets Manager : CREATE SECRET ( TYPE lance, PROVIDER credential_chain, SCOPE 's3://bucket/' ); SELECT * FROM 's3://⟨bucket/path/to/out.lance⟩' LIMIT 10; Write a Lance Dataset...te to s3://... paths, configure a TYPE lance secret for that scope using the Secrets Manager : CREATE SECRET ( TYPE lance, PROVIDER credential_chain, SCOPE 's3://⟨bucket⟩/' ); COPY (SELECT 1 AS id) TO 's3://⟨bucket/path/to/out.lance⟩' (FORMAT lance, MODE... - https://man.mtool.work/duckdb/2.0/core_extensions/lance.html
DuckDB 2.0 Documentation / man.mtool.work