Results of 1 - 10 of about 32 for credentials (0.001 sec.)
- AWS Extension 14289
- The aws extension adds functionality, e.g., authentication, on top of the httpfs extension's S3 capabilities , using the AWS SDK. Installing and Loading The aws extension will be transparently autoloaded on first use from the official exten
...et manually. It is part of the httpfs extension and is documented in the S3 API page . Use it when you already have static credentials. The credential_chain provider, described below, which fetches credentials automatically using the AWS SDK. It is provided...) is documented in the S3 API page . credential_chain Provider The credential_chain provider allows automatically fetching credentials using mechanisms provided by the AWS SDK. For example, to use the AWS SDK default provider: CREATE OR REPLACE SECRET secre...The credential_chain provider also allows overriding the automatically fetched config. For example, to automatically load credentials, and then override the region, run: CREATE OR REPLACE SECRET secret ( TYPE s3, PROVIDER credential_chain, CHAIN config, RE... - https://man.mtool.work/duckdb/2.0/core_extensions/aws.html
- S3 and AWS Authentication 11582
- This page is the entry point for authenticating to S3 and other AWS services from DuckDB. Authentication is configured using secrets , and the detailed reference material lives on the S3 API Support and AWS Extension pages. The sections bel
...ference. Choosing an Authentication Method There are two S3 secret providers. Use the config provider when you have static credentials, and the credential_chain provider when you want DuckDB to fetch credentials automatically through the AWS SDK. Situation...Method Reference You have a static access key and secret config provider config provider Credentials come from ~/.aws/credentials or ~/.aws/config profiles credential_chain with a profile Selecting a Profile You sign in wit...credential_chain provider is AWS-specific and is provided by the aws extension , which is autoloaded on first use. Static Credentials: the config Provider Supply the key and secret directly: CREATE OR REPLACE SECRET secret ( TYPE s3, PROVIDER config, KEY_I... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/authentication.html
- S3 Parquet Export 10133
- To write a Parquet file to S3, the httpfs extension is required. This can be installed using the INSTALL SQL command. This only needs to be run once. INSTALL httpfs; To load the httpfs extension for usage, use the LOAD SQL command: LOAD htt
...pfs; After loading the httpfs extension, set up the credentials to write data. Note that the region parameter should match the region of the bucket you want to access. CREATE SECRET ( TY...nnection error for HTTP HEAD ), configure the endpoint explicitly via ENDPOINT 's3.⟨your-region⟩.amazonaws.com' . To fetch credentials automatically instead — or to use a named profile, SSO, or an assumed role — see the aws extension . The default looks lik...e: CREATE SECRET ( TYPE s3, PROVIDER credential_chain ); After the httpfs extension is set up and the S3 credentials are correctly configured, Parquet files can be written to S3 using the following command: COPY ⟨table_name⟩ TO 's3://⟨s3-b... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/s3_export.html
- SeaweedFS Import 9456
- Prerequisites For SeaweedFS , the S3-compatible gateway allows you to use DuckDB's S3 support to read and write from SeaweedFS buckets. This requires the httpfs extension , which can be installed using the INSTALL SQL command. This only nee
...ds to be run once. To try SeaweedFS locally, create a file s3config.json with the S3 credentials: { "identities": [ { "name": "analyst", "credentials": [ { "accessKey": "your_access_key_id", "secretKey": "your_secret_ac...-name ⟨your-bucket⟩" | \ docker exec -i seaweedfs weed shell -master=localhost:9333 The S3 endpoint listens on port 8333. Credentials and Configuration Create an S3 secret with the credentials from the SeaweedFS S3 configuration: CREATE SECRET my_secret (...endpoint such as the local setup above; omit it when the gateway runs behind TLS. Querying After setting up the SeaweedFS credentials, you can query the data using DuckDB's built-in methods, such as read_csv or read_parquet : SELECT * FROM 's3://⟨your-buck... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/seaweedfs_import....
- S3 Iceberg Import 9279
- Prerequisites Loading an Iceberg file from S3 requires both the httpfs and iceberg extensions. Install them using the INSTALL SQL command. You only need to install extensions once. INSTALL httpfs; INSTALL iceberg; To load the extensions, us
...e the LOAD command: LOAD httpfs; LOAD iceberg; Credentials After loading the extensions, set up the credentials and S3 region to read data. You may either use an access key and secr...s3, KEY_ID '⟨AKIAIOSFODNN7EXAMPLE⟩', SECRET '⟨wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY⟩', REGION '⟨us-east-1⟩' ); To fetch credentials automatically instead — or to use a named profile, SSO, or an assumed role — see the aws extension . The default looks lik...E SECRET ( TYPE s3, PROVIDER credential_chain ); Loading Iceberg Tables from S3 After the extensions are set up and the S3 credentials are correctly configured, Iceberg tables can be read from S3 using the following command: SELECT * FROM iceberg_scan('s3:/... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/s3_iceberg_import...
- Legacy Authentication Scheme for S3 API 9037
- Prior to version 0.10.0, DuckDB did not have a Secrets manager . Hence, the configuration of and authentication to S3 endpoints was handled via variables. This page documents the legacy authentication scheme for the S3 API. Warning This pag
.... After configuring the correct endpoint and region, public files can be read. To also read private files, authentication credentials can be added: SET s3_access_key_id = '⟨aws_access_key_id⟩'; SET s3_secret_access_key = '⟨aws_secret_access_key⟩'; Alternat...ively, temporary S3 credentials are also supported. They require setting an additional session token: SET s3_session_token = '⟨aws_session_token⟩'; The aw...s extension allows for loading AWS credentials. Per-Request Configuration Deprecated Per-request configuration through S3 URL query parameters is deprecated and disabled... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/s3api_legacy_authentica...
- S3 API Support 8973
- The httpfs extension supports reading/writing/ globbing files on object storage servers using the S3 API. S3 offers a standard API to read and write to remote files (while regular http servers, predating S3, do not offer a common write API)
...OM 's3://⟨your-bucket⟩/⟨your_file⟩.parquet'; credential_chain Provider The credential_chain provider automatically fetches credentials using the AWS SDK (profiles, SSO, assumed roles, web identities, instance metadata, and more). It is provided by the aws e...e secret of the key to use S3 , GCS , R2 STRING - SESSION_TOKEN Optionally, a session token can be passed to use temporary credentials S3 , GCS , R2 STRING - URL_COMPATIBILITY_MODE Can help when URLs contain problematic characters S3 , GCS , R2 BOOLEAN fals...NG - REQUESTER_PAYS Allows use of "requester pays" S3 buckets S3 BOOLEAN false REFRESH Set to auto to periodically refresh credentials (see the aws extension ) S3 , GCS , R2 STRING - Automatic Credential Refresh Independently of the REFRESH secret parameter... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/s3api.html
- S3 Parquet Import 8795
- Prerequisites To load a Parquet file from S3, the httpfs extension is required. This can be installed using the INSTALL SQL command. This only needs to be run once. INSTALL httpfs; To load the httpfs extension for usage, use the LOAD SQL co
...mmand: LOAD httpfs; Credentials and Configuration After loading the httpfs extension, set up the credentials and S3 region to read data: CREATE SECRET ( T...nnection error for HTTP HEAD ), configure the endpoint explicitly via ENDPOINT 's3.⟨your-region⟩.amazonaws.com' . To fetch credentials automatically instead — or to use a named profile, SSO, or an assumed role — see the aws extension . The default looks lik... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/s3_import.html
- PostgreSQL Extension and the Secret Manager 8683
- User credentials and other PostgreSQL database connection details can be stored using the DuckDB Secrets Manager . The following syntax can be used to create a secret: CREATE SECRET ( TYPE postgres, HOST '127.0.0.1', PORT 5432, DATABASE pos
...nnection string to override individual options. For example, to connect to a different database while still using the same credentials, we can override only the database name in the following manner. ATTACH 'dbname=my_other_db' AS postgres_db (TYPE postgres...arameter in the ATTACH . ATTACH '' AS postgres_db_one (TYPE postgres, SECRET postgres_secret_one); Warning Avoid including credentials, such as a password, directly in the connection string. If a connection error occurs, the full connection string, includin...g your credentials, may appear in the resulting error message. For better security, store credentials using DuckDB-managed secrets. Secret Co... - https://man.mtool.work/duckdb/2.0/core_extensions/postgres/secrets.html
- Cloudflare R2 Import 8248
- Prerequisites For Cloudflare R2, the S3 Compatibility API allows you to use DuckDB's S3 support to read and write from R2 buckets. This requires the httpfs extension , which can be installed using the INSTALL SQL command. This only needs to
...be run once. Credentials and Configuration You will need to generate an S3 auth token and create an R2 secret in DuckDB: CREATE SECRET ( TYPE r2, K.../K7MDENG/bPxRfiCYEXAMPLEKEY⟩', ACCOUNT_ID '⟨your-32-character-hexadecimal-account-ID⟩' ); Querying After setting up the R2 credentials, you can query the R2 data using DuckDB's built-in methods, such as read_csv or read_parquet : SELECT * FROM read_parquet(... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/cloudflare_r2_imp...
DuckDB 2.0 Documentation / man.mtool.work