Results of 1 - 10 of about 18 for chain (0.002 sec.)
- AWS Extension 12988
- The aws extension adds functionality, e.g., authentication, on top of the httpfs extension's S3 capabilities , using the AWS SDK. Installing and Loading The aws extension will be transparently autoloaded on first use from the official exten
...he httpfs extension and is documented in the S3 API page . Use it when you already have static credentials. The credential_chain provider, described below, which fetches credentials automatically using the AWS SDK. It is provided by the aws extension...( ENDPOINT , REGION , URL_STYLE , USE_SSL , KMS_KEY_ID , REQUESTER_PAYS , …) is documented in the S3 API page . credential_chain Provider The credential_chain provider allows automatically fetching credentials using mechanisms provided by the AWS SDK....For example, to use the AWS SDK default provider: CREATE OR REPLACE SECRET secret ( TYPE s3, PROVIDER credential_chain ); To query a file using the above secret, simply query any s3:// prefixed file. DuckDB also allows specifying a specific... - https://man.mtool.work/duckdb/2.0/core_extensions/aws.html
- S3 and AWS Authentication 12628
- This page is the entry point for authenticating to S3 and other AWS services from DuckDB. Authentication is configured using secrets , and the detailed reference material lives on the S3 API Support and AWS Extension pages. The sections bel
...ion Method There are two S3 secret providers. Use the config provider when you have static credentials, and the credential_chain provider when you want DuckDB to fetch credentials automatically through the AWS SDK. Situation Method Reference You have...y and secret config provider config provider Credentials come from ~/.aws/credentials or ~/.aws/config profiles credential_chain with a profile Selecting a Profile You sign in with AWS IAM Identity Center (SSO) credential_chain with the sso chain Sing...le Sign-On (SSO) You assume an IAM role via STS credential_chain with the sts chain Assuming a Role (STS) You run on Amazon EKS with IAM Roles for Service Accounts (IRSA) credential_chain... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/authentication.html
- S3 API Support 10037
- The httpfs extension supports reading/writing/ globbing files on object storage servers using the S3 API. S3 offers a standard API to read and write to remote files (while regular http servers, predating S3, do not offer a common write API)
...the above secret, simply query any s3:// prefixed file: SELECT * FROM 's3://⟨your-bucket⟩/⟨your_file⟩.parquet'; credential_chain Provider The credential_chain provider automatically fetches credentials using the AWS SDK (profiles, SSO, assumed roles,...tension . For example, to use the AWS SDK default provider: CREATE OR REPLACE SECRET secret ( TYPE s3, PROVIDER credential_chain ); For the full set of credential_chain options — CHAIN values, profile selection, assuming roles, SSO, web identity (IRSA...Secret Parameters Below is a complete list of the supported parameters that can be used for both the config and credential_chain providers: Name Description Secret Type Default ENDPOINT Specify a custom S3 endpoint S3 , GCS , R2 STRING s3.amazonaws.co... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/s3api.html
- Azure Extension 9625
- The azure extension lets DuckDB read and write files in Azure Blob Storage and Azure Data Lake Storage (ADLS) Gen2. Installing and Loading The azure extension will be transparently autoloaded on first use from the official extension reposit
...age account name: CREATE SECRET secret2 ( TYPE azure, PROVIDER config, ACCOUNT_NAME '⟨storage_account_name⟩' ); credential_chain Provider The credential_chain provider allows connecting using credentials automatically fetched by the Azure SDK via the...Azure credential chain. By default, the provider uses the Azure SDK's DefaultAzureCredential chain. The token is fetched at query time, not when...the secret is created. For example: CREATE SECRET secret3 ( TYPE azure, PROVIDER credential_chain, ACCOUNT_NAME '⟨storage_account_name⟩' ); DuckDB also allows specifying a specific chain using the CHAIN keyword. This tak... - https://man.mtool.work/duckdb/2.0/core_extensions/azure.html
- Amazon RDS with IAM Authentication 9419
- Managed PostgreSQL databases on Amazon RDS and Aurora can authenticate with short-lived IAM tokens instead of a static password. DuckDB supports this through the postgres and aws extensions: the aws extension generates the token from an AWS
...credential chain, and the postgres extension uses it as the connection password, refreshing it automatically. Installation and Loading INST...INSTALL aws; LOAD postgres; LOAD aws; Creating the Secrets First, create a secret of type rds . It uses the AWS credential chain to mint the IAM token: CREATE SECRET aws_rds_secret ( TYPE rds, PROVIDER credential_chain, CHAIN 'env;sso', REGION '⟨eu-we...RDS IAM Authentication — the full reference, including verifying the setup with psql first. AWS extension — the credential chain options available to the rds secret.... - https://man.mtool.work/duckdb/2.0/guides/database_integration/rds_iam.html
- Relational API on Pandas 8459
- DuckDB offers a relational API that can be used to chain together query operations. These are lazily evaluated so that DuckDB can optimize their execution. These operators can act on Pandas DataFrames, DuckDB tables or views (which can poin
...2, 3, 4], 'j': ["one", "two", "three", "four"]}) # create a DuckDB relation from a dataframe rel = con.from_df(input_df) # chain together relational operators (this is a lazy operation, so the operations are not yet executed) # equivalent to: SELECT i...'i DESC').limit(2) # trigger execution by requesting .df() of the relation # .df() could have been added to the end of the chain above - it was separated for clarity output_df = transformed_rel.df() Relational operators can also be used to group rows,... - https://man.mtool.work/duckdb/2.0/guides/python/relational_api_pandas.html
- PostgreSQL Extension and the Secret Manager 8287
- User credentials and other PostgreSQL database connection details can be stored using the DuckDB Secrets Manager . The following syntax can be used to create a secret: CREATE SECRET ( TYPE postgres, HOST '127.0.0.1', PORT 5432, DATABASE pos
...rds generate-db-auth-token command in the example above: sql CREATE SECRET aws_rds_secret1 ( TYPE rds, PROVIDER credential_chain, CHAIN 'env;sso;', REGION 'eu-west-1', RDS_USER 'postgres', RDS_HOST 'database-1-instance-1.xxxxxxxxxxxx.eu-west-1.rds.ama...ws_rds_secret1 ); The secret of type rds requires the aws extension to be installed and allows to configure AWS Credential Chain the same way as with the secret of type s3 , see details in the AWS extension documentation . Storing Secrets inside a Pos..., SECRET_STORAGE_TABLE duckdb_secrets ); CREATE OR REPLACE SECRET s3_secret1 IN postgres_p1 ( TYPE s3, PROVIDER credential_chain, CHAIN 'env;sso;', PROFILE 'DatabaseAdministrator-⟨account_id⟩', REGION 'eu-west-1' ); When IN postgres_⟨attached_database... - https://man.mtool.work/duckdb/2.0/core_extensions/postgres/secrets.html
- Hugging Face Support 8047
- The httpfs extension introduces support for the hf:// protocol to access datasets hosted in Hugging Face repositories. See the announcement blog post for details. Usage Hugging Face repositories can be queried using the following URL patter
...CONFIG provider, use the following command: CREATE SECRET hf_token ( TYPE huggingface, TOKEN 'your_hf_token' ); credential_chain Automatically tries to fetch credentials. For the Hugging Face token, it will try to get it from ~/.cache/huggingface/toke...n . To create a secret using the credential_chain provider, use the following command: CREATE SECRET hf_token ( TYPE huggingface, PROVIDER credential_chain );... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/hugging_face.html
- Delta Extension 7927
- The delta extension adds support for the Delta Lake open-source storage format . It is built using the Delta Kernel . The extension reads Delta tables, local and remote, and writes them: it can create tables and append to them. For implemen
...ROM delta_scan('s3://⟨my_bucket⟩/⟨path⟩'); To authenticate, create a secret : CREATE SECRET ( TYPE s3, PROVIDER credential_chain ); SELECT * FROM delta_scan('s3://⟨my_bucket⟩/⟨path⟩'); For a public bucket, you may need to set its region in a secret: C...lta_scan('az://⟨my_container⟩/⟨path⟩'); To authenticate, create a secret : CREATE SECRET ( TYPE azure, PROVIDER credential_chain, ACCOUNT_NAME '⟨storage_account_name⟩' ); SELECT * FROM delta_scan('az://⟨my_container⟩/⟨path⟩'); Reading from Google Clou...ror. Attach its catalog instead, for example with the unity_catalog extension , and query the table through it. Credential Chains in Delta For some network operations, the delta extension uses delta-kernel-rs and object_store , which try credential so... - https://man.mtool.work/duckdb/2.0/core_extensions/delta.html
- Catalogs 7635
- The iceberg extension supports attaching Iceberg REST Catalogs. Attaching a catalog is required for writing to Iceberg . Before attaching an Iceberg REST Catalog, you must install the iceberg extension by following the instructions located
...~/.aws directory by creating the following secret using the Secrets Manager : CREATE SECRET ( TYPE s3, PROVIDER credential_chain ); Alternatively, you can set the values manually: CREATE SECRET ( TYPE s3, KEY_ID '⟨key_id⟩', SECRET '⟨secret⟩', REGION '...se (a.k.a. AWS Glue) catalog. Create an S3 secret using the Secrets Manager : CREATE SECRET ( TYPE s3, PROVIDER credential_chain, CHAIN sts, ASSUME_ROLE_ARN 'arn:aws:iam::⟨account_id⟩:role/⟨role⟩', REGION 'us-east-2' ); In this example we use an STS t... - https://man.mtool.work/duckdb/2.0/core_extensions/iceberg/catalogs.html
DuckDB 2.0 Documentation / man.mtool.work