Results of 1 - 10 of about 26 for ⟩', (0.001 sec.)
- Catalogs 14448
- The iceberg extension supports attaching Iceberg REST Catalogs. Attaching a catalog is required for writing to Iceberg . Before attaching an Iceberg REST Catalog, you must install the iceberg extension by following the instructions located
...workflow to store login credentials for the OAuth2 service. CREATE SECRET iceberg_secret ( TYPE iceberg, CLIENT_ID '⟨admin⟩', CLIENT_SECRET '⟨password⟩', OAUTH2_SERVER_URI '⟨http://iceberg_rest_catalog_url.com/v1/oauth/tokens⟩' ); If you already ha...E s3, PROVIDER credential_chain ); Alternatively, you can set the values manually: CREATE SECRET ( TYPE s3, KEY_ID '⟨key_id⟩', SECRET '⟨secret⟩', REGION '⟨region⟩' ); For the full range of credential options (assumed roles, SSO, web identity, and mo...ager : CREATE SECRET ( TYPE s3, PROVIDER credential_chain, CHAIN sts, ASSUME_ROLE_ARN 'arn:aws:iam::⟨account_id⟩:role/⟨role⟩', REGION 'us-east-2' ); In this example we use an STS token, but other authentication methods are supported . Then, connect... - https://man.mtool.work/duckdb/2.0/core_extensions/iceberg/catalogs.html
- HTTP(S) Support 10480
- With the httpfs extension, it is possible to directly query files over the HTTP(S) protocol. This works for all files supported by DuckDB or its various extensions, and provides read-only access. SELECT * FROM 'https://domain.tld/file.exten
...es. You can add an HTTP proxy using the Secrets Manager : CREATE SECRET http_proxy ( TYPE http, HTTP_PROXY '⟨http_proxy_url⟩', HTTP_PROXY_USERNAME '⟨username⟩', HTTP_PROXY_PASSWORD '⟨password⟩' ); You can also set the scope for an HTTP proxy using t...he SCOPE keyword. CREATE SECRET http_proxy ( TYPE HTTP, SCOPE ['⟨https://duckdb.org⟩', '⟨https://some-other-website.org⟩'], HTTP_PROXY '⟨http_proxy_url⟩', HTTP_PROXY_USERNAME '⟨username⟩', HTTP_PROXY_PASSWORD...e advantage of the scope parameter: CREATE SECRET enable_ssl_for_your_website ( TYPE HTTP, SCOPE 'https://⟨your-website.com⟩', VERIFY_SSL 1 );... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/https.html
- Amazon RDS with IAM Authentication 10001
- Managed PostgreSQL databases on Amazon RDS and Aurora can authenticate with short-lived IAM tokens instead of a static password. DuckDB supports this through the postgres and aws extensions: the aws extension generates the token from an AWS
...mint the IAM token: CREATE SECRET aws_rds_secret ( TYPE rds, PROVIDER credential_chain, CHAIN 'env;sso', REGION '⟨eu-west-1⟩', RDS_USER '⟨postgres⟩', RDS_HOST '⟨instance⟩.⟨identifier⟩.⟨region⟩.rds.amazonaws.com', RDS_PORT '5432' ); Then create a sec...CRET pg_rds_secret ( TYPE postgres, HOST '⟨instance⟩.⟨identifier⟩.⟨region⟩.rds.amazonaws.com', PORT '5432', USER '⟨postgres⟩', DATABASE '⟨postgres⟩', SSLMODE require, AWS_RDS_SECRET aws_rds_secret ); Connecting and Querying Attach the database using... - https://man.mtool.work/duckdb/2.0/guides/database_integration/rds_iam.html
- AWS Extension 9340
- The aws extension adds functionality, e.g., authentication, on top of the httpfs extension's S3 capabilities , using the AWS SDK. Installing and Loading The aws extension will be transparently autoloaded on first use from the official exten
...LACE SECRET secret ( TYPE s3, PROVIDER credential_chain, CHAIN 'sts', ASSUME_ROLE_ARN 'arn:aws:iam::⟨account_id⟩:role/⟨role⟩', EXTERNAL_ID '⟨external_id⟩', REGION '⟨us-east-1⟩' ); The sts chain value requires an ASSUME_ROLE_ARN value. If the selecte...ET secret ( TYPE s3, PROVIDER credential_chain, CHAIN 'web_identity', ASSUME_ROLE_ARN 'arn:aws:iam::⟨account_id⟩:role/⟨role⟩', WEB_IDENTITY_TOKEN_FILE '⟨/var/run/secrets/eks.amazonaws.com/serviceaccount/token⟩' ); Single Sign-On (SSO) DuckDB can use...it on the secret: CREATE OR REPLACE SECRET secret ( TYPE s3, PROVIDER credential_chain, HTTP_PROXY '⟨proxy.example.com:8080⟩', HTTP_PROXY_USERNAME '⟨username⟩', HTTP_PROXY_PASSWORD '⟨password⟩' ); Region Resolution If no region is provided explicitl... - https://man.mtool.work/duckdb/2.0/core_extensions/aws.html
- Azure Extension 9340
- The azure extension lets DuckDB read and write files in Azure Blob Storage and Azure Data Lake Storage (ADLS) Gen2. Installing and Loading The azure extension will be transparently autoloaded on first use from the official extension reposit
...ECT_ID or RESOURCE_ID : CREATE SECRET azure_mi ( TYPE azure, PROVIDER managed_identity, ACCOUNT_NAME '⟨storage_account_name⟩', CLIENT_ID '⟨client_id⟩' ); Without an ID, the provider works like credential_chain . With several identities and no ID, th...cipal (SPN) . With a client secret: CREATE SECRET azure_spn ( TYPE azure, PROVIDER service_principal, TENANT_ID '⟨tenant_id⟩', CLIENT_ID '⟨client_id⟩', CLIENT_SECRET '⟨client_secret⟩', ACCOUNT_NAME '⟨storage_account_name⟩' ); Or with a certificate:...CREATE SECRET azure_spn_cert ( TYPE azure, PROVIDER service_principal, TENANT_ID '⟨tenant_id⟩', CLIENT_ID '⟨client_id⟩', CLIENT_CERTIFICATE_PATH '⟨client_cert_path⟩', ACCOUNT_NAME '⟨storage_account_name⟩' ); ACCESS_TOK... - https://man.mtool.work/duckdb/2.0/core_extensions/azure.html
- Fastly Object Storage Import 9240
- Prerequisites For Fastly Object Storage, the S3 Compatibility API allows you to use DuckDB's S3 support to read and write from Fastly buckets. This requires the httpfs extension , which can be installed using the INSTALL SQL command. This o
...nerate an S3 auth token and create an S3 secret in DuckDB: CREATE SECRET my_secret ( TYPE s3, KEY_ID '⟨AKIAIOSFODNN7EXAMPLE⟩', SECRET '⟨wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY⟩', URL_STYLE 'path', REGION '⟨us-east⟩', ENDPOINT '⟨us-east⟩.object.fast... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/fastly_object_sto...
- S3 API Support 9059
- The httpfs extension supports reading/writing/ globbing files on object storage servers using the S3 API. S3 offers a standard API to read and write to remote files (while regular http servers, predating S3, do not offer a common write API)
...lly providing a key. For example: CREATE OR REPLACE SECRET secret ( TYPE s3, PROVIDER config, KEY_ID '⟨AKIAIOSFODNN7EXAMPLE⟩', SECRET '⟨wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY⟩', REGION '⟨us-east-1⟩' ); Tip If you get an IO Error ( Connection error...e KMS_KEY_ID option: CREATE OR REPLACE SECRET secret ( TYPE s3, PROVIDER credential_chain, CHAIN config, REGION '⟨eu-west-1⟩', KMS_KEY_ID 'arn:aws:kms:⟨region⟩:⟨account_id⟩:⟨key⟩/⟨key_id⟩', SCOPE 's3://⟨bucket-sub-path⟩' ); R2 Secrets While Cloudfla...t type, R2 , to make configuring it a bit simpler: CREATE OR REPLACE SECRET secret ( TYPE r2, KEY_ID '⟨AKIAIOSFODNN7EXAMPLE⟩', SECRET '⟨wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY⟩', ACCOUNT_ID '⟨my_account_id⟩' ); Note the addition of the ACCOUNT_ID w... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/s3api.html
- S3 Parquet Export 8959
- To write a Parquet file to S3, the httpfs extension is required. This can be installed using the INSTALL SQL command. This only needs to be run once. INSTALL httpfs; To load the httpfs extension for usage, use the LOAD SQL command: LOAD htt
...parameter should match the region of the bucket you want to access. CREATE SECRET ( TYPE s3, KEY_ID '⟨AKIAIOSFODNN7EXAMPLE⟩', SECRET '⟨wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY⟩', REGION '⟨us-east-1⟩' ); Tip If you get an IO Error ( Connection error...ou need to create HMAC keys and provide the credentials as follows: CREATE SECRET ( TYPE gcs, KEY_ID '⟨AKIAIOSFODNN7EXAMPLE⟩', SECRET '⟨wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY⟩' ); After setting up the GCS credentials, you can export using: COPY ⟨t... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/s3_export.html
- Cloudflare R2 Import 8397
- Prerequisites For Cloudflare R2, the S3 Compatibility API allows you to use DuckDB's S3 support to read and write from R2 buckets. This requires the httpfs extension , which can be installed using the INSTALL SQL command. This only needs to
...need to generate an S3 auth token and create an R2 secret in DuckDB: CREATE SECRET ( TYPE r2, KEY_ID '⟨AKIAIOSFODNN7EXAMPLE⟩', SECRET '⟨wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY⟩', ACCOUNT_ID '⟨your-32-character-hexadecimal-account-ID⟩' ); Querying A... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/cloudflare_r2_imp...
- S3 Express One 8397
- In late 2023, AWS announced the S3 Express One Zone , a high-speed variant of traditional S3 buckets. DuckDB can read S3 Express One buckets using the httpfs extension . Credentials and Configuration The configuration of S3 Express One buck
...an S3 Express One bucket, configure the Secrets manager as follows: CREATE SECRET ( TYPE s3, KEY_ID '⟨AKIAIOSFODNN7EXAMPLE⟩', SECRET '⟨wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY⟩', REGION '⟨us-east-1⟩', ENDPOINT 's3express-⟨use1-az5⟩.⟨us-east-1⟩.amaz... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/s3_express_one.ht...
DuckDB 2.0 Documentation / man.mtool.work