Results of 1 - 10 of about 14 for Google (0.001 sec.)
- Catalogs 11885
- The iceberg extension supports attaching Iceberg REST Catalogs. Attaching a catalog is required for writing to Iceberg . Before attaching an Iceberg REST Catalog, you must install the iceberg extension by following the instructions located
...implementation, see Amazon S3 Tables , AWS Glue , Cloudflare R2 Data Catalog , Apache Polaris , Lakekeeper , SeaweedFS and Google Cloud BigLake . Attaching an Iceberg REST Catalog Most Iceberg REST Catalogs authenticate via OAuth2. You can use the exis...total FROM seaweedfs_catalog.sales.orders GROUP BY region ORDER BY total DESC; DROP TABLE seaweedfs_catalog.sales.orders; Google Cloud BigLake To attach to a Google Cloud BigLake catalog, you can use extra HTTP headers to specify the GCP project for b...illing purposes. First, get your Google Cloud access token: gcloud auth application-default print-access-token Then create a secret with the token and extra heade... - https://man.mtool.work/duckdb/2.0/core_extensions/iceberg/catalogs.html
- Google Cloud Storage Import 11693
- Prerequisites The Google Cloud Storage (GCS) can be used via the httpfs extension . This can be installed with the INSTALL httpfs SQL command. This only needs to be run once. Credentials and Configuration You need to create HMAC keys and de
...le in read-only mode: LOAD httpfs; ATTACH 'gs://⟨gcs_bucket⟩/⟨file.duckdb⟩' AS ⟨duckdb_database⟩ (READ_ONLY); Databases in Google Cloud Storage can only be attached in read-only mode.... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/gcs_import.html
- S3 API Support 10940
- The httpfs extension supports reading/writing/ globbing files on object storage servers using the S3 API. S3 offers a standard API to read and write to remote files (while regular http servers, predating S3, do not offer a common write API)
...API, that is now common among industry storage providers. Platforms The httpfs filesystem is tested with AWS S3 , Minio , Google Cloud and lakeFS . Other services that implement the S3 API (such as Cloudflare R2 , SeaweedFS , and Tigris ) should also...with r2:// , for example: SELECT * FROM read_parquet('r2://⟨some-file-that-uses-an-r2-secret⟩.parquet'); GCS Secrets While Google Cloud Storage is accessed by DuckDB using the S3 API, DuckDB has a special Secret type, GCS , to make configuring it a bit...⟩', SECRET '⟨my_hmac_secret_key⟩' ); Important : The KEY_ID and SECRET values must be HMAC keys generated specifically for Google Cloud Storage interoperability. These are not the same as regular GCP service account keys or access tokens. You can creat... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/s3api.html
- S3 Parquet Import 10940
- Prerequisites To load a Parquet file from S3, the httpfs extension is required. This can be installed using the INSTALL SQL command. This only needs to be run once. INSTALL httpfs; To load the httpfs extension for usage, use the LOAD SQL co
...rectly, Parquet files can be read from S3 using the following command: SELECT * FROM read_parquet('s3://⟨bucket⟩/⟨file⟩'); Google Cloud Storage (GCS) and Cloudflare R2 DuckDB can also handle Google Cloud Storage (GCS) and Cloudflare R2 via the S3 API.... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/s3_import.html
- Attach to a DuckDB Database over HTTPS or S3 8454
- You can establish a read-only connection to a DuckDB instance via HTTPS or the S3 API. Prerequisites This guide requires the httpfs extension , which can be installed using the INSTALL httpfs SQL command. This only needs to be run once. Att
...: SELECT count(*) AS num_stations FROM stations_db.stations; num_stations 578 Connecting to S3-compatible APIs such as the Google Cloud Storage ( gs:// ) is also supported. Limitations Only read-only connections are allowed, writing the database via th... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/duckdb_over_https...
- Using fsspec Filesystems 8454
- DuckDB support for fsspec filesystems allows querying data in filesystems that DuckDB's httpfs extension does not support. fsspec has a large number of inbuilt filesystems , and there are also many external implementations . This capability
...tpfs extension is available in many DuckDB clients. Example The following is an example of using fsspec to query a file in Google Cloud Storage (instead of using their S3-compatible API). Firstly, you must install duckdb and fsspec , and a filesystem i... - https://man.mtool.work/duckdb/2.0/guides/python/filesystems.html
- S3 Parquet Export 8262
- To write a Parquet file to S3, the httpfs extension is required. This can be installed using the INSTALL SQL command. This only needs to be run once. INSTALL httpfs; To load the httpfs extension for usage, use the LOAD SQL command: LOAD htt
...s can be written to S3 using the following command: COPY ⟨table_name⟩ TO 's3://⟨s3-bucket⟩/⟨filename⟩.parquet'; Similarly, Google Cloud Storage (GCS) is supported through the Interoperability API. You need to create HMAC keys and provide the credential... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/s3_export.html
- Jupyter Notebooks 8262
- DuckDB's Python client can be used directly in Jupyter notebooks with no additional configuration, as the first section shows. Additional libraries can simplify SQL query development; the rest of this guide uses the JupySQL package. See oth
...ook. Add .df() to keep the result as a pandas DataFrame. Library Installation This JupySQL workflow is also available as a Google Colab notebook . Four additional libraries improve the DuckDB experience in Jupyter notebooks. jupysql : Convert a Jupyter...B (optional) Run these pip install commands from the command line if Jupyter Notebook is not yet installed. Otherwise, see Google Colab link above for an in-notebook example: pip install duckdb Install Jupyter Notebook: pip install notebook Or JupyterL... - https://man.mtool.work/duckdb/2.0/guides/python/jupyter.html
- S3 and AWS Authentication 7702
- This page is the entry point for authenticating to S3 and other AWS services from DuckDB. Authentication is configured using secrets , and the detailed reference material lives on the S3 API Support and AWS Extension pages. The sections bel
...S , and more) is documented in the Overview of S3 Secret Parameters . Platform-specific secret types for Cloudflare R2 and Google Cloud Storage are also available. Legacy Authentication Before DuckDB had a Secrets manager , credentials were loaded thro... - https://man.mtool.work/duckdb/2.0/core_extensions/httpfs/authentication.html
- Multiple Python Threads 7509
- This page demonstrates how to simultaneously insert into and read from a DuckDB database across multiple Python threads. This could be useful in scenarios where new data is flowing in and an analysis should be periodically re-run. Note that
...s is all within a single Python process (see the FAQ for details on DuckDB concurrency). Feel free to follow along in this Google Colab notebook . Setup First, import DuckDB and several modules from the Python standard library. Note: if using Pandas, a... - https://man.mtool.work/duckdb/2.0/guides/python/multiple_threads.html
DuckDB 2.0 Documentation / man.mtool.work