Results of 1 - 7 of about 7 for streams (0.001 sec.)
- Integration with PyTorch 13104
- PyTorch tensors can be built from DuckDB query results by exporting each result batch as Apache Arrow . This pattern is useful when training or serving models on Parquet data because DuckDB can filter and project the input before the batche
...tallation pip install -U duckdb numpy pyarrow torch DuckDB to PyTorch This example queries a Parquet-backed Arrow dataset, streams the result as RecordBatch objects and converts each batch into feature and label tensors. import pathlib import tempfile i...he dataset scan, so only the requested rows and columns are converted to tensors. The to_arrow_reader method of a relation streams the result in batches instead of materializing the full result in memory at once. While the loop holds the reader, other q... - https://man.mtool.work/duckdb/2.0/guides/python/pytorch.html
- HTTP CSV Import 9942
- To load a CSV file directly over HTTP(S), the httpfs extension is required. This can be installed using the INSTALL SQL command. This only needs to be run once. INSTALL httpfs; To load the httpfs extension for usage, use the LOAD SQL comman
...1 │ │ Casablanca (Golden Mile) │ 15.0 │ 1 │ └────────────────────────────┴──────────────────────┴───────┘ Because read_csv streams the file rather than materializing it first, this works the same way for CSVs far larger than fit in memory, and pairs nat... - https://man.mtool.work/duckdb/2.0/guides/network_cloud_storage/http_csv_import.h...
- Export to Apache Arrow 8869
- All results of a query can be exported to an Apache Arrow Table using the to_arrow_table function. Alternatively, results can be returned as a RecordBatchReader using the to_arrow_reader function and results can be read one batch at a time.
...Call these functions on the relation that duckdb.sql , con.sql or DuckDB's Relational API returns: its reader streams the result. On a con.execute() result they first copy all rows and warn with a UserWarning , and the reader does not strea... - https://man.mtool.work/duckdb/2.0/guides/python/export_arrow.html
- Securing Quack with a Reverse Proxy 8524
- By default, the Quack server binds to localhost and speaks plain HTTP. See Security for the rationale behind these configurations. For any deployment beyond local-only, the recommended pattern is the same as for any other HTTP-based databas
...ons to keep server-side `quack_connection_id` state alive. proxy_http_version 1.1; proxy_set_header Connection ""; # Quack streams results via repeated FETCH responses; buffering # through nginx defeats the streaming and inflates memory. proxy_buffering... - https://man.mtool.work/duckdb/2.0/quack/setup/reverse_proxy.html
- Python DB API 7815
- The standard DuckDB Python API provides a SQL interface compliant with the DB-API 2.0 specification described by PEP 249 similar to the SQLite Python API . Connection To use the module, you must first create a DuckDBPyConnection object that
...use a relation instead: con.sql(query).to_arrow_table() , con.sql(query).pl() or con.sql(query).to_arrow_reader() , which streams. Transactions con.begin() , con.commit() and con.rollback() start, commit and roll back a transaction, as the BEGIN , COMM... - https://man.mtool.work/duckdb/2.0/clients/python/dbapi.html
- Quack Remote Protocol 7815
- The Quack extension turns a DuckDB instance into a server that other DuckDB instances (clients) can connect to over HTTP. This page covers the protocol at a glance and walks through basic usage on both sides of the wire. For the full list o
...SELECT 42', token = '⟨MY_QUACK_TOKEN_01234567890ABCDEF⟩', disable_ssl = true ); The query executes remotely and the server streams the result back. Errors occurring on the server (parse errors, missing tables, etc.) are shown locally in the DuckDB clien... - https://man.mtool.work/duckdb/2.0/quack/overview.html
- Write User Defined Functions 7470
- Overview The JDBC driver can register user-defined functions (UDFs) written in Java, including scalar functions and table functions. This page covers building and registering both, from the simple functional-interface overloads to the vecto
...as to live in the init (or local-init) state so it survives across calls, rather than in a local variable. The apply below streams a bounded integer series in chunks. It also dispatches on each column's declared type with DuckDBWritableVector.getType()... - https://man.mtool.work/duckdb/2.0/clients/java/functions.html
PREV
1
NEXT
DuckDB 2.0 Documentation / man.mtool.work