Results of 1 - 10 of about 22 for stream (0.002 sec.)
- Handle Results 14444
- Overview Beyond the standard JDBC ResultSet , the DuckDB driver can hand results to Apache Arrow , stream large results instead of materializing them, and expose raw columnar data chunks. This page covers each of these result-handling optio
...esult set integrates with Apache Arrow through the Java Arrow bindings . A DuckDBResultSet can export its rows as an Arrow stream for consumption by Arrow-based tools, and an Arrow stream produced elsewhere can be registered on a connection and queried...as a DuckDB table. Refer to the API Reference for the type signatures. Arrow Export The following example exports an Arrow stream from a query result and consumes it using the Java Arrow bindings. Each batch is loaded in turn, and its vectors are read...esultSet) stmt.executeQuery(); var allocator = new RootAllocator()) { try (var reader = (ArrowReader) resultset.arrowExportStream(allocator, 256)) { while (reader.loadNextBatch()) { System.out.println(reader.getVectorSchemaRoot().getVector("generate_se... - https://man.mtool.work/duckdb/2.0/clients/java/result_handling.html
- ADBC Client 11363
- Installation To use the DuckDB ADBC client, download the libduckdb archive for your platform and follow the instructions below . For Python, install the duckdb package instead, as described in the Python section . The latest stable version
...hat retrieve metadata about the database. In general, these functions will return Arrow objects, specifically an ArrowArrayStream. Function name Description Arguments Example ConnectionGetInfo Get metadata about the driver and database. (AdbcConnection...*, const uint32_t*, size_t, ArrowArrayStream*, AdbcError*) AdbcConnectionGetInfo(&adbc_connection, NULL, 0, &arrow_stream, &adbc_error) ConnectionGetObjects Get a hier...as, tables and columns. (AdbcConnection*, int, const char*, const char*, const char*, const char**, const char*, ArrowArrayStream*, AdbcError*) AdbcConnectionGetObjects(&adbc_connection, ADBC_OBJECT_DEPTH_ALL, NULL, NULL, NULL, NULL, NULL, &arrow_strea... - https://man.mtool.work/duckdb/2.0/clients/adbc.html
- Handle Results 9949
- Overview DuckDB is columnar, and beyond reading a result set row by row through database/sql , the Go client can exchange whole results with Apache Arrow as a stream of record batches, and register an Arrow stream as a queryable view. Both
...) } QueryContext() accepts bound parameters as trailing arguments, the same as database/sql queries . Registering an Arrow Stream as a View An Arrow stream produced elsewhere in a program, an array.RecordReader , can be registered as a DuckDB view and... - https://man.mtool.work/duckdb/2.0/clients/go/result_handling.html
- Export to Apache Arrow 9595
- All results of a query can be exported to an Apache Arrow Table using the to_arrow_table function. Alternatively, results can be returned as a RecordBatchReader using the to_arrow_reader function and results can be read one batch at a time.
...Call these functions on the relation that duckdb.sql , con.sql or DuckDB's Relational API returns: its reader streams the result. On a con.execute() result they first copy all rows and warn with a UserWarning , and the reader does not stre...a single chunk here print(batch.to_pandas()) Export through the Arrow PyCapsule Interface A relation implements __arrow_c_stream__ of the Arrow PyCapsule interface , so libraries that accept an Arrow stream read it directly: pyarrow.table(rel) , pyarr...ow.RecordBatchReader.from_stream(rel) and polars.DataFrame(rel) . Polars reads it this way even without pyarrow installed. import duckdb import polars as p... - https://man.mtool.work/duckdb/2.0/guides/python/export_arrow.html
- Handle Results 8804
- Overview Beyond reading a result set row by row , the Rust client can hand a query result to Apache Arrow as a stream of record batches, register an Arrow batch as a queryable table, and return results as Polars data frames. Each of these c
...t_batches(&batches).unwrap(); get_schema() on the returned handle reports the Arrow schema DuckDB inferred for the result. Streaming Arrow Results query_arrow() runs the statement to completion and buffers the whole result on the client side, so a larg...anced one batch at a time. For a result that should be consumed lazily, fetching chunks only as the iterator advances, use stream_arrow() , which is otherwise identical: let mut stmt = conn.prepare("SELECT * FROM big_table")?; for batch in stmt.stream_...time println!("{} rows", batch.num_rows()); } DuckDB may still materialize the result internally for some statements. The streaming iterator panics if fetching or Arrow conversion fails after execution has started. Querying an Arrow Batch An Arrow Rec... - https://man.mtool.work/duckdb/2.0/clients/rust/result_handling.html
- Node.js Client (Neo) 8636
- Installation To use the DuckDB Node.js client, visit the Node.js installation page . The latest stable version of the DuckDB Node.js (Neo) client is 2.0.0. An API for using DuckDB in Node.js . The primary package, @duckdb/node-api , is a hi
...S timestampSecondsValue TIMESTAMP_MS timestampMillisValue TIMESTAMP_NS timestampNanosValue UNION unionValue UUID uuidValue Stream Results Streaming results evaluate lazily when rows are read. const result = await connection.stream('from range(10_000)')...nection.runAndReadAll('from test_all_types()'); const rows = reader.getRows(); // OR: const columns = reader.getColumns(); Stream and read up to (at least) some number of rows: const reader = await connection.streamAndReadUntil( 'from range(5000)', 100...ws(); // rows.length === 2048. (Rows are read in chunks of 2048.) Read rows incrementally: const reader = await connection.streamAndRead('from range(5000)'); reader.readUntil(2000); // reader.currentRowCount === 2048 (Rows are read in chunks of 2048.)... - https://man.mtool.work/duckdb/2.0/clients/node_neo/overview.html
- CONNECT and DISCONNECT Statements 8636
- New Support for the CONNECT and DISCONNECT statements was introduced in DuckDB 2.0. The CONNECT statement routes the current session to a remote database, so that subsequent queries execute on that remote and their results stream back. DISC
...erver.example.com' AS qk (TOKEN '⟨my_token⟩'); CONNECT qk; SELECT count(*) FROM events; -- executes on the server, results stream back DISCONNECT; Connect directly to a remote database using a connection string. For databases other than DuckDB, the con... - https://man.mtool.work/duckdb/2.0/sql/statements/connect.html
- Connect 8450
- Overview Every DuckDB feature reachable from Java starts with a Connection . This page covers the JDBC URL forms the driver accepts, how DuckDB and driver options are passed and which of them win, how connections come to share (or not share
...ols that accept a connection string but no Properties object: jdbc:duckdb:/tmp/my_database;threads=4;memory_limit=4GB;jdbc_stream_results=true Each entry is a key=value pair and surrounding whitespace is trimmed. The driver splits the URL on ; and each...EAD_ONLY , READ_WRITE , or AUTOMATIC . custom_user_agent Append a custom string to the user agent reported to DuckDB. jdbc_stream_results Stream result sets instead of materializing them. See Streaming Results . jdbc_auto_commit Set the default auto-co...the database instance alive after its last connection closes. Disabled by default. Enabled by default, together with jdbc_stream_results , for ducklake: URLs, so that BI tools such as Metabase do not repeatedly close and re-create the instance or acci... - https://man.mtool.work/duckdb/2.0/clients/java/connecting.html
- Data Ingestion 8181
- This page contains examples for data ingestion to Python using DuckDB. First, import the DuckDB package: import duckdb Then, proceed with any of the following sections. CSV Files CSV files can be read using the read_csv function, called eit
...Custom Arrow Objects DuckDB can also consume any object that implements the Arrow PyCapsule interface through an __arrow_c_stream__ method. Such objects can be queried directly through replacement scans and passed to duckdb.from_arrow() : import duckdb...import pyarrow as pa class MyArrowStream: def __init__(self, table): self._table = table def __arrow_c_stream__(self, requested_schema=None): return self._table.__...arrow_c_stream__(requested_schema) obj = MyArrowStream(pa.table({"i": [1, 2, 3]})) print(duckdb.sql("SELECT sum(i) AS s FROM obj").fetcha... - https://man.mtool.work/duckdb/2.0/clients/python/data_ingestion.html
- Quack Remote Protocol 8181
- The Quack extension turns a DuckDB instance into a server that other DuckDB instances (clients) can connect to over HTTP. This page covers the protocol at a glance and walks through basic usage on both sides of the wire. For the full list o
...round-trip per query. After the initial connection handshake, a query needs only one request–response pair. Large results stream back in chunks via follow-up FETCH requests, optionally parallelized on multiple threads. Default port: 9494 . All URIs us...SELECT 42', token = '⟨MY_QUACK_TOKEN_01234567890ABCDEF⟩', disable_ssl = true ); The query executes remotely and the server streams the result back. Errors occurring on the server (parse errors, missing tables, etc.) are shown locally in the DuckDB clie..., the CONNECT statement routes the whole session to it, so that subsequent queries execute on the server and their results stream back. This is the recommended way to work with a remote Quack database: you write plain SQL without qualifying every table... - https://man.mtool.work/duckdb/2.0/quack/overview.html
DuckDB 2.0 Documentation / man.mtool.work