Results of 1 - 6 of about 6 for recordbatch (0.001 sec.)
- Handle Results 13088
- Overview Beyond reading a result set row by row , the Rust client can hand a query result to Apache Arrow as a stream of record batches, register an Arrow batch as a queryable table, and return results as Polars data frames. Each of these c
...w types through duckdb::arrow . Reading a Result as Arrow Call query_arrow() on a prepared statement to get an iterator of RecordBatch . Collecting it materializes the whole result, while iterating it reads one batch at a time: use duckdb::{Connection, Resu...lt}; use duckdb::arrow::record_batch::RecordBatch; use duckdb::arrow::util::pretty::print_batches; let conn = Connection::open_in_memory()?; let mut stmt = conn.prepare("SE...LECT * FROM generate_series(1, 5)")?; let batches: Vec<RecordBatch> = stmt.query_arrow([])?.collect(); print_batches(&batches).unwrap(); get_schema() on the returned handle reports the Arro... - https://man.mtool.work/duckdb/2.0/clients/rust/result_handling.html
- SQL on Apache Arrow 9157
- DuckDB can query multiple different types of Apache Arrow objects. DuckDB pushes the filters of a query that it can translate, such as comparisons, into the scan of an Arrow dataset. Arrow tables, record batches, scanners and RecordBatchRea
...ders are read in full and filtered afterwards. A RecordBatchReader can be scanned only once; to query its data more than once, read it into a table with read_all() before the first qu...arrow_scanner" and return as an Arrow Table results = con.sql("SELECT * FROM arrow_scanner").to_arrow_table() Apache Arrow RecordBatchReaders Arrow RecordBatchReaders are a reader for Arrow's streaming binary format and can also be queried directly as if th...between language runtimes. import duckdb import pyarrow as pa # connect to an in-memory database con = duckdb.connect() my_recordbatch = pa.RecordBatch.from_pydict({'i': [1, 2, 3, 4], 'j': ["one", "two", "three", "four"]}) my_recordbatchreader = pa.ipc.Reco... - https://man.mtool.work/duckdb/2.0/guides/python/sql_on_arrow.html
- Import Data 8554
- Overview The Rust client offers two ways to get data into DuckDB in bulk: the high-performance Appender for inserting rows from Rust, and reading directly from data files such as Parquet, CSV, and JSON. Both are described below. Appender Th
...on open), and Panic . Appending Arrow Data With the appender-arrow feature enabled, an Appender can append an Apache Arrow RecordBatch directly, which avoids building a parameter list per row. See Handle Results for the Arrow integration. Reading Data Files..._parquet() , adapted from the crate's parquet example : use duckdb::{Connection, Result}; use duckdb::arrow::record_batch::RecordBatch; let conn = Connection::open_in_memory()?; conn.execute_batch("INSTALL parquet; LOAD parquet;")?; let batches: Vec<RecordB...Import — DuckDB's full set of bulk-loading options. Handle Results — the Apache Arrow integration used to append and read record batches. Run Queries — sending the CREATE TABLE and COPY statements the file readers build on.... - https://man.mtool.work/duckdb/2.0/clients/rust/data_import.html
- Handle Results 8280
- Overview DuckDB is columnar, and beyond reading a result set row by row through database/sql , the Go client can exchange whole results with Apache Arrow as a stream of record batches, and register an Arrow stream as a queryable view. Both
..., then call QueryContext() to run a query and get back an array.RecordReader . Iterate it with Next() , reading one arrow.RecordBatch at a time, and release it when done: connector, err := duckdb.NewConnector("", nil) if err != nil { log.Fatal(err) } defer...FROM generate_series(1, 10)") if err != nil { log.Fatal(err) } defer reader.Release() for reader.Next() { record := reader.RecordBatch() // Process one record batch at a time. log.Printf("%d rows", record.NumRows()) } if err := reader.Err(); err != nil { lo... - https://man.mtool.work/duckdb/2.0/clients/go/result_handling.html
- Rust Client 7951
- Installation To use the DuckDB Rust client, visit the Rust installation page . The latest stable version of the DuckDB Rust client is 2.0.0. The DuckDB Rust client, duckdb-rs , is an ergonomic wrapper over the DuckDB C API that exposes an i
...ab Base support for table functions . vtab-arrow Apache Arrow integration for table functions, converting between an Arrow RecordBatch and DuckDB data chunks. Implies vtab . vscalar Scalar functions . Implies vtab-arrow . vscalar-arrow Arrow-optimized scala...r functions, intended for use alongside vscalar . appender-arrow Append Arrow RecordBatch es through the Appender . Implies vtab-arrow . loadable-extension Experimental. Build a loadable DuckDB extension rather t... - https://man.mtool.work/duckdb/2.0/clients/rust/overview.html
- Integration with PyTorch 7750
- PyTorch tensors can be built from DuckDB query results by exporting each result batch as Apache Arrow . This pattern is useful when training or serving models on Parquet data because DuckDB can filter and project the input before the batche
...-U duckdb numpy pyarrow torch DuckDB to PyTorch This example queries a Parquet-backed Arrow dataset, streams the result as RecordBatch objects and converts each batch into feature and label tensors. import pathlib import tempfile import duckdb import numpy... - https://man.mtool.work/duckdb/2.0/guides/python/pytorch.html
PREV
1
NEXT
DuckDB 2.0 Documentation / man.mtool.work