Results of 1 - 10 of about 24 for record (0.002 sec.)
- Merge Statement for SCD Type 2 13711
- This is a practical, step-by-step guide to using DuckDB's MERGE statement (introduced in v1.4.0) to perform upserts and build Slowly Changing Dimension Type 2 (SCD Type 2) tables . Type 2 SCDs let you keep full historical versions of record
...This table represents the type 2 SCD data (i.e., transaction data with history). CREATE TABLE IF NOT EXISTS master_ducks ( record_id INTEGER PRIMARY KEY, duck_id INTEGER NOT NULL, duck_name VARCHAR, breed VARCHAR, location VARCHAR, begin_date DATE NOT...NULL, end_date DATE, is_current BOOLEAN NOT NULL DEFAULT true ); CREATE SEQUENCE IF NOT EXISTS duck_record_seq START 1; INSERT INTO master_ducks VALUES (nextval('duck_record_seq'), 101, 'Quackers', 'Mallard', 'Pond A', CURRENT_DA...TE - INTERVAL '2 days', NULL, true), (nextval('duck_record_seq'), 102, 'Waddles', 'Pekin', 'Pond A', CURRENT_DATE - INTERVAL '2 days', NULL, true), (nextval('duck_record_seq'), 103,... - https://man.mtool.work/duckdb/2.0/guides/sql_features/merge.html
- ADBC Client 12543
- Installation To use the DuckDB ADBC client, download the libduckdb archive for your platform and follow the instructions below . For Python, install the duckdb package instead, as described in the Python section . The latest stable version
...hods, such as fetchone and fetchall . To load Arrow data into a table, pass the table name and the data, such as a pyarrow record batch or table, to cur.adbc_ingest . By default it creates the table and fails if the table exists; its mode argument also...ate_append" and "replace" , as in the ingestion modes above. import adbc_driver_duckdb.dbapi import pyarrow data = pyarrow.record_batch( [[1, 2, 3, 4], ["a", "b", "c", "d"]], names = ["ints", "strs"], ) with adbc_driver_duckdb.dbapi.connect("test.db")...for detailed installation instructions. The following example uses an in-memory DuckDB database to modify in-memory Arrow RecordBatches via SQL queries: package main import ( "bytes" "context" "fmt" "io" "github.com/apache/arrow-adbc/go/adbc" "github.... - https://man.mtool.work/duckdb/2.0/clients/adbc.html
- Handle Results 10408
- Overview DuckDB is columnar, and beyond reading a result set row by row through database/sql , the Go client can exchange whole results with Apache Arrow as a stream of record batches, and register an Arrow stream as a queryable view. Both
...described here are not compiled in. The interface builds on apache/arrow-go , so its array and arrow packages provide the record and schema types. Reading a Result as Arrow Wrap a raw DuckDB connection with duckdb.NewArrowFromConn() , then call QueryC...ontext() to run a query and get back an array.RecordReader . Iterate it with Next() , reading one arrow.RecordBatch at a time, and release it when done: connector, err := duck...und(), "SELECT * FROM generate_series(1, 10)") if err != nil { log.Fatal(err) } defer reader.Release() for reader.Next() { record := reader.RecordBatch() // Process one record batch at a time. log.Printf("%d rows", record.NumRows()) } if err := reader.... - https://man.mtool.work/duckdb/2.0/clients/go/result_handling.html
- Avro Extension 9457
- The avro extension enables DuckDB to read Apache Avro files. The avro extension was released as a community extension in late 2024 and became a core extension in early 2025. The read_avro Function The extension adds a single DuckDB function
...ossible: An Avro union of any type and the special null type is simplified to just the non-null type. For example, an Avro record of the union type ["int","null"] becomes a DuckDB INTEGER , which just happens to be NULL sometimes. Similarly, an Avro un...ion that contains only a single type is converted to the type it contains. For example, an Avro record of the union type ["int"] also becomes a DuckDB INTEGER . The extension also "flattens" the Avro schema. Avro defines tabl...es as root-level "record" fields, which are the same as DuckDB STRUCT fields. For more convenient handling, this extension turns the entries of a s... - https://man.mtool.work/duckdb/2.0/core_extensions/avro.html
- Handle Results 9391
- Overview Beyond reading a result set row by row , the Rust client can hand a query result to Apache Arrow as a stream of record batches, register an Arrow batch as a queryable table, and return results as Polars data frames. Each of these c
...w types through duckdb::arrow . Reading a Result as Arrow Call query_arrow() on a prepared statement to get an iterator of RecordBatch . Collecting it materializes the whole result, while iterating it reads one batch at a time: use duckdb::{Connection,...Result}; use duckdb::arrow::record_batch::RecordBatch; use duckdb::arrow::util::pretty::print_batches; let conn = Connection::open_in_memory()?; let mut stmt...= conn.prepare("SELECT * FROM generate_series(1, 5)")?; let batches: Vec<RecordBatch> = stmt.query_arrow([])?.collect(); print_batches(&batches).unwrap(); get_schema() on the returned handle reports the... - https://man.mtool.work/duckdb/2.0/clients/rust/result_handling.html
- Export to Apache Arrow 8807
- All results of a query can be exported to an Apache Arrow Table using the to_arrow_table function. Alternatively, results can be returned as a RecordBatchReader using the to_arrow_reader function and results can be read one batch at a time.
...lose it first, or run the other queries on a cursor from con.cursor() . Deprecated On relations, fetch_arrow_table , fetch_record_batch and fetch_arrow_reader are deprecated aliases of to_arrow_table and to_arrow_reader . Results of con.execute() and t...rrow_table" and return as an Arrow Table results = duckdb.sql("SELECT * FROM my_arrow_table").to_arrow_table() Export as a RecordBatchReader import duckdb con = duckdb.connect() rel = con.sql("SELECT i FROM range(5) AS t(i)") with rel.to_arrow_reader(b...the Relational API. A relation can be converted to an Arrow table using DuckDBPyRelation.to_arrow_table , and to an Arrow record batch reader using DuckDBPyRelation.to_arrow_reader . import duckdb # connect to an in-memory database con = duckdb.connec... - https://man.mtool.work/duckdb/2.0/guides/python/export_arrow.html
- Conversion between DuckDB and Python 8223
- This page documents the rules for converting DuckDB results to Python and Python objects to DuckDB . Result Conversion: DuckDB Results to Python A result is read from a relation or, when using the DB-API / con.execute() , from the connectio
...fault) rows. Apache Arrow to_arrow_table() fetches the data as an Arrow table to_arrow_reader(batch_size) returns an Arrow record batch reader with at most batch_size rows per batch (the default is 131072) arrow() is an older name of to_arrow_reader()...n the same way. For Arrow output, run the query with con.sql(query) . Deprecated on relations, fetch_arrow_table() , fetch_record_batch() and fetch_arrow_reader() are deprecated aliases of to_arrow_table() and to_arrow_reader() . After con.execute() ,...fetch_arrow_table() , fetch_record_batch() , arrow() and pl() no longer exist, and neither do the module functions duckdb.fetch_arrow_table() , duckdb.fetch_... - https://man.mtool.work/duckdb/2.0/clients/python/conversion.html
- Iceberg Functions 8223
- The iceberg extension provides the following functions: Function Description iceberg_scan(source) Reads an Iceberg table from a path. iceberg_metadata(source) Returns the manifest entries of an Iceberg table. iceberg_snapshots(source) Retur
...', allow_moved_paths = true); manifest_path manifest_sequence_number manifest_content status content file_path file_format record_count lineitem_iceberg/metadata/10eaca8a-1e1c-421e-ad6d-b232e5ee23d3-m1.avro 2 DATA ADDED EXISTING lineitem_iceberg/data/0...llow arbitrary key-value properties to be attached at the schema (namespace) level. These properties are typically used to record ownership, descriptions, default storage locations, or any other metadata that applies to every table in a schema. Functio..._from_timestamp Returns one row per data file and delete file of the snapshot, including the file path, format, partition, record count, file size and column-level statistics ( column_sizes , value_counts , null_value_counts , lower_bounds , upper_boun... - https://man.mtool.work/duckdb/2.0/core_extensions/iceberg/iceberg_functions.html
- Import Data 8073
- Overview The Rust client offers two ways to get data into DuckDB in bulk: the high-performance Appender for inserting rows from Rust, and reading directly from data files such as Parquet, CSV, and JSON. Both are described below. Appender Th
...on open), and Panic . Appending Arrow Data With the appender-arrow feature enabled, an Appender can append an Apache Arrow RecordBatch directly, which avoids building a parameter list per row. See Handle Results for the Arrow integration. Reading Data...and query read_parquet() , adapted from the crate's parquet example : use duckdb::{Connection, Result}; use duckdb::arrow::record_batch::RecordBatch; let conn = Connection::open_in_memory()?; conn.execute_batch("INSTALL parquet; LOAD parquet;")?; let b...atches: Vec<RecordBatch> = conn .prepare("SELECT * FROM read_parquet(?)")? .query_arrow(["data.parquet"])? .collect(); The same pattern reads... - https://man.mtool.work/duckdb/2.0/clients/rust/data_import.html
- Data Ingestion 7773
- This page contains examples for data ingestion to Python using DuckDB. First, import the DuckDB package: import duckdb Then, proceed with any of the following sections. CSV Files CSV files can be read using the read_csv function, called eit
...ckDB tables and views Replacement scans DuckDB supports querying multiple types of Apache Arrow objects including tables , record batches, arrays and chunked arrays, datasets , RecordBatchReaders and scanners . See the Python guides for more examples....The filters of a query are pushed into Arrow datasets and Polars LazyFrames. Tables, record batches, readers, scanners and Polars DataFrames are read whole, and the filter applies to what they return. A RecordBatch...Reader can be read once. A second scan raises InvalidInputException: This RecordBatchReader has already been consumed and cannot be scanned again. To scan its data more than once, read it into a table fi... - https://man.mtool.work/duckdb/2.0/clients/python/data_ingestion.html
DuckDB 2.0 Documentation / man.mtool.work