Results of 1 - 10 of about 50 for arrow (0.003 sec.)
- SQL on Apache Arrow 11354
- DuckDB can query multiple different types of Apache Arrow objects. DuckDB pushes the filters of a query that it can translate, such as comparisons, into the scan of an Arrow dataset. Arrow tables, record batches, scanners and RecordBatchRea
...r can be scanned only once; to query its data more than once, read it into a table with read_all() before the first query. Arrow tables are scanned on all threads, so an aggregate that depends on row order, such as first(i) without ORDER BY , can retu...rn a different row on each run. Apache Arrow Tables Arrow Tables stored in local variables can be queried as if they are regular tables within DuckDB. import duckdb im...port pyarrow as pa # connect to an in-memory database con = duckdb.connect() my_arrow_table = pa.Table.from_pydict({'i': [1, 2, 3, 4],... - https://man.mtool.work/duckdb/2.0/guides/python/sql_on_arrow.html
- Handle Results 10918
- Overview Beyond reading a result set row by row , the Rust client can hand a query result to Apache Arrow as a stream of record batches, register an Arrow batch as a queryable table, and return results as Polars data frames. Each of these c
...olumnar result-handling options is described below. Apache Arrow DuckDB is columnar, and its native way to return a result to Rust in bulk is Apache Arrow . The crate re-exports the arrow...crate, so no separate Arrow dependency or version alignment is required: reach the Arrow types through duckdb::arrow . Reading a Result as Arrow Call...query_arrow() on a prepared statement to get an iterator of RecordBatch . Collecting it materializes the whole result, while iterating... - https://man.mtool.work/duckdb/2.0/clients/rust/result_handling.html
- Export to Apache Arrow 10798
- All results of a query can be exported to an Apache Arrow Table using the to_arrow_table function. Alternatively, results can be returned as a RecordBatchReader using the to_arrow_reader function and results can be read one batch at a time.
...ader to the end or close it first, or run the other queries on a cursor from con.cursor() . Deprecated On relations, fetch_arrow_table , fetch_record_batch and fetch_arrow_reader are deprecated aliases of to_arrow_table and to_arrow_reader . Results o...f con.execute() and the duckdb module no longer have them. Export to an Arrow Table import duckdb import pyarrow as pa my_arrow_table = pa.Table.from_pydict({'i': [1, 2, 3, 4], 'j': ["one", "two", "th...ree", "four"]}) # query the Apache Arrow Table "my_arrow_table" and return as an Arrow Table results = duckdb.sql("SELECT * FROM my_arrow_table").to_arrow_table()... - https://man.mtool.work/duckdb/2.0/guides/python/export_arrow.html
- ADBC Client 10076
- Installation To use the DuckDB ADBC client, download the libduckdb archive for your platform and follow the instructions below . For Python, install the duckdb package instead, as described in the Python section . The latest stable version
...of the DuckDB ADBC client is 2.0.0. Arrow Database Connectivity (ADBC) , similarly to ODBC and JDBC, is a C-style API that enables code portability between differen...abase systems without using code specific to that system. The main difference between ADBC and ODBC/JDBC is that ADBC uses Arrow to transfer data between the database system and the application. DuckDB has an ADBC driver, which takes advantage of the...zero-copy integration between DuckDB and Arrow to efficiently transfer data. Please refer to the ADBC documentation page for a more extensive discussion on ADBC and a de... - https://man.mtool.work/duckdb/2.0/clients/adbc.html
- Handle Results 10046
- Overview DuckDB is columnar, and beyond reading a result set row by row through database/sql , the Go client can exchange whole results with Apache Arrow as a stream of record batches, and register an Arrow stream as a queryable view. Both
...are described below. Enabling the Arrow Interface The DuckDB Arrow interface is a heavy dependency, so starting with v2 it is opt-in. Enable it with the duckdb_ar...row build tag : go build -tags=duckdb_arrow Without the tag, the Arrow types and functions described here are not compiled in. The interface builds on apache/arrow-go..., so its array and arrow packages provide the record and schema types. Reading a Result as Arrow Wrap a raw DuckDB connection with duckdb.NewArrowF... - https://man.mtool.work/duckdb/2.0/clients/go/result_handling.html
- Handle Results 9986
- Overview Beyond the standard JDBC ResultSet , the DuckDB driver can hand results to Apache Arrow , stream large results instead of materializing them, and expose raw columnar data chunks. This page covers each of these result-handling optio
...ns. Arrow Methods The DuckDB result set integrates with Apache Arrow through the Java Arrow bindings . A DuckDBResultSet can export...its rows as an Arrow stream for consumption by Arrow-based tools, and an Arrow stream produced elsewhere can be registered on a connection and...queried as a DuckDB table. Refer to the API Reference for the type signatures. Arrow Export The following example exports an Arrow stream from a query result and consumes it using the Java Arrow bindings. Ea... - https://man.mtool.work/duckdb/2.0/clients/java/result_handling.html
- Conversion between DuckDB and Python 9926
- This page documents the rules for converting DuckDB results to Python and Python objects to DuckDB . Result Conversion: DuckDB Results to Python A result is read from a relation or, when using the DB-API / con.execute() , from the connectio
...ny(size=1) , fetchall() Python dicts to_dicts() NumPy fetchnumpy() fetchnumpy() Pandas df() df() , fetch_df_chunk() Apache Arrow to_arrow_table() , to_arrow_reader() to_arrow_table() , to_arrow_reader() , both make a copy Polars pl() PyTorch torch() t...ult itself. To get specific rows of a result with the relational API you would use limit(n).fetchall() , iter_rows() or to_arrow_reader() instead. Note that with the DB-API, after a partial read with fetchone() or fetchmany() , the df() , fetchnumpy()...and Arrow methods of the connection return only the remaining rows. Examples Below are some examples. See the Python guides for more... - https://man.mtool.work/duckdb/2.0/clients/python/conversion.html
- Import from Apache Arrow 9715
- CREATE TABLE AS and INSERT INTO can be used to create a table from any query. We can then create tables or insert into existing tables by referring to the Apache Arrow object in the query. This example imports from an Arrow Table , but Duck
...DB can query different Apache Arrow formats as seen in the SQL on Arrow guide . import duckdb import pyarrow as pa # connect to an in-memory database my_arrow...= pa.Table.from_pydict({'a': [42]}) # create the table "my_table" from the DataFrame "my_arrow" duckdb.sql("CREATE TABLE my_table AS SELECT * FROM my_arrow") # insert into the table "my_table" from the DataFrame "my_a...rrow" duckdb.sql("INSERT INTO my_table SELECT * FROM my_arrow")... - https://man.mtool.work/duckdb/2.0/guides/python/import_arrow.html
- Complete API 9204
- This page contains the reference for DuckDB's C API. The reference contains several deprecation notices. These concern methods whose long-term availability is not guaranteed as they may be removed in the future. That said, DuckDB's develope
...tion_get_client_context ( duckdb_connection connection , duckdb_client_context * out_context ); void duckdb_connection_get_arrow_options ( duckdb_connection connection , duckdb_arrow_options * out_arrow_options ); idx_t duckdb_client_context_get_conne...ckdb_client_context context ); void duckdb_destroy_client_context ( duckdb_client_context * context ); void duckdb_destroy_arrow_options ( duckdb_arrow_options * arrow_options ); const char * duckdb_library_version (); duckdb_value duckdb_get_table_na...pe ( duckdb_result result ); duckdb_logical_type duckdb_column_logical_type ( duckdb_result * result , idx_t col ); duckdb_arrow_options duckdb_result_get_arrow_options ( duckdb_result * result ); idx_t duckdb_column_count ( duckdb_result * result );... - https://man.mtool.work/duckdb/2.0/clients/c/api.html
- Startup & Shutdown 8903
- To use DuckDB, you must first initialize a duckdb_database handle using duckdb_open() . duckdb_open() takes as parameter the database file to read and write from. The special value NULL ( nullptr ) can be used to create an in-memory databas
...tion_get_client_context ( duckdb_connection connection , duckdb_client_context * out_context ); void duckdb_connection_get_arrow_options ( duckdb_connection connection , duckdb_arrow_options * out_arrow_options ); idx_t duckdb_client_context_get_conne...ckdb_client_context context ); void duckdb_destroy_client_context ( duckdb_client_context * context ); void duckdb_destroy_arrow_options ( duckdb_arrow_options * arrow_options ); const char * duckdb_library_version (); duckdb_value duckdb_get_table_na...ntext : The client context of the connection. Must be destroyed with duckdb_destroy_client_context . duckdb_connection_get_arrow_options Retrieves the arrow options of the connection. Syntax void duckdb_connection_get_arrow_options ( duckdb_connection... - https://man.mtool.work/duckdb/2.0/clients/c/connect.html
DuckDB 2.0 Documentation / man.mtool.work