Results of 1 - 10 of about 28 for Apache (0.002 sec.)
- SQL on Apache Arrow 12896
- DuckDB can query multiple different types of Apache Arrow objects. DuckDB pushes the filters of a query that it can translate, such as comparisons, into the scan of an Arrow dataset. Arrow tables, record batches, scanners and RecordBatchRea
...ds, so an aggregate that depends on row order, such as first(i) without ORDER BY , can return a different row on each run. Apache Arrow Tables Arrow Tables stored in local variables can be queried as if they are regular tables within DuckDB. import duc...kdb.connect() my_arrow_table = pa.Table.from_pydict({'i': [1, 2, 3, 4], 'j': ["one", "two", "three", "four"]}) # query the Apache Arrow Table "my_arrow_table" and return as an Arrow Table results = con.sql("SELECT * FROM my_arrow_table WHERE i = 2").to..._arrow_table() Apache Arrow Datasets Arrow Datasets stored as variables can also be queried as if they were regular tables. Datasets are useful... - https://man.mtool.work/duckdb/2.0/guides/python/sql_on_arrow.html
- Handle Results 11912
- Overview Beyond the standard JDBC ResultSet , the DuckDB driver can hand results to Apache Arrow , stream large results instead of materializing them, and expose raw columnar data chunks. This page covers each of these result-handling optio
...ns. Arrow Methods The DuckDB result set integrates with Apache Arrow through the Java Arrow bindings . A DuckDBResultSet can export its rows as an Arrow stream for consumption by Arrow-...ing the Java Arrow bindings. Each batch is loaded in turn, and its vectors are read from the VectorSchemaRoot . import org.apache.arrow.memory.RootAllocator; import org.apache.arrow.vector.ipc.ArrowReader; import org.duckdb.DuckDBResultSet; try (var co...registered, the stream is addressed by the name passed to that method and can be queried like any other table. import org.apache.arrow.c.ArrowArrayStream; import org.apache.arrow.c.Data; import org.apache.arrow.memory.RootAllocator; import org.apache.... - https://man.mtool.work/duckdb/2.0/clients/java/result_handling.html
- Import Data 11339
- Overview DuckDB-Wasm has multiple ways to import data, depending on the format of the data. Importing happens in two steps. The data file is registered in a virtual file system using one of the register functions ( registerFileBuffer , regi
...nnection const conn = await db.connect(); // ... import data // Close the connection to release memory await conn.close(); Apache Arrow Apache Arrow is DuckDB-Wasm's native data protocol, so Arrow data can be inserted directly. Pass an existing arrow.T...able to insertArrowTable() : Warning Use the same major version of the apache-arrow package that DuckDB-Wasm depends on (currently ^17 ). A newer major version causes Arrow inserts such as insertArrow...Table() and insertArrowFromIPCStream() to fail — often silently, leaving the target table uncreated. Check DuckDB-Wasm's apache-arrow dependency and pin your apache-arrow install to match. import { tableFromArrays } from 'apache-arrow'; const arrowTa... - https://man.mtool.work/duckdb/2.0/clients/wasm/data_ingestion.html
- Guides 10847
- The guides section contains compact how-to guides that are focused on achieving a single goal. For API references and examples, see the rest of the documentation. Note that there are many tools using DuckDB, which are not covered in the off
...How to export data to a Pandas DataFrame NumPy How to create a table from NumPy arrays How to export data to NumPy arrays Apache Arrow How to execute SQL on Apache Arrow How to create a DuckDB table from Apache Arrow How to export data to Apache Arrow...thon Library Integrations How to use Ibis to query DuckDB with or without SQL How to use DuckDB with Polars DataFrames via Apache Arrow How to stream DuckDB query results to PyTorch via Apache Arrow SQL Features Friendly SQL As-of join Full-text search... - https://man.mtool.work/duckdb/2.0/guides/overview.html
- Import from Apache Arrow 9094
- CREATE TABLE AS and INSERT INTO can be used to create a table from any query. We can then create tables or insert into existing tables by referring to the Apache Arrow object in the query. This example imports from an Arrow Table , but Duck
...DB can query different Apache Arrow formats as seen in the SQL on Arrow guide . import duckdb import pyarrow as pa # connect to an in-memory database my... - https://man.mtool.work/duckdb/2.0/guides/python/import_arrow.html
- Integration with PyTorch 9094
- PyTorch tensors can be built from DuckDB query results by exporting each result batch as Apache Arrow . This pattern is useful when training or serving models on Parquet data because DuckDB can filter and project the input before the batche
...tion returns a dictionary with one tensor per column. To learn more about querying Arrow objects directly, see the “SQL on Apache Arrow” guide and the “Export to Apache Arrow” guide .... - https://man.mtool.work/duckdb/2.0/guides/python/pytorch.html
- Handle Results 8504
- Overview Beyond reading a result set row by row , the Rust client can hand a query result to Apache Arrow as a stream of record batches, register an Arrow batch as a queryable table, and return results as Polars data frames. Each of these c
...olumnar result-handling options is described below. Apache Arrow DuckDB is columnar, and its native way to return a result to Rust in bulk is Apache Arrow . The crate re-exports the... - https://man.mtool.work/duckdb/2.0/clients/rust/result_handling.html
- ADBC Client 8406
- Installation To use the DuckDB ADBC client, download the libduckdb archive for your platform and follow the instructions below . For Python, install the duckdb package instead, as described in the Python section . The latest stable version
...se to modify in-memory Arrow RecordBatches via SQL queries: package main import ( "bytes" "context" "fmt" "io" "github.com/apache/arrow-adbc/go/adbc" "github.com/apache/arrow-adbc/go/adbc/drivermgr" "github.com/apache/arrow-go/v18/arrow" "github.com/ap...ache/arrow-go/v18/arrow/array" "github.com/apache/arrow-go/v18/arrow/ipc" "github.com/apache/arrow-go/v18/arrow/memory" ) func _makeSampleArrowRecord() arrow.Record { b :=... - https://man.mtool.work/duckdb/2.0/clients/adbc.html
- DuckDB Wasm Client 8307
- Installation To use the DuckDB Wasm client, visit the duckdb-wasm GitHub repository . The latest stable version of the DuckDB WebAssembly client is 2.0.0. DuckDB-Wasm is a version of DuckDB that has been compiled to WebAssembly , the portab
...lease resources await conn.close(); await db.terminate(); await worker.terminate(); How DuckDB-Wasm Works DuckDB-Wasm uses Apache Arrow as its data protocol for both data import and query results. Arrow is a columnar, database-friendly format that the...selection patterns for jsDelivr, webpack, Vite, and statically served files. Import Data — registering files and inserting Apache Arrow, CSV, JSON, and Parquet data. Run Queries — materialized and streaming queries, prepared statements, and exporting r... - https://man.mtool.work/duckdb/2.0/clients/wasm/overview.html
- Integration with Polars 8111
- Polars is a DataFrames library built in Rust with bindings for Python and Node.js. It uses Apache Arrow's columnar format as its memory model. DuckDB can read Polars DataFrames and convert query results to Polars DataFrames. It does this in
...ternally using the efficient Apache Arrow integration. Note that the pyarrow library must be installed for the integration to work. Installation pip install -... - https://man.mtool.work/duckdb/2.0/guides/python/polars.html
DuckDB 2.0 Documentation / man.mtool.work