Results of 1 - 10 of about 14 for polars (0.002 sec.)
- Integration with Polars 13741
- Polars is a DataFrames library built in Rust with bindings for Python and Node.js. It uses Apache Arrow's columnar format as its memory model. DuckDB can read Polars DataFrames and convert query results to Polars DataFrames. It does this in
...egration. Note that the pyarrow library must be installed for the integration to work. Installation pip install -U duckdb 'polars[pyarrow]' Polars to DuckDB DuckDB can natively query Polars DataFrames and LazyFrames by referring to their names as they...the current scope. DuckDB pushes the column selections of a query, and the filters it can translate, into a LazyFrame, so Polars collects only the rows and columns the query needs. A DataFrame is converted in full and filtered afterwards. import duckd...b import polars as pl df = pl.DataFrame( { "A": [1, 2, 3, 4, 5], "fruits": ["banana", "banana", "apple", "apple", "banana"], "B": [5, 4, 3... - https://man.mtool.work/duckdb/2.0/guides/python/polars.html
- Handle Results 11588
- Overview Beyond reading a result set row by row , the Rust client can hand a query result to Apache Arrow as a stream of record batches, register an Arrow batch as a queryable table, and return results as Polars data frames. Each of these c
...other table. arrow_recordbatch_to_query_params() expands a batch into the two parameters the arrow(?, ?) function expects. Polars Data Frames With the polars feature enabled, a query result can be returned as Polars DataFrame s. Call query_polars() on...a prepared statement to get an iterator of data frames, one per result chunk: use duckdb::{Connection, Result}; use polars::prelude::DataFrame; let conn = Connection::open_in_memory()?; let mut stmt = conn.prepare("SELECT * FROM test")?; let dfs...: Vec<DataFrame> = stmt.query_polars([])?.collect(); To combine the chunks into a single DataFrame , use accumulate_dataframes_vertical_unchecked from polars_c... - https://man.mtool.work/duckdb/2.0/clients/rust/result_handling.html
- Python API 9242
- Installation To use the DuckDB Python client, visit the Python installation page . The latest stable version of the DuckDB Python client is 2.0.0. Installation The DuckDB Python API can be installed using pip : pip install duckdb . Please s
...b.sql("SELECT * FROM 'example.json'") # directly query a JSON file DataFrames DuckDB can directly query Pandas DataFrames, Polars DataFrames and Arrow tables. Note that these are read-only, i.e., editing these tables via INSERT or UPDATE statements is..._df = pd.DataFrame({"a": [42]}) duckdb.sql("SELECT * FROM pandas_df") ┌───────┐ │ a │ │ int64 │ ├───────┤ │ 42 │ └───────┘ Polars To directly query a Polars DataFrame, run: import duckdb import polars as pl polars_df = pl.DataFrame({"a": [42]}) duckdb....sql("SELECT * FROM polars_df") ┌───────┐ │ a │ │ int64 │ ├───────┤ │ 42 │ └───────┘ PyArrow To directly query a PyArrow table, run: import duckdb im... - https://man.mtool.work/duckdb/2.0/clients/python/overview.html
- Rust Client 8910
- Installation To use the DuckDB Rust client, visit the Rust installation page . The latest stable version of the DuckDB Rust client is 2.0.0. The DuckDB Rust client, duckdb-rs , is an ergonomic wrapper over the DuckDB C API that exposes an i
...dium branch. Tip The Rust client offers a large set of Cargo feature flags that turn on file-format readers, the Arrow and Polars integrations, connection pooling, and the function and extension interfaces. Enable only the features a project uses to ke...kDB extension rather than a client application. See Building a Loadable Extension . Ecosystem Integrations Feature Enables polars Exchange query results as Polars data frames. See Handle Results . r2d2 A connection pool through the r2d2 crate. See Conn...precated vtab-excel . extensions-full json , parquet and vtab-full . modern-full chrono , serde_json , url , r2d2 , uuid , polars and rust_decimal . Deprecated Features Feature Enables vtab-excel A no-op retained for feature compatibility. vtab-loadabl... - https://man.mtool.work/duckdb/2.0/clients/rust/overview.html
- Conversion between DuckDB and Python 8594
- This page documents the rules for converting DuckDB results to Python and Python objects to DuckDB . Result Conversion: DuckDB Results to Python A result is read from a relation or, when using the DB-API / con.execute() , from the connectio
...fetch_df_chunk() Apache Arrow to_arrow_table() , to_arrow_reader() to_arrow_table() , to_arrow_reader() , both make a copy Polars pl() PyTorch torch() torch() TensorFlow tf() tf() Relations are recommended for most conversion, except when you want to d...y A relation is also an Arrow stream through the Arrow PyCapsule Interface . pyarrow.table(rel) returns an Arrow table and polars.DataFrame(rel) a Polars DataFrame, which unlike pl() needs no pyarrow After con.execute() , to_arrow_table() and to_arrow_...han as its original duration type. This is expected. The resolution and unit of the original Arrow type are not preserved. Polars pl() fetches the data of a relation as a Polars DataFrame, and pl(lazy=True) returns a Polars LazyFrame Object Conversion:... - https://man.mtool.work/duckdb/2.0/clients/python/conversion.html
- Export to Apache Arrow 8542
- All results of a query can be exported to an Apache Arrow Table using the to_arrow_table function. Alternatively, results can be returned as a RecordBatchReader using the to_arrow_reader function and results can be read one batch at a time.
...ibraries that accept an Arrow stream read it directly: pyarrow.table(rel) , pyarrow.RecordBatchReader.from_stream(rel) and polars.DataFrame(rel) . Polars reads it this way even without pyarrow installed. import duckdb import polars as pl import pyarrow... - https://man.mtool.work/duckdb/2.0/guides/python/export_arrow.html
- Data Ingestion 8437
- This page contains examples for data ingestion to Python using DuckDB. First, import the DuckDB package: import duckdb Then, proceed with any of the following sections. CSV Files CSV files can be read using the read_csv function, called eit
..."SELECT * FROM test_df").fetchall()) [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')] The same works for Pandas Series, Polars DataFrames, LazyFrames and Series, NumPy arrays, a dict of one-dimensional NumPy arrays, relations and Arrow objects. The...hReaders and scanners . See the Python guides for more examples. The filters of a query are pushed into Arrow datasets and Polars LazyFrames. Tables, record batches, readers, scanners and Polars DataFrames are read whole, and the filter applies to what...scanned again. To scan its data more than once, read it into a table first, for example with read_all() . Arrow tables and Polars DataFrames are scanned on all threads, even when they hold a single chunk. A query that depends on the order of rows witho... - https://man.mtool.work/duckdb/2.0/clients/python/data_ingestion.html
- marimo Notebooks 8052
- marimo is an open-source reactive notebook for Python and SQL that's tightly integrated with DuckDB's Python client, letting you mix and match Python and SQL in a single git-versionable notebook. Unlike traditional notebooks, when you run a
...notebooks as scripts. Installation To get started, install marimo from your terminal. Its sql extra also installs DuckDB, Polars and PyArrow: pip install "marimo[sql]" # or uv add "marimo[sql]" Run a tutorial: marimo tutorial sql SQL in marimo Create...ataframe. If you have a database connection with a table of the same name, the database table will be used instead. import polars as pl df = pl.DataFrame({"value": [1, 2, 3]}) SELECT * FROM df WHERE value > 2 Reference the Output of a SQL Cell Defining... - https://man.mtool.work/duckdb/2.0/guides/python/marimo.html
- Migrating Python Code to DuckDB 2.0 7894
- This page lists what Python code written for DuckDB 1.x must change to run on DuckDB 2.0. Each item gives the 1.x form, the 2.0 form and the page that explains it. For changes outside the Python client, see Migrating to DuckDB v2.0 . Python
...ilds of CPython are not supported. On Python 3.10, pip install duckdb installs a 1.x release. See Installation . Arrow and Polars output of execute() . 1.x: con.execute(query).fetch_arrow_table() , fetch_record_batch() , arrow() and pl() . 2.0: these m...Use a relation: con.sql(query).to_arrow_table() , con.sql(query).to_arrow_reader() or con.sql(query).pl() . See Arrow and Polars Output . Fetching from a relation. 1.x: rel.fetchone() , rel.fetchmany(n) , rel.execute() and rel.fetch_df_chunk() . 2.0:... - https://man.mtool.work/duckdb/2.0/clients/python/migration.html
- Python DB API 7737
- The standard DuckDB Python API provides a SQL interface compliant with the DB-API 2.0 specification described by PEP 249 similar to the SQLite Python API . Connection To use the module, you must first create a DuckDBPyConnection object that
...is connection, or use a separate connection from cursor() con.complete() con.sql("SELECT 42").fetchall() [(42,)] Arrow and Polars Output A result of execute() has no fetch_arrow_table() , fetch_record_batch() , arrow() or pl() method. Its to_arrow_tabl...and emit a UserWarning , and that reader does not stream: its first batch arrives after the whole query ran. For Arrow and Polars output, use a relation instead: con.sql(query).to_arrow_table() , con.sql(query).pl() or con.sql(query).to_arrow_reader()... - https://man.mtool.work/duckdb/2.0/clients/python/dbapi.html
DuckDB 2.0 Documentation / man.mtool.work