Results of 1 - 10 of about 14 for DataFrames (0.001 sec.)
- Julia Client 12901
- The DuckDB Julia package provides a high-performance front-end for DuckDB. Much like SQLite, DuckDB runs in-process within the Julia client, and provides a DBInterface front-end. The package also supports multi-threaded execution. It uses J
...ead they can be run with DuckDB.query() instead of DuckDB.execute() and will always return a materialized result. Scanning DataFrames The DuckDB Julia package also provides support for querying Julia DataFrames. Note that the DataFrames are directly read b...load data from a DataFrame into a DuckDB table you can run a CREATE TABLE ... AS or INSERT INTO query. using DuckDB using DataFrames # create a new in-memory database con = DBInterface.connect(DuckDB.DB) # create a DataFrame df = DataFrame(a = [1, 2, 3],...rows have been appended, close() should be used to finalize the Appender and clean up the resulting memory. using DuckDB, DataFrames, Dates db = DuckDB.DB() # create a table DBInterface.execute(db, "CREATE OR REPLACE TABLE data (id INTEGER PRIMARY KEY, va... - https://man.mtool.work/duckdb/2.0/clients/tertiary_clients/julia.html
- Integration with Polars 12130
- Polars is a DataFrames library built in Rust with bindings for Python and Node.js. It uses Apache Arrow's columnar format as its memory model. DuckDB can read Polars DataFrames and convert query results to Polars DataFrames. It does this in
...ntegration to work. Installation pip install -U duckdb 'polars[pyarrow]' Polars to DuckDB DuckDB can natively query Polars DataFrames and LazyFrames by referring to their names as they exist in the current scope. DuckDB pushes the column selections of a qu...etle", "beetle", "beetle"], } ) duckdb.sql("SELECT * FROM df").show() DuckDB to Polars DuckDB can output results as Polars DataFrames using the .pl() method of a relation, the object that duckdb.sql and con.sql return. Results of con.execute() have no .pl(... - https://man.mtool.work/duckdb/2.0/guides/python/polars.html
- SQL on Pandas 9872
- Pandas DataFrames stored in local variables can be queried as if they are regular tables within DuckDB. import duckdb import pandas # Create a Pandas dataframe my_df = pandas.DataFrame.from_dict({'a': [42]}) # query the Pandas DataFrame "my
...default in-memory database connection results = duckdb.sql("SELECT * FROM my_df").df() The seamless integration of Pandas DataFrames into DuckDB SQL queries is allowed by replacement scans , which replace instances of accessing the my_df table (which does... - https://man.mtool.work/duckdb/2.0/guides/python/sql_on_pandas.html
- Data Ingestion 9662
- This page contains examples for data ingestion to Python using DuckDB. First, import the DuckDB package: import duckdb Then, proceed with any of the following sections. CSV Files CSV files can be read using the read_csv function, called eit
...'example.json'") Call read_json from within SQL: duckdb.sql("SELECT * FROM read_json('example.json')") Directly Accessing DataFrames and Arrow Objects DuckDB can query a Pandas DataFrame by the name of its variable, as if it were a table: import duckdb im...* FROM test_df").fetchall()) [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')] The same works for Pandas Series, Polars DataFrames, LazyFrames and Series, NumPy arrays, a dict of one-dimensional NumPy arrays, relations and Arrow objects. The masked elem...ters of a query are pushed into Arrow datasets and Polars LazyFrames. Tables, record batches, readers, scanners and Polars DataFrames are read whole, and the filter applies to what they return. A RecordBatchReader can be read once. A second scan raises Inv... - https://man.mtool.work/duckdb/2.0/clients/python/data_ingestion.html
- Jupyter Notebooks 8542
- DuckDB's Python client can be used directly in Jupyter notebooks with no additional configuration, as the first section shows. Additional libraries can simplify SQL query development; the rest of this guide uses the JupySQL package. See oth
...das dataframe, otherwise, it is a ResultSet that can be converted to Pandas with the DataFrame() function. Querying Pandas Dataframes DuckDB is able to find and query any dataframe stored as a variable in the Jupyter notebook. input_df = pd.DataFrame.from_...s directly through the engine (avoiding both the download of the entire file and loading all of it into Pandas in memory). Dataframes can be read as tables in SQL, and SQL results can be output into Dataframes. Happy analyzing! An alternative to jupysql is... - https://man.mtool.work/duckdb/2.0/guides/python/jupyter.html
- Appender 8419
- Use an appender to insert rows that your Python code produces. It collects rows and writes them together in one SQL statement; this is called flushing. The with block below flushes on successful exit: import duckdb con = duckdb.connect() co
...or data already in a DataFrame or Arrow table, INSERT INTO people SELECT * FROM df is faster than the appender. For pandas DataFrames, con.append("people", df) is also faster. See Directly Accessing DataFrames and Arrow Objects . Use con.appender(table) fo...cepts, such as "INTEGER" , duckdb.sqltypes.INTEGER or int . A declared column that the statement does not read is allowed. DataFrames and other Python objects named in source or in the statement are looked up once, where the appender is created, and reused... - https://man.mtool.work/duckdb/2.0/clients/python/appender.html
- Python API 8192
- Installation To use the DuckDB Python client, visit the Python installation page . The latest stable version of the DuckDB Python client is 2.0.0. Installation The DuckDB Python API can be installed using pip : pip install duckdb . Please s
...example.parquet'") # directly query a Parquet file duckdb.sql("SELECT * FROM 'example.json'") # directly query a JSON file DataFrames DuckDB can directly query Pandas DataFrames, Polars DataFrames and Arrow tables. Note that these are read-only, i.e., edit... - https://man.mtool.work/duckdb/2.0/clients/python/overview.html
- Guides 8192
- The guides section contains compact how-to guides that are focused on achieving a single goal. For API references and examples, see the rest of the documentation. Note that there are many tools using DuckDB, which are not covered in the off
...Arrow How to create a DuckDB table from Apache Arrow How to export data to Apache Arrow Relational API How to query Pandas DataFrames with the Relational API Python Library Integrations How to use Ibis to query DuckDB with or without SQL How to use DuckDB...with Polars DataFrames via Apache Arrow How to stream DuckDB query results to PyTorch via Apache Arrow SQL Features Friendly SQL As-of join Full-... - https://man.mtool.work/duckdb/2.0/guides/overview.html
- Glossary of Terms 7754
- This page contains a glossary of a few common terms used in DuckDB. Terms In-Process Database Management System The DBMS runs in the client application's process instead of running as a separate process, which is common in the traditional c
...ubstitute another data source instead of the table. Using replacement scans allows DuckDB to, e.g., seamlessly read Pandas DataFrames or read input data from remote sources without explicitly invoking the functions that perform this (e.g., reading Parquet... - https://man.mtool.work/duckdb/2.0/guides/glossary.html
- Import from Pandas 7754
- CREATE TABLE ... AS and INSERT INTO can be used to create a table from any query. We can then create tables or insert into existing tables by referring to the Pandas DataFrame in the query. There is no need to register the DataFrames manual
... - https://man.mtool.work/duckdb/2.0/guides/python/import_pandas.html
DuckDB 2.0 Documentation / man.mtool.work