Results of 1 - 10 of about 25 for DataFrame (0.001 sec.)
- Import from Pandas 13101
- CREATE TABLE ... AS and INSERT INTO can be used to create a table from any query. We can then create tables or insert into existing tables by referring to the Pandas DataFrame in the query. There is no need to register the DataFrames manual
...DB can find them in the Python process by name thanks to replacement scans . import duckdb import pandas # Create a Pandas dataframe my_df = pandas.DataFrame.from_dict({'a': [42]}) # create the table "my_table" from the DataFrame "my_df" # Note: duckdb.sq...database connection duckdb.sql("CREATE TABLE my_table AS SELECT * FROM my_df") # insert into the table "my_table" from the DataFrame "my_df" duckdb.sql("INSERT INTO my_table SELECT * FROM my_df") If the order of columns is different or not all columns are...present in the DataFrame, use INSERT INTO ... BY NAME : duckdb.sql("INSERT INTO my_table BY NAME SELECT * FROM my_df") Alternatively, the append me... - https://man.mtool.work/duckdb/2.0/guides/python/import_pandas.html
- Jupyter Notebooks 11075
- DuckDB's Python client can be used directly in Jupyter notebooks with no additional configuration, as the first section shows. Additional libraries can simplify SQL query development; the rest of this guide uses the JupySQL package. See oth
...e DuckDB and Python together. Querying in a Python Cell In a plain Python cell, duckdb.sql runs a query and finds a pandas DataFrame by its variable name: import duckdb import pandas as pd sales = pd.DataFrame({"item": ["jeans", "hammer"], "price": [20, 4...OM sales WHERE price > 30") The last expression displays a table in the notebook. Add .df() to keep the result as a pandas DataFrame. Library Installation This JupySQL workflow is also available as a Google Colab notebook . Four additional libraries impro...g DuckDB Single line SQL queries can be run using %sql at the start of a line. Query results will be displayed as a Pandas DataFrame. %sql SELECT 'Off and flying!' AS a_duckdb_column; An entire Jupyter cell can be used as a SQL cell by placing %%sql at th... - https://man.mtool.work/duckdb/2.0/guides/python/jupyter.html
- SQL on Pandas 10992
- Pandas DataFrames stored in local variables can be queried as if they are regular tables within DuckDB. import duckdb import pandas # Create a Pandas dataframe my_df = pandas.DataFrame.from_dict({'a': [42]}) # query the Pandas DataFrame "my
...default in-memory database connection results = duckdb.sql("SELECT * FROM my_df").df() The seamless integration of Pandas DataFrames into DuckDB SQL queries is allowed by replacement scans , which replace instances of accessing the my_df table (which doe...s not exist in DuckDB) with a table function that reads the my_df dataframe.... - https://man.mtool.work/duckdb/2.0/guides/python/sql_on_pandas.html
- Relational API on Pandas 9581
- DuckDB offers a relational API that can be used to chain together query operations. These are lazily evaluated so that DuckDB can optimize their execution. These operators can act on Pandas DataFrames, DuckDB tables or views (which can poin
...e format that DuckDB can read, such as CSV or Parquet files, etc.). Here we show a simple example of reading from a Pandas DataFrame and returning a DataFrame. import duckdb import pandas # connect to an in-memory database con = duckdb.connect() input_df...= pandas.DataFrame.from_dict({'i': [1, 2, 3, 4], 'j': ["one", "two", "three", "four"]}) # create a DuckDB relation from a dataframe rel = con... - https://man.mtool.work/duckdb/2.0/guides/python/relational_api_pandas.html
- Julia Client 9315
- The DuckDB Julia package provides a high-performance front-end for DuckDB. Much like SQLite, DuckDB runs in-process within the Julia client, and provides a DBInterface front-end. The package also supports multi-threaded execution. It uses J
...ead they can be run with DuckDB.query() instead of DuckDB.execute() and will always return a materialized result. Scanning DataFrames The DuckDB Julia package also provides support for querying Julia DataFrames. Note that the DataFrames are directly read...by DuckDB – they are not inserted or copied into the database itself. If you wish to load data from a DataFrame into a DuckDB table you can run a CREATE TABLE ... AS or INSERT INTO query. using DuckDB using DataFrames # create a new i...n-memory database con = DBInterface.connect(DuckDB.DB) # create a DataFrame df = DataFrame(a = [1, 2, 3], b = [42, 84, 42]) # register it as a view in the database DuckDB.register_data_frame(con, df... - https://man.mtool.work/duckdb/2.0/clients/tertiary_clients/julia.html
- Data Ingestion 9049
- This page contains examples for data ingestion to Python using DuckDB. First, import the DuckDB package: import duckdb Then, proceed with any of the following sections. CSV Files CSV files can be read using the read_csv function, called eit
...'example.json'") Call read_json from within SQL: duckdb.sql("SELECT * FROM read_json('example.json')") Directly Accessing DataFrames and Arrow Objects DuckDB can query a Pandas DataFrame by the name of its variable, as if it were a table: import duckdb i...mport pandas as pd test_df = pd.DataFrame.from_dict({"i": [1, 2, 3, 4], "j": ["one", "two", "three", "four"]}) print(duckdb.sql("SELECT * FROM test_df").fetchall())...[(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')] The same works for Pandas Series, Polars DataFrames, LazyFrames and Series, NumPy arrays, a dict of one-dimensional NumPy arrays, relations and Arrow objects. The masked ele... - https://man.mtool.work/duckdb/2.0/clients/python/data_ingestion.html
- marimo Notebooks 8950
- marimo is an open-source reactive notebook for Python and SQL that's tightly integrated with DuckDB's Python client, letting you mix and match Python and SQL in a single git-versionable notebook. Unlike traditional notebooks, when you run a
...s you quickly navigate your schemas to pull tables and columns into your SQL queries. Data Sources Panel Reference a Local Dataframe Reference a local dataframe in your SQL cell by using the name of the Python variable that holds the dataframe. If you hav...a database connection with a table of the same name, the database table will be used instead. import polars as pl df = pl.DataFrame({"value": [1, 2, 3]}) SELECT * FROM df WHERE value > 2 Reference the Output of a SQL Cell Defining a non-private (non-unde...rscored) output variable in the SQL cell allows you to reference the resulting dataframe in other Python and SQL cells. Reference the SQL result in Python Reactive SQL Cells marimo allows you to create reactive... - https://man.mtool.work/duckdb/2.0/guides/python/marimo.html
- Integration with Polars 8950
- Polars is a DataFrames library built in Rust with bindings for Python and Node.js. It uses Apache Arrow's columnar format as its memory model. DuckDB can read Polars DataFrames and convert query results to Polars DataFrames. It does this in
...ntegration to work. Installation pip install -U duckdb 'polars[pyarrow]' Polars to DuckDB DuckDB can natively query Polars DataFrames and LazyFrames by referring to their names as they exist in the current scope. DuckDB pushes the column selections of a q...uery, and the filters it can translate, into a LazyFrame, so Polars collects only the rows and columns the query needs. A DataFrame is converted in full and filtered afterwards. import duckdb import polars as pl df = pl.DataFrame( { "A": [1, 2, 3, 4, 5],...etle", "beetle", "beetle"], } ) duckdb.sql("SELECT * FROM df").show() DuckDB to Polars DuckDB can output results as Polars DataFrames using the .pl() method of a relation, the object that duckdb.sql and con.sql return. Results of con.execute() have no .pl... - https://man.mtool.work/duckdb/2.0/guides/python/polars.html
- Python API 8784
- Installation To use the DuckDB Python client, visit the Python installation page . The latest stable version of the DuckDB Python client is 2.0.0. Installation The DuckDB Python API can be installed using pip : pip install duckdb . Please s
...example.parquet'") # directly query a Parquet file duckdb.sql("SELECT * FROM 'example.json'") # directly query a JSON file DataFrames DuckDB can directly query Pandas DataFrames, Polars DataFrames and Arrow tables. Note that these are read-only, i.e., edi...ting these tables via INSERT or UPDATE statements is not possible. Pandas To directly query a Pandas DataFrame, run: import duckdb import pandas as pd pandas_df = pd.DataFrame({"a": [42]}) duckdb.sql("SELECT * FROM pandas_df") ┌─────...──┐ │ a │ │ int64 │ ├───────┤ │ 42 │ └───────┘ Polars To directly query a Polars DataFrame, run: import duckdb import polars as pl polars_df = pl.DataFrame({"a": [42]}) duckdb.sql("SELECT * FROM polars_df") ┌─────... - https://man.mtool.work/duckdb/2.0/clients/python/overview.html
- Export to Pandas 8701
- The result of a query can be converted to a Pandas DataFrame using the df() function. import duckdb # read the result of an arbitrary SQL query to a Pandas DataFrame results = duckdb.sql("SELECT 42").df() results 42 0 42 See Also DuckDB als
... - https://man.mtool.work/duckdb/2.0/guides/python/export_pandas.html
DuckDB 2.0 Documentation / man.mtool.work