Results of 1 - 10 of about 13 for NumPy (0.005 sec.)
- Types API 13214
- The DuckDBPyType class, duckdb.sqltypes.DuckDBPyType , represents one of the data types . Use DuckDBPyType to describe SQL types when declaring function parameters, return types or casts. Python types such as int are accepted too: int descr
...ypes to DuckDB type. Built-in types DuckDB type bool BOOLEAN bytearray BLOB bytes BLOB float DOUBLE int BIGINT str VARCHAR NumPy Scalar Types The table below shows the mapping of NumPy scalar types to DuckDB type. A NumPy dtype object does not convert...: DuckDBPyType(numpy.dtype("int32")) raises a NotImplementedException . Type DuckDB type numpy.bool_ BOOLEAN numpy.float32 FLOAT numpy.float64...DOUBLE numpy.int16 SMALLINT numpy.int32 INTEGER numpy.int64 BIGINT numpy.int8 TINYINT numpy.uint16 USMALLINT numpy.uint32 UINTEGER nump... - https://man.mtool.work/duckdb/2.0/clients/python/types.html
- Export to Numpy 11419
- The result of a query can be converted to Numpy arrays using the fetchnumpy() function, which returns a dictionary that maps each column name to a Numpy array. For example: import duckdb import numpy as np my_arr = duckdb.sql("SELECT unnest
...([1, 2, 3]) AS x, 5.0 AS y").fetchnumpy() my_arr {'x': array([1, 2, 3], dtype=int32), 'y': array([5., 5., 5.])} Then, the array can be processed using Numpy funct...ions, e.g.: np.sum(my_arr['x']) np.int64(6) See Also DuckDB also supports importing from Numpy .... - https://man.mtool.work/duckdb/2.0/guides/python/export_numpy.html
- Integration with PyTorch 9783
- PyTorch tensors can be built from DuckDB query results by exporting each result batch as Apache Arrow . This pattern is useful when training or serving models on Parquet data because DuckDB can filter and project the input before the batche
...s are converted to tensors. Installation pip install -U duckdb numpy pyarrow torch DuckDB to PyTorch This example queries a Parquet-backed Arrow dataset, streams the result as RecordBatch obj...ects and converts each batch into feature and label tensors. import pathlib import tempfile import duckdb import numpy as np import pyarrow as pa import pyarrow.dataset as ds import pyarrow.parquet as pq import torch base_path = pathlib.Path...= 1 """).to_arrow_reader(batch_size=2) for batch in reader: features = torch.tensor( np.column_stack( [ batch.column(0).to_numpy(), batch.column(1).to_numpy(), ] ), dtype=torch.float32, ) labels = torch.tensor(batch.column(2).to_numpy(), dtype=torch.i... - https://man.mtool.work/duckdb/2.0/guides/python/pytorch.html
- Conversion between DuckDB and Python 9378
- This page documents the rules for converting DuckDB results to Python and Python objects to DuckDB . Result Conversion: DuckDB Results to Python A result is read from a relation or, when using the DB-API / con.execute() , from the connectio
...Python tuples fetchall() , iter_rows(batch_size=1024) fetchone() , fetchmany(size=1) , fetchall() Python dicts to_dicts() NumPy fetchnumpy() fetchnumpy() Pandas df() df() , fetch_df_chunk() Apache Arrow to_arrow_table() , to_arrow_reader() to_arrow_t...o_arrow_reader() instead. Note that with the DB-API, after a partial read with fetchone() or fetchmany() , the df() , fetchnumpy() and Arrow methods of the connection return only the remaining rows. Examples Below are some examples. See the Python gui...m value count 0 jeans 20.0 1 1 hammer 42.2 2 2 laptop 2000.0 1 3 chainsaw 500.0 10 4 iphone 300.0 2 Fetch as dictionary of NumPy arrays: arr = con.execute("SELECT * FROM items").fetchnumpy() print(arr) {'item': array(['jeans', 'hammer', 'laptop', 'cha... - https://man.mtool.work/duckdb/2.0/clients/python/conversion.html
- Import from Numpy 9149
- It is possible to query Numpy arrays from DuckDB. There is no need to register the arrays manually – DuckDB can find them in the Python process by name thanks to replacement scans . For example: import duckdb import numpy as np columns = {
...──────┐ │ column0 │ │ double │ ├─────────┤ │ 1.0 │ │ NULL │ │ 3.0 │ └─────────┘ See Also DuckDB also supports exporting to Numpy .... - https://man.mtool.work/duckdb/2.0/guides/python/import_numpy.html
- Troubleshooting 8621
- Troubleshooting Running EXPLAIN Renders Newlines In Python, the output of the EXPLAIN statement contains hard line breaks ( \n ): In [1]: import duckdb ...: duckdb.sql("EXPLAIN SELECT 42 AS x") Out[1]: ┌───────────────┬─────────────────────
...sive / hard to track down. Below is a list of these issues that you might have to be aware of, depending on your workflow. Numpy Import Multithreading When making use of multi threading and fetching results either directly as Numpy arrays or indirectl...y through a Pandas DataFrame, it might be necessary to ensure that numpy.core.multiarray is imported. If this module has not been imported from the main thread, and a different thread during exec...ution attempts to import it this causes either a deadlock or a crash. To avoid this, it's recommended to import numpy.core.multiarray before starting up threads. Many Threads per Database Besides its worker threads, each database starts a p... - https://man.mtool.work/duckdb/2.0/clients/python/known_issues.html
- Guides 8269
- The guides section contains compact how-to guides that are focused on achieving a single goal. For API references and examples, see the rest of the documentation. Note that there are many tools using DuckDB, which are not covered in the off
...o execute SQL on a Pandas DataFrame How to create a table from a Pandas DataFrame How to export data to a Pandas DataFrame NumPy How to create a table from NumPy arrays How to export data to NumPy arrays Apache Arrow How to execute SQL on Apache Arrow... - https://man.mtool.work/duckdb/2.0/guides/overview.html
- Data Ingestion 7865
- This page contains examples for data ingestion to Python using DuckDB. First, import the DuckDB package: import duckdb Then, proceed with any of the following sections. CSV Files CSV files can be read using the read_csv function, called eit
...'one'), (2, 'two'), (3, 'three'), (4, 'four')] The same works for Pandas Series, Polars DataFrames, LazyFrames and Series, NumPy arrays, a dict of one-dimensional NumPy arrays, relations and Arrow objects. The masked elements in a NumPy masked array a... - https://man.mtool.work/duckdb/2.0/clients/python/data_ingestion.html
- Python Function API 7812
- You can create a DuckDB user-defined function (UDF) from a Python function so it can be used in SQL queries. Similarly to regular functions , they need to have a name, a return type and parameter types. Creating Functions To register a Pyth
...duckdb.connect() con.create_function("add_one", add_one) print(con.sql("SELECT add_one(41)").fetchall()) [(42,)] UDFs need NumPy. Without the numpy package, create_function raises InvalidInputException: 'numpy' is required for this operation, but it w... - https://man.mtool.work/duckdb/2.0/clients/python/function.html
- Python Client API 7214
- Connection class duckdb. Appender Bases: object append_row ( self , * args ) → None Buffer one row, one value per column, each converted as an executemany parameter would be append_rows ( self , rows : object ) → None Buffer each row of an
...owing execute() fetchmany ( self , size : int = 1 ) → list Fetch the next set of rows from a result following execute fetchnumpy ( self ) → dict Fetch a result as list of NumPy arrays following execute fetchone ( self ) → object Fetch a single row fro...lse ) → Any Execute and fetch all rows as a pandas DataFrame. Detailed examples can be found at Relational API page . fetchnumpy ( ) → dict [ str , Any ] Execute and fetch all rows as a Python dict mapping each column to one numpy arrays. Detailed exa...duckdb. ConstantExpression ( value : object ) → Expression A value; a duckdb.Value is written as a cast to its type, and a numpy array as the list it holds. duckdb. count_all ( ) → Expression How many rows there are, as count(*) ; unlike col(“x”).coun... - https://man.mtool.work/duckdb/2.0/clients/python/reference/
DuckDB 2.0 Documentation / man.mtool.work