Results of 1 - 10 of about 13 for readers (0.002 sec.)
- Multiple Python Threads 14094
- This page demonstrates how to simultaneously insert into and read from a DuckDB database across multiple Python threads. This could be useful in scenarios where new data is flowing in and an analysis should be periodically re-run. Note that
...AS read_time FROM my_inserts """, (thread_name,)).fetchall() print(results) Create Threads We define how many writers and readers to use, and define a list to track all of the threads that will be created. Then, create first writer and then reader thre...ads. Next, shuffle them so that they will be kicked off in a random order to simulate simultaneous writers and readers. Note that the threads have not yet been executed, only defined. write_thread_count = 50 read_thread_count = 5 threads = [...rget = read_from_thread, args = (duckdb_con,), name = 'read_thread_' + str(j))) # Shuffle the threads to simulate a mix of readers and writers random.seed(6) # Set the seed to ensure consistent results when testing random.shuffle(threads) Run Threads an... - https://man.mtool.work/duckdb/2.0/guides/python/multiple_threads.html
- Importing Data 10539
- The first step to using a database system is to insert data into that system. DuckDB can directly connect to many popular data sources and offers several data ingestion methods that allow you to easily and efficiently fill up the database.
...on'; For more details, see the page on JSON loading . Returning the Filename Since DuckDB 1.3.0, the CSV, JSON and Parquet readers support the filename virtual column: COPY (FROM (VALUES (42), (43)) t(x)) TO 'test.parquet'; SELECT *, filename FROM 'test....parquet'; Returning the File Index The CSV, JSON and Parquet readers also support the file_index virtual column, which returns the zero-based index of the file that each row came from, in the... - https://man.mtool.work/duckdb/2.0/data/overview.html
- Import Data 9835
- Overview The Go client offers two ways to load data into DuckDB in bulk: the high-performance Appender for inserting rows generated in Go, and reading directly from data files such as Parquet, CSV, and JSON. Both are described below. For in
...is often the simplest way to load external data: the Go client just sends the query. Because the Parquet , CSV , and JSON readers are among the bundled extensions , no extra installation is needed. Query a file in place with the matching table function...ta Import — DuckDB's full set of bulk-loading options. Run Queries — sending the CREATE TABLE and COPY statements the file readers build on. Connect — creating the Connector and connection the Appender operates on.... - https://man.mtool.work/duckdb/2.0/clients/go/data_import.html
- Avro Extension 8410
- The avro extension enables DuckDB to read Apache Avro files. The avro extension was released as a community extension in late 2024 and became a core extension in early 2025. The read_avro Function The extension adds a single DuckDB function
...o files we have seen so far had their schema embedded. There is currently no support for the union_by_name flag that other readers in DuckDB support. This is planned for the future.... - https://man.mtool.work/duckdb/2.0/core_extensions/avro.html
- Azure Extension 8410
- The azure extension lets DuckDB read and write files in Azure Blob Storage and Azure Data Lake Storage (ADLS) Gen2. Installing and Loading The azure extension will be transparently autoloaded on first use from the official extension reposit
...e_write_block_size bytes (default: 8 MiB). By default, the uploaded blocks are committed when the file is closed, so other readers see the new content only after the write completes. To commit earlier, set azure_write_staged_blocks_per_commit to the num...r of blocks after which an intermediate commit happens. Warning An intermediate commit makes an incomplete file visible to readers. A Parquet file cannot be read until its footer is written at the end, so keep the default 0 for Parquet files and for Del... - https://man.mtool.work/duckdb/2.0/core_extensions/azure.html
- Troubleshooting 8164
- Troubleshooting Running EXPLAIN Renders Newlines In Python, the output of the EXPLAIN statement contains hard line breaks ( \n ): In [1]: import duckdb ...: duckdb.sql("EXPLAIN SELECT 42 AS x") Out[1]: ┌───────────────┬─────────────────────
...t after connecting instead: import duckdb con = duckdb.connect() con.execute("SET max_temp_directory_size = '10GB'") Arrow Readers Hang on Windows after a Subinterpreter Was Created On Windows, once the process has created a subinterpreter, a query that... - https://man.mtool.work/duckdb/2.0/clients/python/known_issues.html
- Import Data 8164
- Overview The Rust client offers two ways to get data into DuckDB in bulk: the high-performance Appender for inserting rows from Rust, and reading directly from data files such as Parquet, CSV, and JSON. Both are described below. Appender Th
...ow integration used to append and read record batches. Run Queries — sending the CREATE TABLE and COPY statements the file readers build on.... - https://man.mtool.work/duckdb/2.0/clients/rust/data_import.html
- Reading Multiple Files 8164
- DuckDB can read multiple files of different types (CSV, Parquet, JSON files) at the same time using either the glob syntax, or by providing a list of files to read. See the combining schemas page for tips on reading files with different sch
...et'); IO Error: No files found that match the pattern "test/*.parquet" Starting with DuckDB 2.0, the CSV, JSON and Parquet readers accept the allow_empty option, which returns an empty result (with a single empty column of type BOOLEAN ) instead of rais... - https://man.mtool.work/duckdb/2.0/data/multiple_files/overview.html
- COPY Statement 8164
- Examples Read a CSV file into the lineitem table, using auto-detected CSV options: COPY lineitem FROM 'lineitem.csv'; Read a CSV file into the lineitem table, using manually specified CSV options: COPY lineitem FROM 'lineitem.csv' (DELIMITE
...o 0 to disable dictionary encoding. BIGINT ROW_GROUP_SIZE / 5 WRITE_BLOOM_FILTER Whether to write Bloom filters that allow readers to skip row groups. BOOLEAN true BLOOM_FILTER_FALSE_POSITIVE_RATIO The target false positive ratio of the written Bloom fi...MP_AS_INT96 When true , writes TIMESTAMP columns using the legacy Parquet INT96 physical type for compatibility with older readers. Added in DuckDB 2.0. BOOLEAN false New As of DuckDB 2.0, the TIMESTAMP_IS_ADJUSTED_TO_UTC and WRITE_TIMESTAMP_AS_INT96 op... - https://man.mtool.work/duckdb/2.0/sql/statements/copy.html
- Rust Client 7935
- Installation To use the DuckDB Rust client, visit the Rust installation page . The latest stable version of the DuckDB Rust client is 2.0.0. The DuckDB Rust client, duckdb-rs , is an ergonomic wrapper over the DuckDB C API that exposes an i
...e line, use the v1.4-andium branch. Tip The Rust client offers a large set of Cargo feature flags that turn on file-format readers, the Arrow and Polars integrations, connection pooling, and the function and extension interfaces. Enable only the feature... - https://man.mtool.work/duckdb/2.0/clients/rust/overview.html
DuckDB 2.0 Documentation / man.mtool.work