Results of 1 - 10 of about 17 for operating system (0.002 sec.)
operating (23), system (112)
- I/O 20882
- Warning DuckDB ships with safe defaults. The configuration options described on this page are advanced options, so proceed with caution when changing them. Asynchronous I/O New Asynchronous I/O was introduced in DuckDB 2.0. Starting with v2
...e waiting for responses (e.g., HTTP requests), there are more of them than CPU cores: by default, four times the number of system threads, capped at 256. Its size is controlled by the async_threads setting. For example, to set the number of asynchronou.... By default the capacity is derived automatically from the number of CPU threads and asynchronous threads, bounded by the operating system file-descriptor limit, and capped at 256. Resizing async_threads also resizes the automatic pool so that it tracks...T http_client_pool_capacity = 8; Synchronizing Writes to Disk When DuckDB persists changes to a database on the local file system, it asks the operating system to flush the written data to stable storage. The fsync_mode configuration option controls ho... - https://man.mtool.work/duckdb/2.0/data/io.html
- jemalloc 20233
- On supported platforms, DuckDB replaces the system's memory allocator with jemalloc . jemalloc is statically linked into DuckDB and cannot be installed or loaded during runtime. For technical reasons, we used to distribute jemalloc as an ex
...tension. Since DuckDB 1.5.3, it is distributed as a third-party library. Operating System Support The availability of jemalloc depends on the operating system. Linux 64-bit Linux distributions of DuckDB sh...assed to CMake as -DENABLE_JEMALLOC=0 . macOS and Windows On macOS and Windows, jemalloc is not available. DuckDB uses the system's memory allocator on these platforms. Configuration Environment Variables The jemalloc allocator in DuckDB can be configu... - https://man.mtool.work/duckdb/2.0/internals/jemalloc.html
- My Workload Is Slow 20145
- If you find that your workload in DuckDB is slow, we recommend performing the following checks. More detailed instructions are linked for each point. Do you have enough memory? DuckDB works best if you have 1-4 GB memory per thread . Is you
...r system maybe overcommitting memory, forcing the operating system to swap? Try lowering the amount of memory available from the de...rformance, especially in memory-constrained environments where other processes are likely using more than 20% of the total system memory. Are you using a fast disk? Network-attached disks (such as cloud block storage) cause write-intensive and larger t...MB to 10 GB? Does the query plan look right? Study it with EXPLAIN . Is the workload running in parallel ? Use htop or the operating system's task manager to observe this. Is DuckDB using too many threads? Try limiting the amount of threads . Are you awar... - https://man.mtool.work/duckdb/2.0/guides/performance/my_workload_is_slow.html
- Environment 19431
- The environment where DuckDB is run has an obvious impact on performance. This page focuses on the effects of the hardware configuration and the operating system used. Hardware Configuration CPU DuckDB's officially supported architectures a
...based storage but they do turn compression off for in-memory storage. Read more in the “How to Tune Workloads” page . File systems. On Linux, DuckDB performs best with the XFS file system but it also performs well with other file systems such as ext4....s, we recommend using NTFS and avoiding FAT32. DuckDB databases have built-in checksums, so integrity checks from the file system are not required to prevent data corruption. Network-Attached Disks Special care needs to be taken when using network-atta...hich avoids the concurrent read-write access patterns on a single large file that network-attached storage handles poorly. Operating System We recommend using the latest stable version of operating systems: macOS, Windows, and Linux are all well-tested an... - https://man.mtool.work/duckdb/2.0/guides/performance/environment.html
- Glossary of Terms 18371
- This page contains a glossary of a few common terms used in DuckDB. Terms In-Process Database Management System The DBMS runs in the client application's process instead of running as a separate process, which is common in the traditional c
...lient–server setup. An alternative term is embeddable database management system. In general, the term “embedded database management system” should be avoided, as it can be confused with DBMSs targeting...embedded systems (which run on e.g., microcontrollers). Replacement Scan In DuckDB, replacement scans are used when a table name used by a...s, and domain-specific functionality. For details, see the Extensions page . Platform The platform is a combination of the operating system (e.g., Linux, macOS, Windows), system architecture (e.g., AMD64, ARM64), and, optionally, the compiler used (e.g.,... - https://man.mtool.work/duckdb/2.0/guides/glossary.html
- CSV Auto Detection 17130
- When using read_csv , the system tries to automatically infer how to read the CSV file using the CSV sniffer . This step is necessary because CSV files are not self-describing and come in many different dialects. The auto-detection works ro
...ng rule, escape). Detect the types of each of the columns. Detect whether or not the file has a header row. By default the system will try to auto-detect all options. However, options can be individually overridden by the user. This can be useful in ca...se the system makes a mistake. For example, if the delimiter is chosen incorrectly, we can override it by calling the read_csv with an e...xplicit delimiter (e.g., read_csv('file.csv', delim = '|') ). Sample Size The type detection works by operating on a sample of the file. The size of the sample can be modified by setting the sample_size parameter. The default sample s... - https://man.mtool.work/duckdb/2.0/data/csv/auto_detection.html
- Securing DuckDB 16292
- DuckDB is a powerful analytical database engine. It can read and write files, access the network, load extensions, and use system resources. Like any powerful tool, these capabilities require appropriate configuration when working with sens
...tainer (e.g., Docker with restricted capabilities) Use a virtual machine or separate process with minimal privileges Apply operating system-level sandboxing Use network isolation to prevent data exfiltration Implement strict query timeouts at the applicat...they don't execute raw SQL strings, they still trigger DuckDB operations that can read files, access the network, and use system resources. Example considerations for non-SQL APIs: File paths: Functions like duckdb.read_csv(path) or duckdb.read_parque...ons DuckDB has a flexible extension mechanism that adds functionality such as new file formats, functions, and remote file system access. Extensions run with the same privileges as the DuckDB process itself, so they warrant careful consideration in sec... - https://man.mtool.work/duckdb/2.0/operations_manual/securing_duckdb/overview.htm...
- Out-of-Memory Issues 15882
- Configuration Options If you are experiencing out-of-memory issues with DuckDB, try tweaking the following configuration options: Reduce the number of threads using the SET threads = ... command. Setting the memory limit lower than the defa
...ger and thus they can reserve more memory than allowed by the memory limit. If this happens (e.g., DuckDB is killed by the operating system or an OOM reaper process), set the memory limit to just 50-60% of the total system memory by using the SET memory_l...short, try the settings: SET threads = ⟨lower_than_the_number_of_available_threads⟩; SET memory_limit = ⟨lower_than_80%_of_system_memory⟩; SET preserve_insertion_order = false; Schema and Indexes If you run in a memory-constrained environment, using sm... - https://man.mtool.work/duckdb/2.0/guides/performance/oom.html
- Crashes 15010
- DuckDB is thoroughly tested via an extensive test suite. However, bugs can still occur and these can sometimes lead to crashes. This page contains practical information on how to troubleshoot DuckDB crashes. Types of Crashes There are a few
...n. Out of memory errors: In rare cases, DuckDB can crash due to memory exhaustion. This crash can also be a symptom of the operating system killing the process. For example, many Linux distributions run an OOM reaper or OOM killer process , which kills pr...ocesses to free up their memory and thus prevents the operating system from running out of memory. If your DuckDB session runs out of memory, consult the “OOM errors” page . Corrupted da... - https://man.mtool.work/duckdb/2.0/guides/troubleshooting/crashes.html
- DuckDB Environment 15009
- DuckDB provides a number of functions and PRAGMA options to retrieve information on the running DuckDB instance and its environment. Version The version() function returns the version number of DuckDB. SELECT version() AS version; version v
...version; library_version source_id codename v2.0.0 d54cbbe3df Cyanoptera Platform The platform information consists of the operating system, system architecture, and, optionally, the compiler. The platform is used when installing extensions . To retrieve... - https://man.mtool.work/duckdb/2.0/guides/meta/duckdb_environment.html
DuckDB 2.0 Documentation / man.mtool.work