Results of 1 - 10 of about 11 for Zstd (0.002 sec.)
- Reading and Writing Parquet Files 13016
- Examples Read a single Parquet file: SELECT * FROM 'test.parquet'; Figure out which columns/types are in a Parquet file: DESCRIBE SELECT * FROM 'test.parquet'; Create a table from a Parquet file: CREATE TABLE test AS SELECT * FROM 'test.par
...ecific compression and row group size: COPY (FROM generate_series(100_000)) TO 'test.parquet' (FORMAT parquet, COMPRESSION zstd, ROW_GROUP_SIZE 100_000); Export the table contents of the entire database as parquet: EXPORT DATABASE 'target_directory'...to a Snappy-compressed Parquet file: COPY (SELECT * FROM tbl) TO 'result-snappy.parquet' (FORMAT parquet); Write tbl to a zstd-compressed Parquet file: COPY tbl TO 'result-zstd.parquet' (FORMAT parquet, COMPRESSION zstd); Write tbl to a zstd-compres...sed Parquet file with the lowest compression level yielding the fastest compression: COPY tbl TO 'result-zstd.parquet' (FORMAT parquet, COMPRESSION zstd, COMPRESSION_LEVEL 1); Write to Parquet file with key-value metadata : COPY ( S... - https://man.mtool.work/duckdb/2.0/data/parquet/overview.html
- COPY Statement 10864
- Examples Read a CSV file into the lineitem table, using auto-detected CSV options: COPY lineitem FROM 'lineitem.csv'; Read a CSV file into the lineitem table, using manually specified CSV options: COPY lineitem FROM 'lineitem.csv' (DELIMITE
...result of a query to a Parquet file: COPY (SELECT l_orderkey, l_partkey FROM lineitem) TO 'lineitem.parquet' (COMPRESSION zstd); Copy the entire content of database db1 to database db2 : COPY FROM DATABASE db1 TO db2; Copy only the schema (catalog e...fault this will be detected automatically from the file extension (e.g., file.csv.gz will use gzip , file.csv.zst will use zstd , and file.csv will use none ). Options are none , gzip , zstd . VARCHAR auto DATEFORMAT Specifies the date format to use...ng Parquet files. Name Description Type Default COMPRESSION The compression format to use ( uncompressed , snappy , gzip , zstd , brotli , lz4 , lz4_raw ). VARCHAR snappy COMPRESSION_LEVEL Compression level, set between 1 (lowest compression, fastest... - https://man.mtool.work/duckdb/2.0/sql/statements/copy.html
- Text Types 9637
- In DuckDB, strings can be stored in the VARCHAR field. The field allows storage of Unicode characters. Internally, the data is encoded as UTF-8. Name Aliases Description VARCHAR CHAR , BPCHAR , STRING , TEXT Variable-length character string
...Compression Type You can specify a compression type for a string with the USING COMPRESSION clause. For example, to apply zstd compression, run: CREATE TABLE tbl (s VARCHAR USING COMPRESSION zstd); Text Type Values Values of the text type are charac... - https://man.mtool.work/duckdb/2.0/sql/data_types/text.html
- EXPORT and IMPORT DATABASE Statements 9371
- The EXPORT DATABASE command allows you to export the contents of the database to a specific directory. The IMPORT DATABASE command allows you to then read the contents again. Examples Export the database to the target directory target_direc
...'target_directory' (FORMAT parquet); Export to directory target_directory , tables serialized as Parquet, compressed with Zstd, with a ROW_GROUP_SIZE of 100,000: EXPORT DATABASE 'target_directory' ( FORMAT parquet, COMPRESSION zstd, ROW_GROUP_SIZE 1... - https://man.mtool.work/duckdb/2.0/sql/statements/export.html
- File Formats 9246
- Handling Parquet Files DuckDB has advanced support for Parquet files, which includes directly querying Parquet files . When deciding on whether to query these files directly or to first load them to the database, you need to consider severa
...nsive decompression time every time the file is accessed. Meanwhile, lightweight compression methods like Snappy, LZ4, and zstd, are faster to decompress. You may use the parquet_metadata function to find out the compression algorithm used. Microbenc...ssion options on the attached catalog: ATTACH 'ducklake:metadata.ducklake' AS d; CALL d.set_option('parquet_compression', 'zstd'); CALL d.set_option('parquet_version', 2); CALL d.set_option('parquet_compression_level', 22); COPY FROM DATABASE service...s TO d; Using a heavyweight zstd compression level trades a longer write time for smaller files, so it is most beneficial for data that is written once and... - https://man.mtool.work/duckdb/2.0/guides/performance/file_formats.html
- PHP Client (PDO) 9104
- The PHP DuckDB PDO extension is a tertiary client and is maintained by a third-party. To use pdo_duckdb , follow the instructions below . The latest version of pdo_duckdb supports DuckDB 2.0.0. pdo_duckdb is a native DuckDB database driver
...t files <?php $db = new PDO('duckdb::memory:'); $db->exec('CREATE TABLE table1 (id INTEGER, text VARCHAR USING COMPRESSION zstd, data JSON)'); $statement = $db->prepare('INSERT INTO table1 VALUES (?, ?, ?)'); $statement->execute([1, 'Hello DuckDB ?',...['foo' => 'bar', 'baz' => 42]]); $db->exec("COPY (SELECT * FROM table1) TO '/tmp/table1.parquet' (COMPRESSION zstd)"); foreach ($db->query("SELECT * FROM '/tmp/table1.parquet'", PDO::FETCH_ASSOC) as $row) { print_r($row); } # Array # ( #...( # [log] => log text 2 # ) # ) $db->exec("COPY (SELECT * FROM '/tmp/logs.json') TO '/tmp/logs_json.parquet' (COMPRESSION zstd)"); Use structured columns with a fixed schema <?php # s is array{v: string, i: int, a: string[], d: float} $db = new PDO(... - https://man.mtool.work/duckdb/2.0/clients/tertiary_clients/php_pdo.html
- Configuration 7486
- DuckDB has a number of configuration options that can be used to change the behavior of the system. The configuration options can be set using either the SET statement or the PRAGMA statement . They can be reset to their original values usi
...is reached, we flush the data to disk. Defaults to 20% of the memory limit divided by the thread count. VARCHAR 413.9 MiB zstd_min_string_length The (average) minimum length at which to enable ZSTD compression, defaults to 4096. UBIGINT 4096 Local C... - https://man.mtool.work/duckdb/2.0/configuration/overview.html
- Loading JSON 7343
- The DuckDB JSON reader can automatically infer which configuration flags to use by analyzing the JSON file. This will work correctly in most situations, and should be the first option attempted. In rare situations where the JSON reader cann
...ted automatically from the file extension (e.g., t.json.gz will use gzip, t.json will use none). Options are none , gzip , zstd and auto_detect . VARCHAR auto_detect filename Whether or not an extra filename column should be included in the result. S... - https://man.mtool.work/duckdb/2.0/data/json/loading_json.html
- Storage Versions and Format 7343
- Compatibility Backward Compatibility Backward compatibility refers to the ability of a newer DuckDB version to read storage files created by an older DuckDB version. Version 0.10 is the first release of DuckDB that supports backward compati
...ST) – VLDB 2020 paper Adaptive Lossless Floating-Point Compression (ALP) – SIGMOD 2024 paper Chimp – VLDB 2022 paper Patas Zstd Disk Usage The disk usage of DuckDB's format depends on a number of factors, including the data type and the data distribu... - https://man.mtool.work/duckdb/2.0/internals/storage.html
- CSV Import 7219
- Examples The following examples use the flights.csv file. Read a CSV file from disk, auto-infer options: SELECT * FROM 'flights.csv'; Use the read_csv function with custom options: SELECT * FROM read_csv('flights.csv', delim = '|', header =
...cted automatically from the file extension (e.g., t.csv.gz will use gzip, t.csv will use none ). Options are none , gzip , zstd . VARCHAR auto dateformat Date format used when parsing and writing dates. VARCHAR (empty) date_format Alias for dateforma... - https://man.mtool.work/duckdb/2.0/data/csv/overview.html
DuckDB 2.0 Documentation / man.mtool.work