Results of 1 - 4 of about 4 for sniff (0.002 sec.)
- CSV Auto Detection 12408
- When using read_csv , the system tries to automatically infer how to read the CSV file using the CSV sniffer . This step is necessary because CSV files are not self-describing and come in many different dialects. The auto-detection works ro
...which we cannot jump – such as a .gz compressed CSV file or stdin – samples are taken only from the beginning of the file. sniff_csv Function It is possible to run the CSV sniffer as a separate step using the sniff_csv(filename) function, which return...s a table with a single row. Besides the commonly used sample_size parameter, which configures the number of rows sampled, sniff_csv accepts the same named parameters as the read_csv function . Any parameter you pass is treated as fixed: the sniffer u...value you provide and only detects the properties you did not specify. For example, passing ignore_errors = true lets the sniffer skip malformed lines while it detects the dialect. The auto_detect parameter must be true (its default) for sniff_csv ;... - https://man.mtool.work/duckdb/2.0/data/csv/auto_detection.html
- CSV Import 8506
- Examples The following examples use the flights.csv file. Read a CSV file from disk, auto-infer options: SELECT * FROM 'flights.csv'; Use the read_csv function with custom options: SELECT * FROM read_csv('flights.csv', delim = '|', header =
...ns. The DuckDB CSV reader can automatically infer which configuration flags to use by analyzing the CSV file using the CSV sniffer . This will work correctly in most situations, and should be the first option attempted. In rare situations where the CS.... Auto-detection ( auto_detect ) is enabled by default. Setting an option explicitly does not turn auto-detection off: the sniffer still detects every option you did not specify, and your explicit value overrides the sniffer only for that option. To d...ted values to NULL values BOOL true auto_detect Auto detect CSV parameters . BOOL true auto_type_candidates Types that the sniffer uses when detecting column types. The VARCHAR type is always included as a fallback option. See example . TYPE[] default... - https://man.mtool.work/duckdb/2.0/data/csv/overview.html
- File Formats 8247
- Handling Parquet Files DuckDB has advanced support for Parquet files, which includes directly querying Parquet files . When deciding on whether to query these files directly or to first load them to the database, you need to consider severa
...parallel gunzip ) and loading from decompressed CSV files 121.3 s Loading Many Small CSV Files The CSV reader runs the CSV sniffer on all files. For many small files, this may cause an unnecessarily high overhead. A potential optimization to speed thi...s up is to turn the sniffer off. Assuming that all files have the same CSV dialect and column names/types, get the sniffer options as follows: .mode...line SELECT Prompt FROM sniff_csv('part-0001.csv'); Prompt = FROM read_csv('file_path.csv', auto_detect=false, delim=',', quote='"', escape='"', new_lin... - https://man.mtool.work/duckdb/2.0/guides/performance/file_formats.html
- Relational API 7785
- The Relational API builds a query one step at a time from Python. A relation, a DuckDBPyRelation , is a piece of SQL text: the methods of the Transformation and Functions sections return a new relation whose text extends the one they were c
.... * sample_size : int, keyword-only, default: None Number of lines sampled to detect the dialect and the types. * files_to_sniff : int, keyword-only, default: None Number of files sampled to detect the schema when reading several files; -1 samples all... - https://man.mtool.work/duckdb/2.0/clients/python/relational_api.html
PREV
1
NEXT
DuckDB 2.0 Documentation / man.mtool.work