Results of 1 - 7 of about 7 for spark (0.002 sec.)
- Spark API 13504
- The DuckDB Spark API implements the PySpark API , allowing you to use the familiar Spark API to interact with DuckDB. All statements are translated to DuckDB's internal plans using our relational API and executed using DuckDB's query engine
.... The functions it covers are tested against PySpark 4.2.0 and 3.5.9. Warning The DuckDB Spark API is experimental and receives bug fixes only: missing features will not be ad...ded, and a later release will deprecate it. We recommend SQLFrame , which runs PySpark DataFrame code on DuckDB, as its replacement. SQLFrame does not support DuckDB 2.0 yet: SQLFrame 4.5.0 requires duckdb<1.6...for its DuckDB engine. Please report bugs in the Spark API on GitHub . Example from duckdb.experimental.spark.sql import SparkSession as session from duckdb.experimental.spark.s... - https://man.mtool.work/duckdb/2.0/clients/python/spark_api.html
- ODBC Extension 8851
- Please note that DuckDB also offers an ODBC client , which allows you to connect to DuckDB via ODBC from other applications. The odbc_scanner extension allows connecting to other databases (using their ODBC drivers ) and run queries with th
...sic types covered Firebird: types coverage status Tier 3: Snowflake: types coverage status ClickHouse: basic types covered Spark: basic types covered Arrow Flight SQL: basic types covered Installing unixODBC Driver Manager on Linux or macOS On Linux u...WFLAKE_SAMPLE_DATA;UID=username;PWD=pwd; ClickHouse: Driver={ClickHouse ODBC Driver (Unicode)};Server=127.0.0.1;Port=8123; Spark: Driver={Simba Spark ODBC Driver};Host=127.0.0.1;Port=10000; Arrow Flight SQL (Dremio ODBC + GizmoSQL): Driver={Dremio Fli... - https://man.mtool.work/duckdb/2.0/core_extensions/odbc/overview.html
- Unity Catalog Extension 8306
- The unity_catalog extension attaches a Unity Catalog catalog, on Databricks or Unity Catalog OSS, as a DuckDB database. You can list, read, insert into and create its Delta Lake tables. It uses the delta extension to access the tables. For
...avel” blog post . A managed table that DuckDB creates uses catalog-managed commits. Tables created by other clients, e.g., Spark or the Unity Catalog CLI, are read and written the same way. Features This extension supports: Listing available tables: S... - https://man.mtool.work/duckdb/2.0/core_extensions/unity_catalog.html
- Delta Extension 8251
- The delta extension adds support for the Delta Lake open-source storage format . It is built using the Delta Kernel . The extension reads Delta tables, local and remote, and writes them: it can create tables and append to them. For implemen
...delta); INSERT INTO my_table SELECT * FROM other_table; Since v1.5.6, if the table schema declares CHAR(n) or VARCHAR(n) (Spark writes these), a write with a value longer than n characters fails as a whole. Time Travel New Time travel by timestamp wa... - https://man.mtool.work/duckdb/2.0/core_extensions/delta.html
- Reading and Writing Parquet Files 8251
- Examples Read a single Parquet file: SELECT * FROM 'test.parquet'; Figure out which columns/types are in a Parquet file: DESCRIBE SELECT * FROM 'test.parquet'; Create a table from a Parquet file: CREATE TABLE test AS SELECT * FROM 'test.par
...ected) int96_as How to read INT96 columns (a deprecated timestamp encoding used by legacy writers such as Impala and older Spark versions). Options are timestamp (microsecond precision), timestamp_ns (preserves nanoseconds; out-of-range values saturat... - https://man.mtool.work/duckdb/2.0/data/parquet/overview.html
- Connect 8197
- Overview Every DuckDB feature reachable from Java starts with a Connection . This page covers the JDBC URL forms the driver accepts, how DuckDB and driver options are passed and which of them win, how connections come to share (or not share
...t add JDBC properties of their own. The driver already discards two options that are known to be passed by mistake: Apache Spark's path and LibreOffice Base's Type . JDBC-Specific Options Alongside DuckDB's own settings, the driver recognizes a number... - https://man.mtool.work/duckdb/2.0/clients/java/connecting.html
- Configuration 8197
- DuckDB has a number of configuration options that can be used to change the behavior of the system. The configuration options can be set using either the SET statement or the PRAGMA statement . They can be reset to their original values usi
...ue dialect_compatibility_mode Enable SQL dialect compatibility for a certain engine, e.g., SET dialect_compatibility_mode='spark' . VARCHAR NONE disable_database_invalidation Disables invalidating the database instance when encountering a fatal error.... - https://man.mtool.work/duckdb/2.0/configuration/overview.html
PREV
1
NEXT
DuckDB 2.0 Documentation / man.mtool.work