What is Duckle?

This open-source ETL platform lets teams run data pipelines on their own hardware. You design pipelines on a visual canvas, in Python, or in SQL, and the tool compiles them to SQL that runs on the embedded DuckDB engine. The same pipeline file works on a laptop, a server, in Docker, or as a standalone executable from the Build Pipeline feature. The headless runner (duckle-runner serve) adds a web console with cron scheduling, role-based access, audit logging, and alerting. The project ships 385+ components for sources (databases, warehouses, SaaS APIs, files, streaming), transforms (joins, aggregates, CDC, AI, data quality), and sinks. It also runs dbt on DuckDB, includes an on-device AI assistant (Duckie) that generates pipelines from natural language, and provides an MCP server for AI agents. Authentication supports built-in accounts and OIDC. Licensed under MIT OR Apache-2.0.

Should you self-host Duckle?

Duckle scores 56/100 on selfhost.directory (how scoring works) and ranks #5 of 8 in Data Pipeline. It has 1 532 GitHub stars.

It shipped 10 stable releases in the past year. The latest, v0.7.4, came out on 25 September 2026. New versions land roughly every 8 days.

Setup is moderate: expect some configuration beyond a single command. It is Apache-2.0-licensed and written mostly in Rust.

Its closest alternative on selfhost.directory is sequin (61/100, 2 219 stars). It scores higher overall, so compare the two before you pick. See Duckle vs sequin side by side.

Features

Data quality validators
dbt on DuckDB
Headless server with web console
MCP server for AI agents
Native DuckDB execution
OIDC AuthenticationOptional; disabled unless oidc.json exists
PostgreSQL CDC

FAQ

Does the Duckie panel need internet after install?

No. Once llama.cpp and the Qwen GGUF are downloaded, Duckie needs no network. It can also use your own OpenAI-compatible endpoint instead of running in-process.

Is the Duckie AI assistant any good?

For 90% of common pipelines (read source -> simple transforms -> sink), yes. For complex pipelines, iterate by describing halves. You can also swap the model by pointing xf.ai.llm at GPT-4 or Claude.

Can I run pipelines headlessly / from CI?

Yes. Build Pipeline produces a single self-contained executable for servers/CI. The embedded duckle-runner can also run a workspace pipeline JSON directly (duckle-runner --pipeline pipeline.json).

Can multiple people collaborate on the same workspace?

Via Git, yes — check the workspace into a repo and use standard branch/PR flows, and deploy to a shared server with roles and audit log. No real-time multiplayer canvas.

Where does my pipeline data live?

In the workspace folder you pick on first launch. Pipelines are plain JSON files you can commit to Git, diff, branch, and review.

Can I use OpenAI / Cohere / Voyage instead of the local Duckie?

Yes. The AI transforms (xf.ai.embed, xf.ai.llm, xf.ai.classify) accept a baseUrl prop pointing to any OpenAI-compatible endpoint with an apiKey.

How big is the binary, exactly?

73-110 MB depending on platform (macOS 73-88 MB, Linux 74-100 MB, Windows 98-110 MB). It embeds the headless runner and MCP server; DuckDB and AI engine download separately.

Do I need DuckDB installed first?

No. Duckle downloads DuckDB (~30 MB with common extensions) on first launch automatically.

How big are pipelines this works well on?

Bigger than people assume — the ceiling is the instance you provision. The same pipeline that runs on a laptop sample runs on a large server without changes. Reference: 96M rows from Postgres to Parquet in 39.9s.

Does Duckle send my data anywhere?

No. Duckle makes no outbound calls of its own. The only network calls are those your pipelines explicitly do (e.g., reading from S3, calling OpenAI for embeddings). The built-in AI assistant runs locally with no API key.

How is Duckle different from Airbyte, dbt, or Talend?

Airbyte focuses on hosted extract-and-load; dbt focuses on SQL transformation; Talend is a heavyweight GUI suite (Open Studio discontinued). Duckle is a single open engine that does extract, transform, and load together, compiles to DuckDB SQL, and runs dbt on DuckDB inside the same tool.

Can I run ETL pipelines without the cloud or a data warehouse?

Yes. Duckle executes on the embedded DuckDB engine, so there is no vendor warehouse to buy, no vendor platform to sign up to, and no account. You run it where you choose: a server, VM, container, or workstation.

Is Duckle an open-source alternative to Fivetran or Airbyte?

Duckle covers similar ground — moving data across 190 sources and destinations — but runs locally with nothing to host and no per-row, per-connector, or per-seat billing. The trade-off is scope: Duckle does not split one query across a cluster.

Is Duckle free? What's the license?

Yes, Duckle is free and open source, dual-licensed under MIT OR Apache-2.0. You can use it commercially, fork it, and sell what you build with it. No usage limits, no telemetry.