What is RAT?

RAT is a self-hostable data platform that makes data engineering accessible to everyone. Write SQL (or Python), run pipelines, and query your data — all from a web-based IDE.

RAT is built with a Go platform, Python execution engine, and Next.js portal. Community Edition is free and open-source.

Why RAT?

  • Anyone can data — If you can write SQL, you can build data pipelines
  • Self-hostable — One docker compose up and you’re running
  • Batteries included — Editor, scheduler, query console, lineage, quality tests — all built-in
  • Apache Iceberg — Your data lives in open table format, not locked in a vendor
  • Git-like isolation — Each pipeline run gets its own branch — bad data never reaches production

How It Works

RAT uses a medallion architecture (Bronze → Silver → Gold) to organize data transformations:

You write SQL pipelines that transform data between layers. RAT handles execution, scheduling, versioning, and quality testing.

Quick Example

Here’s a complete pipeline that deduplicates raw orders into a clean Silver table:

pipeline.sql
-- @merge_strategy: incremental
-- @unique_key: order_id
-- @watermark_column: updated_at
 
SELECT
    order_id,
    customer_id,
    total_amount,
    status,
    updated_at
FROM {{ ref('bronze.raw_orders') }}
{% if is_incremental() %}
WHERE updated_at > '{{ watermark_value }}'
{% endif %}

Platform at a Glance

ComponentWhat It Does
PortalWeb IDE — editor, query console, DAG visualization
ratdGo API server — orchestration, scheduling, auth
runnerPython pipeline executor — DuckDB + Iceberg writes
ratqPython query service — read-only DuckDB queries
PostgresPlatform state (pipelines, runs, schedules)
MinIOS3-compatible object storage for data files
NessieGit-like Iceberg catalog for branch isolation

Get Started

Architecture Overview

RAT runs as 7 containers orchestrated by Docker Compose:

Read more in the Architecture Overview.


🏚️

RAT is built by Le Squat — an underground builders collective. Open-source, community-first.