Part 2: Landing Zones — Real Data Ingestion

In Part 1, you used a VALUES clause to generate sample data. That works for learning, but real platforms ingest data from files. In this part, you will upload real CSV files through landing zones and build pipelines that read from them.

Time required: ~15 minutes

Prerequisites: Completed Part 1

What You Will Build

Two Bronze pipelines that ingest real space launch data:

• mission_log — reads 25 launches from space_launches.csv
• rocket_registry — reads 11 launch vehicles from launch_vehicles.csv

Both pipelines will read from landing zones — RAT’s mechanism for bringing external files into the platform.

What Are Landing Zones?

A landing zone is a named drop area where you upload files (CSV, Parquet, JSON). Pipelines reference landing zones using the landing_zone() Jinja function, and RAT resolves the path to the actual files in S3.

Landing zones are useful because:

• They decouple file upload from pipeline execution
• Files can be uploaded via the portal UI, curl, or S3 API
• You can upload sample files for preview (so you can test before running)
• Landing zone uploads can trigger pipelines automatically (Part 7)

Create the Launch Data Landing Zone

curl -X POST http://localhost:8080/api/v1/landing-zones/launch-data/files \
  -F "file=@docs/data/space_launches.csv"

curl -X POST http://localhost:8080/api/v1/landing-zones/launch-data/samples \
  -F "file=@docs/data/space_launches.csv"

Create the Mission Log Pipeline

-- @description: Space launch mission data from landing zone
 
SELECT
    launch_id,
    mission_name,
    launch_date,
    vehicle,
    launch_site,
    country,
    orbit,
    outcome,
    payload_mass_kg,
    mission_type
FROM read_csv_auto('{{ landing_zone("launch-data") }}')

[INFO]  Starting pipeline: default.bronze.mission_log
[INFO]  Creating branch: run-xyz789
[INFO]  Executing SQL pipeline...
[INFO]  Query produced 25 rows
[INFO]  Writing to Iceberg table: default.bronze.mission_log
[INFO]  Merging branch: run-xyz789 → main
[INFO]  Pipeline completed successfully in 1.8s

SELECT COUNT(*) AS total, COUNT(DISTINCT country) AS countries
FROM "default"."bronze"."mission_log"

Create the Vehicle Catalog Landing Zone

Now repeat the process for the launch vehicles data.

# Upload data file
curl -X POST http://localhost:8080/api/v1/landing-zones/vehicle-catalog/files \
  -F "file=@docs/data/launch_vehicles.csv"
 
# Upload sample file
curl -X POST http://localhost:8080/api/v1/landing-zones/vehicle-catalog/samples \
  -F "file=@docs/data/launch_vehicles.csv"

Create the Rocket Registry Pipeline

-- @description: Launch vehicle specifications from vendor catalog
 
SELECT
    vehicle_name,
    manufacturer,
    country,
    height_m,
    diameter_m,
    mass_kg,
    thrust_kn,
    stages,
    first_flight,
    active
FROM read_csv_auto('{{ landing_zone("vehicle-catalog") }}')

SELECT vehicle_name, manufacturer, country, active
FROM "default"."bronze"."rocket_registry"
ORDER BY vehicle_name

Check Your Progress

At this point, you should have:

Navigate to Explorer to see all three tables in the tree:

default/
  bronze/
    mission_log (25 rows)
    raw_launches (5 rows)
    rocket_registry (11 rows)

You can also check Runs to see the history of all 3 pipeline executions.

What You Learned

• Landing zones are named file drop areas for ingesting external data
• {{ landing_zone("name") }} resolves to the S3 path at execution time
• DuckDB’s read_csv_auto() handles CSV parsing and schema detection
• Sample files enable preview without a full pipeline run
• Files can be uploaded via the portal UI or the REST API (curl)

Next Steps

You now have two Bronze tables with real data — launches and vehicles. In Part 3, you will create a Silver pipeline that joins them together using ref(), and you will see the lineage DAG that visualizes how your pipelines connect.