software*project
terminal etl tool 路 mysql 路 postgresql 路 csv

Datafowk is a terminal ETL tool for MySQL and PostgreSQL written in Rust.

Define pipelines in a single .toml config file using a compact rule expression: select source tables, join them, filter rows, apply a transform chain, and write to a destination table, all in one line.

Comes with an interactive TUI for editing rules and connections, a schema preview, live data preview, dry-run simulation, and support for CSV as a source alongside MySQL and PostgreSQL.

supported sources
[ mysql ]
read 路 write
[ postgresql ]
read 路 write
[ postgis ]
read 路 [_____]
[ csv ]
read 路 [_____]
supported transforms
[ copy / identity ]
pass value through unchanged
[ trim ]
strip leading/trailing whitespace
[ lowercase / uppercase ]
case conversion
[ sum(n) / subtract(n) ]
add or subtract a constant
[ multiply(n) / divide(n) ]
scale numeric values
[ constant(v) ]
replace field with a fixed value
[ concat(sep?) ]
join two source fields into one
[ unique(field) ]
skip duplicate destination rows
[ area / perimeter ]
computed from postgis geometry