Parallelising ETL workflows with the Jongleur gem
Jongleur is a process scheduler and manager. It allows its users to create a number of executable tasks as Ruby classes, define precedence between those tasks and run each task as a separate process. It’s particularly useful for implementing workflows modelled as a DAGs (Directed Acyclic Graph), but can be also used to run multiple tasks in parallel or even sequential workflows where each task needs to run as a separate OS process.
Post a comment