Pentaho Spoon Tool Transformation Order

Question

I am trying to design an ETL structure and i stucked in the below step.

As you can see i have 3 steps and each step holding a FK value from previous step. For example TABLE3 has a column with FK Constraints which shows the PK value in TABLE2 and TABLE2 has same relationship with TABLE1.

The problem is when i start this job all these 3 steps are running simultaneously and error is occuring about FK constraints since TABLE2 trying to set a FK before TABLE1 creation completes.

I am not sure if this is the best practice for creating an ETL process but is there a way to run these 3 steps in specified sequence?

Answer 1

My solution;

In Design panel under the Flow steps there is a usefull tool called "Block this step until steps finish" which helps you observe the steps you want and than start the step which is blocked so that you will have no problem with FK and other dependencies.. Here is the implementation;

Answer 2

Inserting/updating the same database in parallel, i am afraid will throw you an error (like you have done). please try to separate these transformations into multiple ktr files. I assume you will not face the same issue again. Hope it helps:)

Answer 3

The best way is to set up a JOB, which calls the 3 transformations in the specific order.

Job entries run sequentially, whereas transformation steps always run in parallel. If you need even A to take place before event B they should be put in that order inside a job.