Pentaho-kettle: Need to create ETL Jobs dynamicall

问题:

In my application, user can specify the format of their file. Based on user input we dynamically create SSIS package.

http://lakshmik.blogspot.com/2005/05...eate-ssis.html

Dynamically created SSIS package is used for processing user's files.

We want to evaluate Pentaho-Kettle for this requirement. Is this possible with Kettle to dynamically create ETL jobs based on user's inputs?

If not Pentaho, is there any Java ETL tool which allows use to dynamically create ETL jobs?

回答1:

I dont know about others, but this is traditionally quite tricky in Kettle but people have done it in various ways.

The best option for this is the (brand new) inject step which lets you do really clever stuff with metadata - but it only works for some basic steps. I think it will do what you want, read about it in Matt Casters (PDI Creator and god) blog here:

http://www.ibridge.be/?s=inject&submit=Go

If that doesnt work; then your other options are to go down the generic field name route (nasty) or dynamically generate the transformation. This is easier than it sounds - but you will need to get much more heavily involved in the Java side than usual for an ETL tool.

回答2:

It is possible, and not very hard.

You can use the Kettle API to dynamically create transformations that can do anything Kettle does. The GUI designer uses the API to create transformations, so anything you can do with the GUI, you can do thru the API.

If you look in the 'test' source tree you will find lots of examples of how to create transformations dynamically.