I am developing an ETL package using SSIS. I have some semi complex Data Conversion, validation and field definitions. I can implement it by using some components such as Derive Column , Conditional, data conversion and others. My second option is to write C# script and implement all complexities using language (C#) capabilities.
Which method is more efficient on big data (Using components or C# script)?
Thanks
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试):
问题:
回答1:
The out of the box components are in general probably faster then custom code in a Script Component, unless you need a whole bunch of components to accomplish something that can be done in a simple C# line of code.
An other thing is the use of (partial) blocking components (like sort, union, aggregate, etc.). If you need those, but you could also do it in a none blocking Script Component then it could faster to use the Script Component...
You should find a balance between performance and readability of your package. Adding C# could make it hard to maintain for non-programmers, but if a couple of well documented C# lines could take a way a whole bunch of out of the box components....