I have a series of data that is produced by state and information type. For each state I have, for example, 3 files that have an identifier in the file name to tell me which information group it is for. The data in these files consists of rows of data that have varying lengths depending on the file type. Type 1 always has 245 comma separated values, Type 2 has 215, Type 3 has 300.
I have 3 separate SSIS 2005 packages set up to import this data but I was wondering if there is a way to do this in one package with either a Conditional Split or a Scripting Task. I have tried the Conditional Split but it looks to me like it is expecting to have to check a row value. I then looked at the code here that uses a Scripting Task. I cannot get this code to compile as it complains about "DTS not being defined." This is probably one of those cases where the author assumes I have knowledge in some, for him, basic SSIS idea - and that I do not have that knowledge. I have modified the code there to be:
Imports System
Imports System.Data
Imports System.Math
Imports System.IO
Imports Microsoft.SqlServer.Dts.Runtime
Public Class ScriptMain
' Created: Zack Bethem – AmberLeaf
Public Sub Main()
Dim fileLoc, fileName As String
If Dts.Variables.Contains("User::fileName") = True Then
fileName = CStr(Dts.Variables.Item("User::fileName").Value)
If fileName.Contains("0074000") Then
Dts.Variables.Item("User::SexByInd").Value = True
ElseIf fileName.Contains("0072000") Then
Dts.Variables.Item("User::SexByOcc").Value = True
ElseIf fileName.Contains("0022000") Then
Dts.Variables.Item("User::Migration").Value = True
End If
Dts.TaskResult = Dts.Results.Success
Else
Dts.TaskResult = Dts.Results.Failure
End If
End Sub
End Class
As you can see from the code:
I am looping over the files in a directory using a For Each loop.
I am assigning the file name for each file to User::fileName
and then checking if the file name contains one of 3 string elements.
If it contains Type 1 then it goes to a different branch than Type2/Type3, etc.
Since this code does not compile I am cannot check to see if the rest of his example will work. I am not sure what kind of Scripting Task he is using either since I was presented with 3 options but all involved some kind of manipulation of a row or row element of data from as the Input to the Scripting Task. What I want to do is pass the User::fileName
as the input and get then test the boolean values of the other 3 User::
variables to make the 3 branches.
My current thinking is this: Inside a single For Each Loop there is this Data Flow Task:
Scripting Task
/ | \
(Type1) (Type2) (Type3)
| | |
CSV_Files1 CSV_Files2 CSV_Files3
| | |
OLEDB_Dest1 OLEDB_Dest2 OLEDB_Dest3
Where the CSV_Files1-3 use a unique SourceConnectionFlat file to read/map the columns to the OLEDB_Dest tables.
EDITS: