SSIS LookUp is not dealing with NULLs like the doc

问题:

I have an SSIS data flow that uses a lookup. Sometimes the value to be looked up (in my stream, not in the lookup table) is null.

The MSDN Docs say:

consider using full caching, which supports lookup operations on null values.

I am using Full Caching (that is the default).

But when I run I get this error on my null rows:

Row yielded no match during lookup

If I change the result to ignore no-matches then it works fine. But that ignores all no-matches. I just want to allow nulls through (as null). Any other no-match should fail the component.

What am I doing wrong? How can I get nulls to write as nulls, but not ignore any other errors.

(NOTE: I have double checked my look up table. It has ALL the values that are in my source table. It just does not have NULL as a value (because it is weird to have a look up value for null.)

回答1:

I know this is a late answer, but for anyone searching on this like I was, I found this to be the simplest answer:

In the lookup connection, use a SQL query to retrieve your data and add UNION SELECT NULL, NULL to the bottom.

For example:

SELECT CarId, CarName FROM Cars 
UNION SELECT NULL, NULL

Preview will show an additional row of CarId = Null and CarName = Null that will be available in the lookpup.

回答2:

I had never noticed that line in BOL about the full cache mode. The answer to your problem is what you've already stated about having NULL in the reference set of data. A quick proof would be these two sample data flows.

In this example, I generate 4 rows of data: 1, 2, 3, and NULL (Column_isNull = true) and hit an in memory table that has all the values and perform a lookup between Column in the data flow and c1 defined in the in-memory table. It blows up as you described

I then added one more value to the lookup table, NULL and voila, the full-cache lookup is able to match NULL to NULL.

Takeaway

To make a NULL input value match in a lookup component, the reference data set must have a corresponding NULL value available as well as have the cache mode set to FULL.

回答3:

To work around this issue within SSIS, there is an alternative approach to a previous SO answer which can be applied.

Within the Lookup Transformation, you can redirect rows on error and then pass them to another destination which can simply be the same intended destination table within your database.

Therefore your destination table within the database will still receive all the rows (477 in screen shot below).

This approach therefore avoids the need to put dummy NULL values into your lookup table within your database, with the trade offs being:

• An extra step within your SSIS package.
• Error rows (non-NULL non-matches in this case) will always be loaded into the destination table. To help identify these rouge records, you can export the destination table into a txt file and then diff with the input source file to see any differences.

回答4:

You can use a Conditional Split to break the data into two sets, one where the given value is null and the rest. Only perform the lookup on the output that has non-null values and then combine the results back with the data set that contains the null values using a Union All. You can still trap for unmatched values on the lookup and not worry about having to add a null entry to your lookup table