Importing a txt file into SQL Server with a where

Question

I have a .txt file which is 6.00 GB. It is a tab-delimited file so when I try to load it into SQL Server, the column delimiter is tab.

I need to load that .txt file into the database, but I don't need all the rows from the 6.00 Gb file. I need to be able to use a condition like

select * 
into <my table> 
where column5 in ('ab, 'cd')

but this is a text file and am not able to load it into db with that condition.

Can anyone help me with this?

Answer 1

Just do a Bulk Insert into a staging table and form there move the data you actually want into a production table. The Where Clause is for doing something based on a specific condition inside SQL Server, not for loading data into SQL Server.

Answer 2

Have you tried with BULK INSERT command? Take a look at this solution:

--Create temporary table
CREATE TABLE #BulkTemporary
(
  Id int,
  Value varchar(10)
)

--BULK INSERT has no WHERE clause
BULK INSERT #BulkTemporary FROM 'D:\Temp\File.txt'
WITH (FIELDTERMINATOR = '\t', ROWTERMINATOR = '\n')

--Filter results
SELECT * INTO MyTable FROM #BulkTemporary WHERE Value IN ('Row2', 'Row3')

--Drop temporary table
DROP TABLE #BulkTemporary

Hope this helps.