How to load flat file with dynamic columns with di

Question

I want to load a flat file into oracle Database. This flat file may be generated from table A or table B or table C.

So when I am loading this file into oracle table , I am exactly not sure how many columns and what data type is column has that flat file has ( depends on whether it is generated from table A or table B or table C ).

So, pls let me know the generic method, technique to load variable column length file into oracle database.

Example:

• Table A

  1 | 2 | 3 | 4

• Table B

  1 | XYZ | 3 | 4 | 5 | XXX

• Table 3

  xxx | 2013-09-28 | 10.0

So, here each table has variable columns and different datatype. How can I load these file into oracle database.

Thanks in advance.

Answer 1

One option is to use SQLLoader to load files into tables.

Say we have created three tables:

CREATE TABLE tableA(
  col1 int, col2 int, col3 int, col4 int 
);

CREATE TABLE tableB(
  col1 int, col2 varchar2(100), col3 int, col4 int, col5 int, col6 varchar2(100)
);

CREATE TABLE tableC(
  col1 varchar2(100), col2 date, col3 number(10,2)
);

I am assuming that the file has always records in one format only (one of 3 possible formats).
In such a case, you can create 3 different control files for each format:

format_a.ctl

load data
 infile 'c:\tmp\test\file.txt'
 into table tableA
 fields terminated by "|"         
 ( col1, col2, col3, col4 )

format_b.ctl

load data
 infile 'c:\tmp\test\file.txt'
 into table tableB
 fields terminated by "|"         
 ( col1, col2, col3, col4, col5, col6 )

format_c.ctl

infile 'c:\tmp\test\file.txt'
 into table tableC
 fields terminated by "|"         
 ( col1 , 
   col2 date 'yyyy-mm-dd',
   col3 )

Then create a simple script that detects a format of the file and uploads data using an appropriate control file - this is an example for Windows environment:

@echo off
set filename=file.txt
IF NOT EXIST   %filename%  GOTO error

findstr /M "\|.*\|.*\|.*\|.*\|" file.txt
IF NOT ERRORLEVEL 1 GOTO formatB

findstr /M "\|.*\|.*\|" file.txt
IF NOT ERRORLEVEL 1 GOTO formatA

findstr /M "\|.*\|" file.txt
IF NOT ERRORLEVEL 1 GOTO formatC

:error
Echo Error: file %filename% doesn't exist or doesn't match any proper format
goto end

:formatA
set ctl_file=format_a
goto import

:formatB
set ctl_file=format_b
goto import

:formatc
set ctl_file=format_c
goto import

:import
echo Import using: %ctl_file%
sqlldr test/test@//192.168.2.51:1521/orcl control=%ctl_file%.ctl log=%ctl_file%.log
:end

In this line:

sqlldr test/test@//192.168.2.51:1521/orcl control=%ctl_file%.ctl log=%ctl_file%.log

test/test@ is a database user test having password test