Import CSV to mysql table

Question

What is the best/fastest way to upload a csv file into a mysql table? I would like for the first row of data be used as the column names.

Found this:

How to import CSV file to MySQL table

But the only answer was to use a GUI and not shell?

Answer 1

I wrote some code to do this, i'll put in a few snippets:

$dir = getcwd(); // Get current working directory where this .php script lives
$fileList = scandir($dir); // scan the directory where this .php lives and make array of file names

Then get the CSV headers so you can tell mysql how to import (note: make sure your mysql columns exactly match the csv columns):

//extract headers from .csv for use in import command
$headers = str_replace("\"", "`", array_shift(file($path)));
$headers = str_replace("\n", "", $headers);

Then send your query to the mysql server:

mysqli_query($cons, '
        LOAD DATA LOCAL INFILE "'.$path.'"
            INTO TABLE '.$dbTable.'  
            FIELDS TERMINATED by \',\' ENCLOSED BY \'"\'
            LINES TERMINATED BY \'\n\'
            IGNORE 1 LINES
            ('.$headers.')
            ;
        ')or die(mysql_error());

Answer 2

Here's how I did it in Python using csv and the MySQL Connector:

import csv
import mysql.connector

credentials = dict(user='...', password='...', database='...', host='...')
connection = mysql.connector.connect(**credentials)
cursor = connection.cursor(prepared=True)
stream = open('filename.csv', 'rb')
csv_file = csv.DictReader(stream, skipinitialspace=True)

query = 'CREATE TABLE t ('
query += ','.join('`{}` VARCHAR(255)'.format(column) for column in csv_file.fieldnames)
query += ')'
cursor.execute(query)
for row in csv_file:
    query = 'INSERT INTO t SET '
    query += ','.join('`{}` = ?'.format(column) for column in row.keys())
    cursor.execute(query, row.values())

stream.close()
cursor.close()
connection.close()

Key points

• Use prepared statements for the INSERT
• Open the file.csv in 'rb' binary
• Some CSV files may need tweaking, such as the skipinitialspace option.
• If 255 isn't wide enough you'll get errors on INSERT and have to start over.
• Adjust column types, e.g. ALTER TABLE t MODIFY `Amount` DECIMAL(11,2);
• Add a primary key, e.g. ALTER TABLE t ADD `id` INT PRIMARY KEY AUTO_INCREMENT;

Answer 3

if you have the ability to install phpadmin there is a import section where you can import csv files to your database there is even a checkbox to set the header to the first line of the file contains the table column names (if this is unchecked, the first line will become part of the data

Answer 4

First create a table in the database with same numbers of columns that are in the csv file.

Then use following query

LOAD DATA INFILE 'D:/Projects/testImport.csv' INTO TABLE cardinfo
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'

Answer 5

If you start mysql as "mysql -u -p --local-infile ", it will work fine

Answer 6

I have google search many ways to import csv to mysql, include " load data infile ", use mysql workbench, etc.

when I use mysql workbench import button, first you need to create the empty table on your own, set each column type on your own. Note: you have to add ID column at the end as primary key and not null and auto_increment, otherwise, the import button will not visible at later. However, when I start load CSV file, nothing loaded, seems like a bug. I give up.

Lucky, the best easy way so far I found is to use Oracle's mysql for excel. you can download it from here mysql for excel

This is what you are going to do: open csv file in excel, at Data tab, find mysql for excel button

select all data, click export to mysql. Note to set a ID column as primary key.

when finished, go to mysql workbench to alter the table, such as currency type should be decimal(19,4) for large amount decimal(10,2) for regular use. other field type may be set to varchar(255).