Loading .sql files from within PHP

Question

I'm creating an installation script for an application that I'm developing and need to create databases dynamically from within PHP. I've got it to create the database but now I need to load in several .sql files. I had planned to open the file and mysql_query it a line at a time - until I looked at the schema files and realised they aren't just one query per line.

So, how do I load an sql file from within PHP (as phpMyAdmin does with its import command)?

Answer 1

Briefly, the way I have done this is:

1. Read the file (a db dump eg $ mysqldump db > db.sql)

   $sql = file_get_contents(db.sql);
   

2. Import it using mysqli::multi_query

   if ($mysqli->multi_query($sql)) {
       $mysqli->close();
   } else {
       throw new Exception ($mysqli->error);
   }
   

Watch out mysqli_query supports async queries. More here: http://php.net/manual/en/mysqli.multi-query.php and here https://stackoverflow.com/a/6652908/2002493

Answer 2

I have an environment where no mysql tool or phpmyadmin just my php application connecting to a mysql server on a different host but I need to run scripts exported by mysqldump or myadmin. To solve the problem I created a script multi_query as I mentioned here

It can process mysqldump output and phpmyadmin exports without mysql command line tool. I also made some logic to process multiple migration files based on timestamp stored in DB like Rails. I know it needs more error handling but currently does the work for me.

Check it out: https://github.com/kepes/php-migration

It's pure php and don't need any other tools. If you don't process user input with it only scripts made by developers or export tools you can use it safely.

Answer 3

Many hosts will not allow you to create your own database through PHP, but you seem to have solved that.
Once the DB has been created, you can manipulate and populate it simply:

mysql_connect("localhost");
mysql_query("SOURCE file.sql");

Answer 4

In my projects I've used next solution:

<?php

/**
 * Import SQL from file
 *
 * @param string path to sql file
 */
function sqlImport($file)
{

    $delimiter = ';';
    $file = fopen($file, 'r');
    $isFirstRow = true;
    $isMultiLineComment = false;
    $sql = '';

    while (!feof($file)) {

        $row = fgets($file);

        // remove BOM for utf-8 encoded file
        if ($isFirstRow) {
            $row = preg_replace('/^\x{EF}\x{BB}\x{BF}/', '', $row);
            $isFirstRow = false;
        }

        // 1. ignore empty string and comment row
        if (trim($row) == '' || preg_match('/^\s*(#|--\s)/sUi', $row)) {
            continue;
        }

        // 2. clear comments
        $row = trim(clearSQL($row, $isMultiLineComment));

        // 3. parse delimiter row
        if (preg_match('/^DELIMITER\s+[^ ]+/sUi', $row)) {
            $delimiter = preg_replace('/^DELIMITER\s+([^ ]+)$/sUi', '$1', $row);
            continue;
        }

        // 4. separate sql queries by delimiter
        $offset = 0;
        while (strpos($row, $delimiter, $offset) !== false) {
            $delimiterOffset = strpos($row, $delimiter, $offset);
            if (isQuoted($delimiterOffset, $row)) {
                $offset = $delimiterOffset + strlen($delimiter);
            } else {
                $sql = trim($sql . ' ' . trim(substr($row, 0, $delimiterOffset)));
                query($sql);

                $row = substr($row, $delimiterOffset + strlen($delimiter));
                $offset = 0;
                $sql = '';
            }
        }
        $sql = trim($sql . ' ' . $row);
    }
    if (strlen($sql) > 0) {
        query($row);
    }

    fclose($file);
}

/**
 * Remove comments from sql
 *
 * @param string sql
 * @param boolean is multicomment line
 * @return string
 */
function clearSQL($sql, &$isMultiComment)
{
    if ($isMultiComment) {
        if (preg_match('#\*/#sUi', $sql)) {
            $sql = preg_replace('#^.*\*/\s*#sUi', '', $sql);
            $isMultiComment = false;
        } else {
            $sql = '';
        }
        if(trim($sql) == ''){
            return $sql;
        }
    }

    $offset = 0;
    while (preg_match('{--\s|#|/\*[^!]}sUi', $sql, $matched, PREG_OFFSET_CAPTURE, $offset)) {
        list($comment, $foundOn) = $matched[0];
        if (isQuoted($foundOn, $sql)) {
            $offset = $foundOn + strlen($comment);
        } else {
            if (substr($comment, 0, 2) == '/*') {
                $closedOn = strpos($sql, '*/', $foundOn);
                if ($closedOn !== false) {
                    $sql = substr($sql, 0, $foundOn) . substr($sql, $closedOn + 2);
                } else {
                    $sql = substr($sql, 0, $foundOn);
                    $isMultiComment = true;
                }
            } else {
                $sql = substr($sql, 0, $foundOn);
                break;
            }
        }
    }
    return $sql;
}

/**
 * Check if "offset" position is quoted
 *
 * @param int $offset
 * @param string $text
 * @return boolean
 */
function isQuoted($offset, $text)
{
    if ($offset > strlen($text))
        $offset = strlen($text);

    $isQuoted = false;
    for ($i = 0; $i < $offset; $i++) {
        if ($text[$i] == "'")
            $isQuoted = !$isQuoted;
        if ($text[$i] == "\\" && $isQuoted)
            $i++;
    }
    return $isQuoted;
}

function query($sql)
{
    global $mysqli;
    //echo '#<strong>SQL CODE TO RUN:</strong><br>' . htmlspecialchars($sql) . ';<br><br>';
    if (!$query = $mysqli->query($sql)) {
        throw new Exception("Cannot execute request to the database {$sql}: " . $mysqli->error);
    }
}

set_time_limit(0);

$mysqli = new mysqli('localhost', 'root', '', 'test');
$mysqli->set_charset("utf8");

header('Content-Type: text/html;charset=utf-8');
sqlImport('import.sql');

echo "Peak MB: ", memory_get_peak_usage(true)/1024/1024;

On test sql file (41Mb) memory peak usage: 3.25Mb

Answer 5

Are you sure that its not one query per line? Your text editor may be wrapping lines, but in reality each query may be on a single line.

At any rate, olle's method seems best. If you have reasons to run queries one at time, you should be able to read in your file line by line, then use the semicolon at the end of each query to delimit. You're much better off reading in a file line by line than trying to split an enormous string, as it will be much kinder to your server's memory. Example:

$query  = '';
$handle = @fopen("/sqlfile.sql", "r");

if ($handle) {
    while (!feof($handle)) {
        $query.= fgets($handle, 4096);

        if (substr(rtrim($query), -1) == ';') {
            // ...run your query, then unset the string
            $query = '';
        }
    }

    fclose($handle);
}

Obviously, you'll need to consider transactions and the rest if you're running a whole lot of queries in a batch, but it's probably not a big deal for a new-install script.

Answer 6

This may be helpful -->

More or less what it does is to first take the string given to the function (the file_get_contents() value of your file.sql) and remove all the line breaks. Then it splits the data by the ";" character. Next it goes into a while loop, looking at each line of the array that is created. If the line contains the " ` " character, it will know it is a query and execture the myquery() function for the given line data.

Code:

function myquery($query) {

mysql_connect(dbhost, dbuser, dbpass);

mysql_select_db(dbname);

$result = mysql_query($query);

if (!mysql_errno() && @mysql_num_rows($result) > 0) {
}

else {

$result="not";
}
mysql_close();

return $result;

}



function mybatchquery ($str) {

$sql = str_replace("\n","",$str)

$sql = explode(";",$str);

$x=0;

while (isset($str[$x])) {

if (preg_match("/(\w|\W)+`(\w|\W)+) {

myquery($str[$x]);

}

$x++

}

return TRUE;

}




function myrows($result) {

$rows = @mysql_num_rows($result);

return $rows;
}




function myarray($result) {

$array = mysql_fetch_array($result);

return $array;
}




function myescape($query) {

$escape = mysql_escape_string($query);

return $escape;
}



$str = file_get_contents("foo.sql");
mybatchquery($str);