Laravel - seeding large SQL file

Question

A memory exhaustion happens when I run my DB seed script in production.

Below is my seed script.

class MembershipTableSeeder extends Seeder 
{
    public function run()
    {
        DB::table('members')->delete();

        foreach (range(1, 99) as $days){
            Members::create(array('membership_code' => 'test'.$days));
        }

        DB::unprepared(file_get_contents(app_path()."/database/seeds/members.sql"));
    }
}

So what I did was add a no-limit on my seed script.

ini_set('memory_limit', '-1');

The problem now is that when I run the script it logs the output into the terminal the content of the SQL script (which is very, very big).

Is there a good way of running a SQL dump inside my DB seeds that doesn't consume much memory? What I did now was run it manually:

mysql -uuser -p db < script.sql

Answer 1

The problem happens because when using Db::unprepared it also logs the query to the laravel.log file, making in background much more actions then you think, from this side you have memory exhaust. If you are not running the safe mode I would stick to executing the console command like this:

exec("mysql -u ".\Config::get('database.mysql.user')." -p".\Config::get('database.mysql.password')." ".\Config::get('database.mysql.database')." < script.sql")

Answer 2

For others who prefer a more Laravel-ish solution, this is how I handled it:

/**
 * This class is responsible for running the data dump sql.
 * It is recommended to update this class instead of creating new ones for new database content dumps.
 */
class DatabaseDumpSeeder extends Seeder
{
    /**
     * Run the database seeds.
     * @throws \Exception
     */
    public function run()
    {
        // Note: these dump files must be generated with DELETE (or TRUNCATE) + INSERT statements
        $sql = file_get_contents(__DIR__ . '/dumps/dump-20150709.sql');

        if (! str_contains($sql, ['DELETE', 'TRUNCATE'])) {
            throw new Exception('Invalid sql file. This will not empty the tables first.');
        }

        // split the statements, so DB::statement can execute them.
        $statements = array_filter(array_map('trim', explode(';', $sql)));

        foreach ($statements as $stmt) {
            DB::statement($stmt);
        }
    }
}

Answer 3

Create Seeder File "PostalCodeTableSeeder.php" in Project_directory/database/seeds

use Illuminate\Database\Seeder;

class PostalCodeTableSeeder extends Seeder {
    /**
     * Run the database seeds.
     *
     * @return void
     */
    public function run()
    {
        // =============================================================
        // file Path -> Project/app/configs/database.php
        // get the database name, database username, database password
        // =============================================================
        $db     = \Config::get('database.connections.mysql.database');
        $user   = \Config::get('database.connections.mysql.username');
        $pass   = \Config::get('database.connections.mysql.password');

        // $this->command->info($db);
        // $this->command->info($user);
        // $this->command->info($pass);

        // running command line import in php code
        exec("mysql -u " . $user . " -p" . $pass . " " . $db . " < postal_codes.sql");
        // postal_codes.sql is inside root folder
    }
}

Also add the class name into Project_directory/database/seed/DatabaseSeeder.php like code below

use Illuminate\Database\Seeder;

class DatabaseSeeder extends Seeder
{
    /**
     * Run the database seeds.
     *
     * @return void
     */
    public function run()
    {
        $this->call(PostalCodeTableSeeder::class);
        // $this->call(UsersTableSeeder::class);
    }
}