Working with huge files in VIM

Question

I tried opening a huge (~2GB) file in VIM but it choked. I don't actually need to edit the file, just jump around efficiently.

How can I go about working with very large files in VIM?

Answer 1

I had the same problem, but it was a 300GB mysql dump and I wanted to get rid of the DROP and change CREATE TABLE to CREATE TABLE IF NOT EXISTS so didn't want to run two invocations of sed. I wrote this quick Ruby script to dupe the file with those changes:

#!/usr/bin/env ruby

matchers={
    %q/^CREATE TABLE `foo`/ => %q/CREATE TABLE IF NOT EXISTS `foo`/,
    %q/^DROP TABLE IF EXISTS `foo`;.*$/ => "-- DROP TABLE IF EXISTS `foo`;"
}

matchers.each_pair { |m,r|
    STDERR.puts "%s: %s" % [ m, r ]
}

STDIN.each { |line|
    #STDERR.puts "line=#{line}"
    line.chomp!
    unless matchers.length == 0
        matchers.each_pair { |m,r|
            re=/#{m}/
            next if line[re].nil?
            line.sub!(re,r)
            STDERR.puts "Matched: #{m} -> #{r}"
            matchers.delete(m)
            break
        }
    end
    puts line
}

Invoked like

./mreplace.rb < foo.sql > foo_two.sql

Answer 2

It's already late but if you just want to navigate through the file without editing it, cat can do the job too.

% cat filename | less

or alternatively simple:

% less filename

Answer 3

Old thread. But nevertheless( pun :) ).

 $less filename

less works efficiently if you don't want to edit and just look around which is the case for examining huge log files.

Search in less works like vi

Best part, it's available by default on most distros. So won't be problem for production environment as well.

Answer 4

I wrote a little script based on Florian's answer that uses nano (my favorite editor):

#!/bin/sh

if [ "$#" -ne 3 ]; then
  echo "Usage: $0 hugeFilePath startLine endLine" >&2
  exit 1
fi

sed -n -e $2','$3'p' -e $3'q' $1 > hfnano_temporary_file
nano hfnano_temporary_file
(head -n `expr $2 - 1` $1; cat hfnano_temporary_file; sed -e '1,'$3'd' $1) > hfnano_temporary_file2
cat hfnano_temporary_file2 > $1
rm hfnano_temporary_file hfnano_temporary_file2

Use it like this:

sh hfnano yourHugeFile 3 8

In that example, nano will open up lines 3 through 8, you can edit them, and when you save and quit, those lines in the hugefile will automatically be overwritten with your saved lines.