How to edit multi-gigabyte text files? Vim doesn&#

Question

Closed. This question is off-topic. It is not currently accepting answers.

---

Want to improve this question? Update the question so it's on-topic for Stack Overflow.

Closed 5 years ago.

Are there any editors that can edit multi-gigabyte text files, perhaps by only loading small portions into memory at once? It doesn't seem like Vim can handle it =(

Answer 1

It may be plugins that are causing it to choke. (syntax highlighting, folds etc.)

You can run vim without plugins.

vim -u "NONE" hugefile.log

It's minimalist but it will at least give you the vi motions you are used to.

syntax off

is another obvious one. Prune your install down and source what you need. You'll find out what it's capable of and if you need to accomplish a task via other means.

Answer 2

I've tried to do that, mostly with files around 1 GB when I needed to make some small change to an SQL dump. I'm on Windows, which makes it a major pain. It's seriously difficult.

The obvious question is "why do you need to?" I can tell you from experience having to try this more than once, you probably really want to try to find another way.

So how do you do it? There are a few ways I've done it. Sometimes I can get vim or nano to open the file, and I can use them. That's a really tough pain, but it works.

When that doesn't work (as in your case) you only have a few options. You can write a little program to make the changes you need (for example, search & replaces). You could use a command line program that may be able to do it (maybe it could be accomplished with sed/awk/grep/etc?)

If those don't work, you can always split the file into chunks (something like split being the obvious choice, but you could use head/tail to get the part you want) and then edit the part(s) that need it, and recombine later.

Trust me though, try to find another way.

Answer 3

You might want to check out this VIM plugin which disables certain vim features in the interest of speed when loading large files.

Answer 4

I'm using vim 7.3.3 on Win7 x64 with the LargeFile plugin by Charles Campbell to handle multi-gigabyte plain text files. It works really well.

I hope you come right.

Answer 5

A slight improvement on the answer given by @Al pachio with the split + vim solution you can read the files in with a glob, effectively using file chunks as a buffer e.g

$ split -l 5000 myBigFile
xaa
xab
xac
...

$ vim xa*
#edit the files

:nw  #skip forward and write
:n!  #skip forward and don't save 

:Nw  #skip back and write
:N!  #skip back and don't save

Answer 6

I think it is reasonably common for hex editors to handle huge files. On Windows, I use HxD, which claims to handle files up to 8 EB (8 billion gigabytes).