Stack OverflowText editor to open big (giant, huge, large) text files
[+1022] [2] Yuvi
[2008-10-01 20:22:35]
[ windows xml editor text-editor large-files ]
[ ]

I mean 100+ MB big; such text files can push the envelope of editors.

I need to look through a large XML file, but cannot if the editor is buggy.

Any suggestions?

(166) Actually, text files of 100+ MB or even 1+ GB is not as uncommon as you may think (i.e. log files from busy servers). - Anders Sandvig
(15) Sneakyness: And not exactly text. I think the requirements of reading text files and reading binary files differ somewhat. You might pass it through base64 or uuencode, though. - Joey
Back in 1995 I used WinWord to open 64MB files on a 16MB machine. I'm sure it would do just as well 15 years later. - Gabe
To generate random text files instead of binaries, use this: cat /dev/urandom | tr -dc 'A-z' | head -c 1000000, where the last number after -c is the number of bytes in the file. - indiv
Microsoft Office Access can actually read and parse very large xml files, but will only make sense of it if the xml format fits with something that it can translate to a table. - Niels Brinch
if using vim :set binary… - daydreamer
(2) This should be at least a similar question or even linked as it was asked 18 months prior...… - ONDEV
(1) I was also looking for the answer to this exact question in order to read some huge log files that I've generated! - HorseloverFat
Here's my fallback: GigaEdit ( Nothing fancy, but small, portable, free and opens massive files in an instant. - Carl
(1) @BlairHippo I feel the same way, I'm almost nervous when asking a question because chances are high that someone will say "Close this, it should go in WhateverExchange instead" - Rodolfo
@Sneakyness this can be used to generate large files in a matter of seconds as well. grep -r "someText" . > bigfile assuming that there are some files in your dir that contain matching lines with the search criteria. Of course, you would need to stop grep forcefully as this it will make it enter in a endless loop :) - user624558 its so simple to deal with large xml in ruby - Amol Pujari
To view files, I recommend to use this online viewer - - you don't have to install any programming interface, it works in every device and OS. - Ivan Kuckir
On a windows machine with powershell > Get-Content C:\Scripts\Test.txt -totalcount 3 - dhysong Free and very fast - Bernhard
You can try this online - Jenson M John
First ask yourself this: do you actually want to edit a file >1GB in size, or do you just want to view it quickly, and be able to edit other, "normal" files? In the latter case you'll have a much better choice of log viewers and text editors. - rustyx
[+1651] [2008-10-01 20:25:19] Nick Ryberg [ACCEPTED]

Free read-only viewers:

  • Large Text File Viewer [1] (Windows) – Fully customizable theming (colors, fonts, word wrap, tab size). Supports horizontal and vertical split view. Also support file following and regex search. Very fast, simple, and has small executable size.
  • klogg [2] (Windows, macOS, Linux) – A maintained fork of glogg [3]. Its main feature is regular expression search. It supports monitoring file changes (like tail), bookmarks, highlighting patterns using different colors, and has serious optimizations built in. But from a UI standpoint, it's rather minimal.
  • LogExpert [4] (Windows) – "A GUI replacement for tail." It's really a log file analyzer, not a large file viewer, and in one test it required 10 seconds and 700 MB of RAM to load a 250 MB file. But its killer features are the columnizer (parse logs that are in CSV, JSONL, etc. and display in a spreadsheet format) and the highlighter (show lines with certain words in certain colors). Also supports file following, tabs, multifiles, bookmarks, search, plugins, and external tools.
  • Lister [5] (Windows) – Very small and minimalist. It's one executable, barely 500 KB, but it still supports searching (with regexes), printing, a hex editor mode, and settings.

Free editors:

  • Your regular editor or IDE. Modern editors can handle surprisingly large files. In particular, Vim [6] (Windows, macOS, Linux), Emacs [7] (Windows, macOS, Linux), Notepad++ [8] (Windows), Sublime Text [9] (Windows, macOS, Linux), and VS Code [10] (Windows, macOS, Linux) support large (~4 GB) files, assuming you have the RAM.
  • Large File Editor [11] (Windows) – Opens and edits TB+ files, supports Unicode, uses little memory, has XML-specific features, and includes a binary mode.
  • GigaEdit [12] (Windows) – Supports searching, character statistics, and font customization. But it's buggy – with large files, it only allows overwriting characters, not inserting them; it doesn't respect LF as a line terminator, only CRLF; and it's slow.

Builtin programs (no installation required):

  • less [13] (macOS, Linux) – The traditional Unix command-line pager tool. Lets you view text files of practically any size. Can be installed on Windows, too.
  • Notepad [14] (Windows) – Decent with large files, especially with word wrap turned off.
  • MORE [15] (Windows) – This refers to the Windows MORE, not the Unix more. A console program that allows you to view a file, one screen at a time.

Web viewers:

Paid editors/viewers:

  • 010 Editor [17] (Windows, macOS, Linux) – Opens giant (as large as 50 GB) files.
  • SlickEdit [18] (Windows, macOS, Linux) – Opens large files.
  • UltraEdit [19] (Windows, macOS, Linux) – Opens files of more than 6 GB, but the configuration must be changed for this to be practical: Menu » Advanced » Configuration » File Handling » Temporary Files » Open file without temp file...
  • EmEditor [20] (Windows) – Handles very large text files nicely (officially up to 16 TB). The speed of search and replace is very fast. Free version available for personal use.
  • BssEditor [21] (Windows) – Handles large files and very long lines. Don’t require an installation. Free for non commercial use.
  • loxx [22] (Windows) – Supports file following, highlighting, line numbers, huge files, regex, multiple files and views, and much more. The free version can not: process regex, filter files, synchronize timestamps, and save changed files.

(63) VIM, or Emacs... pick your poison, both will handle any file you throw at them. I personally prefer Emacs, but both will beat notepad without so much as a hiccup. - Mike Stone
(28) Emacs has a maximum buffer size, dependent on the underlying architecture (32 or 64 bits). I think that on 32 bit systems you get "maximum buffer size exceeded" error on files larger than 128 MB. - Rafał Dowgird
(108) I just tried Notepad++ with a 561MB log file and it said it was too big - barfoon
I regularly open ~600mb files with gVIM ... - Nippysaurus
(1) I've been asked in the past to edit a couple of plain text files in the multi-GB range, which our users tried to edit with MS Word... well, most of you will know what happened. Just opened it in vim and searched and replaced with the user sitting next to me in a matter of seconds (after that huge file was finally read in of course). - boxofrats
(11) @Rafal Interesting! Looks like on 64bit it is ~1024 petabytes. The reason has to do with the fact that emacs has to track buffer positions (such as the point) - baudtack
(1) I'll second gVim for huge files. I just edited a 950 MB text file with no problems (but it took a while to open and save). When I tried that same file in Notepad2, Windows became concerned with the size of my pagefile, and started resizing it. - Christian Davén
(88) But be careful, vim will only work as long as the files in question have enough line breaks. I once had to edit a ca. 150 MB file without any line breaks, and had to resort to gedit because vim couldnt handle it. - Benno
(5) If you are going to use (g)vim then to improve performance you may want to turn off some features such as syntax highlighting, swapfile and undo. See, and - Dave Kirby
I wonder if 5GB text files exist .. :-O .. if you don't mind, may I know .. (in practical world) where we are forced to use/edit these bulky text files.. (alternate way would have been to break the file and make a few of it.. usually larger files, of any file-type, make system cry to give performance) - Rookie Programmer Aravind
@Rafal: emacs buffer size can be boosted with emacs 23. I don't recall offhand how to do it. - Paul Nathan
(8) I tried all, gVim sucked in that it didnt even tell you it was loading a file - took forever to load the file (only 200k, 5 million lines). SlickEdit opened the entire file in about 3 seconds. Getting the trial license was a PIA tho. Thank you for listing these. - schmoopy
Emacs definitely has a buffer size problem on 32-bit. - Joseph Garvin
(2) I want an editor that mmap()s the file and reads only the parts I am looking at... even gvim seems to load the whole thing into memory first, and even resizing the window freeezes it while it thinks... - Joe Koberg
(1) I had a problem with a single character encoded by 6MB of code. Notepad++ and Netbeans couldn't handle it, but the 010 Editor did it easily! ;D - Jayme Tosi Neto
gVim takes forever to load a 2GB file, and then seeking within it is painfully slow too. It seems to load the entire file into RAM. Maybe the large file plugin would help but the default doesn't seem optimal. - rogerdpack
(3) Wasn't able to get any of these to work on a 1.6GB file, especially gVim or any other Vim for windows that I could find. Had to use filesplitter to break it into 100MB chunks then I used EditPlus to view them. The old (DOS) can handle large files (a few hundred MB) but is not available in 64bit windows. - Dustin Davis
(3) 010Editor successfully opened my 3.3GB MySQL database dump. - Zane
Does 010 Editor has XML formatting option? - FIre Panda
The 32-bit version of Vim 7.3 crashes at about 2G if the swap is on and just gives errors about an incorrect line count if swap is off. The 64-bit version however works ok; I'm currently "browsing" a 7.5G file and I can't resize the window with the mouse, seeking is a bit slow, but it works. (81M lines, a log of all memory allocations in an application.) - dash-tom-bang
(1) @docgnome Aw, crap. What about my 1025 petabyte file? Ugh. I guess Emacs is so stuck in the past that it can't edit it. What does it think the date is, 1997? - JavaAndCSharp
(3) Used a combination of Large Text Viewer and HxD. LTV for nice text view and line seeking, and HxD for actual editing + search and replace. - Pixel Elephant
I had a 2+ gigabyte sql script that I tried to open with gVim. It folded like a pair of 2's, then sat down and cried. Then I tried 010Editor, which shined like a CHAMP. Scrap gvim, use 010. - HerrimanCoder
(3) I use Hxd to open large files. It works very will for me and has many other features. I can open and veiw my entire hard drive with it( ~200 GB) in less than a second. Scrolling through files and editing them is very smooth as well. - user2097804
(3) "But be careful, vim will only work as long as the files in question have enough line breaks." @Benno, That's a configuration setting in vim, not a limitation. You can change it this way: :set display+=lastline. It is kind of weird to not have that as the default, though. - Aaron R.
If you have been using LargeTextFileViewer, switch to glogg! LTF is smaller, but glogg seems to work much better. - Martin Ba
(1) If you only want to view the file contents, then I suggest a very nice tool online: it works on all modern browsers. - dev4life
(2) Large Text File Viewer was able to open a 22GB file without any issues. - Aravind Yarram
Is there a way to see the file with the correct line breaks in HxD?? I see weird "..." as separator and not even at the end of the line.thanks - Millemila
(2) Using Large Text File Viewer, I just opened a 30gb file with no issue. Finding a specific keyword took about 5 minutes though. - PaulBGD
gVim would open my 1.6 GB file but I couldn't really do much with it... I understand it's a big task, but I needed to do a find and replace about 150 million instances. gVim would only make it about half way. - Mike
(2) Emeditor opens fast "up to a 248 GB limit (or 2.1 billion lines)"… For csv files there is also Delimit ("Open data files up to 2 billion rows and 2 million columns large!") - miodf
When using gVim, use this plugin to automatically disable slow features on large files: - Denilson Sá Maia
gVim choked on an 8GB file and 'stopped working'. - BLoB
I couldn't get 010 editor to search across a large 20G text file... it just choked and froze until I killed it - Earlz
See LargeFile plugin to Vim. - Hans Ginzel
You can browse, edit and search/replace in text files of almost any size with XML ValidatorBuddy. In addition you get syntax-coloring for XML documents. The editor lets you select the encoding too. - Clemens
LogExpert fails on long lines. If a line is ~8000 chars, it will get cut off and the following line will also be cut off, much shorter. HxD worked fine. - Dan
I used 010 Editor to open a 42.7 Gb file (yes, Wikipedia XML file 27). It worked (and the file has 634,957,038 lines)! I'm using Windows 10 64-bit on a laptop with 16 Gb RAM, and v6.0.3 of 010 Editor. I could also search for a string that was at the end of the file (although 010 took almost 10 minutes to finish the search). - Andrew Jens
(1) As @user2070775 suggested, is fantastic for a quick view. - bmende
(1) Large Text View Viewer still exists here. - ruffin
(1) I use myself Editpad lite (free for non commercial use) which can edit files larger than 4 GB, even if your PC only has a few GB of RAM. Also, the maximum length of a single line is not limited, which is a problem with many editors claiming to support "unlimited" file sizes. - Eric Bole-Feysot
(1) windows cmd more is ideal for looking and confirming file structure. - clg4
Sublime Text 3 works beautifully for large files; I don't have the rep to add it as answer but just tested with a 1.6G log file. It even shows a loading bar while reading the file, feels fluid to use unlike the others mentioned here. Its free and also has a portable version! - Pavin Joseph
(1) I don't know why I can't reply to the OP. The best options in Windows are: EmEditor, Delimit, Editpad Pro and Texpad, all commercial and all can read and write and do many things with files much bigger than memory. Emeditor and Delimit offer the option to see csv files as a spreadsheet. I've being having problems with EmEditor, it keeps reloading the file every few seconds preventing you from working.You can also try SlickEditor and HippoEdit. - skan
010 Editor was perfect for my situation where I had to edit a 8GB mysql dump that failed and I had to resume, I also removed some GBs from log tables, this saved me a lot of time! - Ultranuke
(1) Also total commander's "view file" menu option (F3) is very good at this. What it does, I think, is virtual scrolling; not loading all the content. - myuce
LargeTextFileViewer worked for me for 2 GB of text. Emurasoft's EmEditor claims to open up to 248GB, might be worth a look if other solutions did not work for you. - gaborous
(1) LiquidSoft has a horrible installer which left me with a non-closable dialog. - Rüdiger Schulz
I have tried logexpert and worked well for 1GB text file. The others were not free or didnt worked for me - Rodolfo Velasco
Liquid Large File Editor does great job and is free (Community Edition): - Ivan Akcheurov
I tried LogExpert 1.5.5493. It always crash after opening a large txt file for less than 2 mins. The txt file is about 2.5G. - smwikipedia
(1) Mixed bag experience with 010Editor on a 19GB JSON trace file. Takes several seconds to "scan for linefeeds", before showing the file. Crashed doing searches, and seems to look for all matches before moving to the first "closest" match (since took a while to show a match, with a full list of matches at that time). Otherwise still allowed me to examine my super-large file, so not a complete waste. FWIW. - ddevienne
(1) Note that Liquid Studio requires you to sign up even for the "free" Community Edition (e-mail, name etc.) which is a bit dodgy. LogExpert tries to load the entire file into memory, so with a 30 GB file and 12 GB RAM it didn't work for me. - Sina Madani
Liquid Studio text search speed is mere 23 MB/s on my computer! Ridiculous. In comparison EmEditor speed is 312 MB/s and is limited by CPU speed. I have a SSD. Benchmarked on 06/19/2018. - Desik
Update: glogg has search performance in between Liquid Studio and EmEditor. EmEditor is still the fastest. - Desik
On 64-bit Windows, the standard Notepad.exe handled opening and searching in a 450 MB csv file perfectly while Sublime Text 3 hung forever and VS Code said it's too large. - Violet Giraffe
(1) Glogg is fantastic. I can easily spot trends and filter down to a log token. Thanks for the recommendation. - Shane
(4) I think the VS Code is a great option. Free and Open Source. Just added to the list. - Faheem
Sublime Text loaded 1.4 Gb .sql file for me & I can edit this file and save it. On saving, it takes a little bit time but you shouldn't click anywhere as it could crash the sublime text and stop the process. - Zeeshan Ahmad Khalil
(3) VSCode no longer opens large text files. - Kelly Elton
With a ~240MB One-Liner XML File VIM, Emacs and Notepad++ don't work. LogExpert somewhat works and with VS Code, you can't click into the text or you get a Out of Memory Exception. Liquid Studio (mentioned as Large File Editor) works but the special features are turned off. - JP-Hundhausen
You wrote Lister support regex search - how did you manage to do regex search with Lister? (Surprisingly I've found it the best to handle large files but I didn't find how to do regex search with it) - BornToCode
Author of Klogg here. Klogg has switched to hyperscan regular expressions library in current dev builds. In our tests searching in big files is almost 2 times faster now. Any feedback is welcome - fav
If emacs performs poorly with a large file try M-x fundamental-mode to switch off highlighting. - Bruce Adams
I don't open enormous text files on a daily basis, but I have used Lister a couple of times now and I found it very easy to use, stable, and performant. No need to install it even, it just works; for every few times in a decade I need this kind of tool. So I cast my vote on Lister for Windows users. If you plan on doing this sort of thing on a regular basis though, I highly recommend that you switch to a more suitable OS to begin with, and when you do that you suddenly open the flood door to endless possibilities. - Samir
(1) @BornToCode To enable RegEx in Lister, you simply click to check the "RegEx" option in the search box, type in your pattern and click OK to start searching. Using only the keyboard you can press Ctrl+F, Alt+X, Tab, Tab, Tab, Tab, type in your pattern and press Enter to start searching. (You have to tab back the focus to search input box after enabling RegEx.) Try something simple like searching for . (dot) with RegEx disabled vs. RegEx enabled to see the difference. Press F3 to continue to the next match. - Samir
GigaEdit's link is dead, could someone update it? I didn't find the new homepage. - Basj
(1) Update 2022: EmEditor(Windows) crashed the system on opening a 17 GB file. Works well for anything less than 5 GB. Wouldn't recommend it for files larger than 5 GB. - abhij89
editpadLite, notepad++ both fail to open sql dump 5.1G - teran
The windows version is not working! just a bunch of weird photos! - najmieh s
I needed to insert two short lines before a single line of around 21 billion characters. Tried most editors, the only suitable one seems to be BssEditor (had to safe it to a new file, though). Most editors wouldn't even load the file. Some acted like they inserted the lines, but they didn't actually insert the NL char, so they only increased the huge single line. UltraEdit went read-only mode, when switching to write mode, all edits got reverted. - xamid
Just opened a 15 GB file in VIM. It took a moment to load into memory, but it was fast and easy to search and edit, or just scroll down once it was open. - Altimus Prime
@xamid, You don't need a text editor for that. Just concatenate your text with the file in the command line and output to a new file so as not to overwrite you original. - Altimus Prime
LogExpert works like a charm! Displays progress of loading (in footer). Perfectly follows the newest lines at the end (the follow tail option). - Bronek
[+216] [2009-08-27 20:25:59] Roboprog

Tips and tricks


Why are you using editors to just look at a (large) file?

Under *nix or Cygwin [1], just use less [2]. (There is a famous saying – "less is more, more or less" – because "less" replaced the earlier Unix command "more", with the addition that you could scroll back up.) Searching and navigating under less is very similar to Vim, but there is no swap file and little RAM used.

There is a Win32 port of GNU less. See the "less" section of the answer above.


Perl is good for quick scripts, and its .. (range flip-flop) operator makes for a nice selection mechanism to limit the crud you have to wade through.

For example:

$ perl -n -e 'print if ( 1000000 .. 2000000)' humongo.txt | less

This will extract everything from line 1 million to line 2 million, and allow you to sift the output manually in less.

Another example:

$ perl -n -e 'print if ( /regex one/ .. /regex two/)' humongo.txt | less

This starts printing when the "regular expression one" finds something, and stops when the "regular expression two" find the end of an interesting block. It may find multiple blocks. Sift the output...


This is another useful tool you can use. To quote the Wikipedia article [3]:

logparser is a flexible command line utility that was initially written by Gabriele Giuseppini, a Microsoft employee, to automate tests for IIS logging. It was intended for use with the Windows operating system, and was included with the IIS 6.0 Resource Kit Tools. The default behavior of logparser works like a "data processing pipeline", by taking an SQL expression on the command line, and outputting the lines containing matches for the SQL expression.

Microsoft describes Logparser as a powerful, versatile tool that provides universal query access to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows operating system such as the Event Log, the Registry, the file system, and Active Directory. The results of the input query can be custom-formatted in text based output, or they can be persisted to more specialty targets like SQL, SYSLOG, or a chart.

Example usage:

C:\>logparser.exe -i:textline -o:tsv "select Index, Text from 'c:\path\to\file.log' where line > 1000 and line < 2000"
C:\>logparser.exe -i:textline -o:tsv "select Index, Text from 'c:\path\to\file.log' where line like '%pattern%'"

The relativity of sizes

100 MB isn't too big. 3 GB is getting kind of big. I used to work at a print & mail facility that created about 2% of U.S. first class mail. One of the systems for which I was the tech lead accounted for about 15+% of the pieces of mail. We had some big files to debug here and there.

And more...

Feel free to add more tools and information here. This answer is community wiki for a reason! We all need more advice on dealing with large amounts of data...


(9) +1, I recently had some really huge xml files (+1 gigabyte) that I needed to look at. I'm on windows and both vim, emacs, notepad++ and several other editors completely choked on the file to the point where my system almost became unusable when trying to open the file. After a while I realized how unnecessary it was to actually attempt to open the file in an -editor- when I just needed to -view- it. Using cygwin (and some clever grep/less/sed-magic) I easily found the part I was interested in and could read it without any hassle. - wasatz
cygwin's less works for viewing a file > 2GB sweet - rogerdpack
(8) you don't need cygwin for less, you can also use it under windows: - ChristophK
I couldn't figure out how to edit files with less but after getting the taste I used vim to edit a bulky JSON object which made the other editors (Brackets, Textmate <- note I'm running on OS X) throw up. - nuala
(2) This XML editor here has also a large file viewer component and does provide syntax coloring also for huge files. The files are not loaded completely into memory so a multi-GB document shouldn't be a problem. In addition this tool can also validate those big XML documents ... In my opinion one of the best approaches to work with huge XML data. - lichtfusion
I'm going to "disavow" the logparser.exe thing that "the community" added to my answer, as I would prefer to stick to POSIX tools, rather than some Microsoft-only thing, which also apparently takes a longer command line to invoke than my other examples. - Roboprog
I tried the less packaged in mingw shell, and it complains about memory on a 400MB file. No good. - goat
less is great as long as the lines aren't too long. I'm in this thread because less (linux) is choking badly on debug log files that contain BIG serialized XML and I need something faster. - Andy Brown
(10) OK so I just fixed my own issue. less with word wrap is slow. less -S without word wrap is lightning fast even on large lines. I'm happy again! - Andy Brown
(9) Great answer. I want to note that if you have Git for Windows installed, you probably have Git bash as well, which includes less. - transistor1
Well, this thing has become a bit of an "Alan Smithe Production". My original answer said nothing about IIS logparser, since I avoid MS-Windows whenever possible. Happy Wiki-ing, I guess :-) - Roboprog
The locked questions are some of the best questions on StackOverflow. - kd4ttc
(1) i recommend glogg which is free and very fast...tested with an 8gb text file. but supports read-only. - Chidi
@AndyBrown -S is great and if you're going to move to the end of the file, -Sn is even better - golimar
I've used V File Viewer ( in the past and worked very well. Haven't used it in some time. - ajz