Text editor to open big (giant, huge, large) text filesI mean 100+ MB big; such text files can push the envelope of editors.
I need to look through a large XML file, but cannot if the editor is buggy.
Any suggestions?
ACCEPTED]
Free read-only viewers:
tail), bookmarks, highlighting patterns using different colors, and has serious optimizations built in. But from a UI standpoint, it's rather minimal.tail." It's really a log file analyzer, not a large file viewer, and in one test it required 10 seconds and 700 MB of RAM to load a 250 MB file. But its killer features are the columnizer (parse logs that are in CSV, JSONL, etc. and display in a spreadsheet format) and the highlighter (show lines with certain words in certain colors). Also supports file following, tabs, multifiles, bookmarks, search, plugins, and external tools.Free editors:
Builtin programs (no installation required):
MORE, not the Unix more. A console program that allows you to view a file, one screen at a time.Web viewers:
Paid editors/viewers:
:set display+=lastline. It is kind of weird to not have that as the default, though. - Aaron R.
more is ideal for looking and confirming file structure. - clg4
glogg has search performance in between Liquid Studio and EmEditor. EmEditor is still the fastest. - Desik
. (dot) with RegEx disabled vs. RegEx enabled to see the difference. Press F3 to continue to the next match. - Samir
GigaEdit's link is dead, could someone update it? I didn't find the new homepage. - Basj
editpadLite, notepad++ both fail to open sql dump 5.1G - teran
Why are you using editors to just look at a (large) file?
Under *nix or Cygwin [1], just use less [2]. (There is a famous saying – "less is more, more or less" – because "less" replaced the earlier Unix command "more", with the addition that you could scroll back up.) Searching and navigating under less is very similar to Vim, but there is no swap file and little RAM used.
There is a Win32 port of GNU less. See the "less" section of the answer above.
Perl is good for quick scripts, and its .. (range flip-flop) operator makes for a nice selection mechanism to limit the crud you have to wade through.
For example:
$ perl -n -e 'print if ( 1000000 .. 2000000)' humongo.txt | less
This will extract everything from line 1 million to line 2 million, and allow you to sift the output manually in less.
Another example:
$ perl -n -e 'print if ( /regex one/ .. /regex two/)' humongo.txt | less
This starts printing when the "regular expression one" finds something, and stops when the "regular expression two" find the end of an interesting block. It may find multiple blocks. Sift the output...
This is another useful tool you can use. To quote the Wikipedia article [3]:
logparser is a flexible command line utility that was initially written by Gabriele Giuseppini, a Microsoft employee, to automate tests for IIS logging. It was intended for use with the Windows operating system, and was included with the IIS 6.0 Resource Kit Tools. The default behavior of logparser works like a "data processing pipeline", by taking an SQL expression on the command line, and outputting the lines containing matches for the SQL expression.
Microsoft describes Logparser as a powerful, versatile tool that provides universal query access to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows operating system such as the Event Log, the Registry, the file system, and Active Directory. The results of the input query can be custom-formatted in text based output, or they can be persisted to more specialty targets like SQL, SYSLOG, or a chart.
Example usage:
C:\>logparser.exe -i:textline -o:tsv "select Index, Text from 'c:\path\to\file.log' where line > 1000 and line < 2000"
C:\>logparser.exe -i:textline -o:tsv "select Index, Text from 'c:\path\to\file.log' where line like '%pattern%'"
100 MB isn't too big. 3 GB is getting kind of big. I used to work at a print & mail facility that created about 2% of U.S. first class mail. One of the systems for which I was the tech lead accounted for about 15+% of the pieces of mail. We had some big files to debug here and there.
Feel free to add more tools and information here. This answer is community wiki for a reason! We all need more advice on dealing with large amounts of data...
[1] http://en.wikipedia.org/wiki/Cygwinless but after getting the taste I used vim to edit a bulky JSON object which made the other editors (Brackets, Textmate <- note I'm running on OS X) throw up. - nuala
less packaged in mingw shell, and it complains about memory on a 400MB file. No good. - goat
less is great as long as the lines aren't too long. I'm in this thread because less (linux) is choking badly on debug log files that contain BIG serialized XML and I need something faster. - Andy Brown
less with word wrap is slow. less -S without word wrap is lightning fast even on large lines. I'm happy again! - Andy Brown
less. - transistor1
glogg which is free and very fast...tested with an 8gb text file. but supports read-only. - Chidi
-S is great and if you're going to move to the end of the file, -Sn is even better - golimar
cat /dev/urandom | tr -dc 'A-z' | head -c 1000000, where the last number after -c is the number of bytes in the file. - indivgrep -r "someText" . > bigfileassuming that there are some files in your dir that contain matching lines with the search criteria. Of course, you would need to stop grep forcefully as this it will make it enter in a endless loop :) - user624558