Text editor which can directly search compressed text files? (only uncompressing them in memory)

text-editor

Solution

`zless` from `cygwin` should be able to do this.

Problem

I have 20 GB of (uncompressed) log files. They're initially compressed though (as one 70 KB .gz file per log file) and total about 700 MB. I need to browse and search them to diagnose some issues. I don't know exactly what I'm searching for and I probably will need to quickly browse through a couple thousand hits for any search I try. I tried doing this by uncompressing all of the files and then searching through them using Notepad++ or Visual Studio. The problem is that the searches are horrendously slow and put a lot of load on the hard disk. I assume that for every search I do, it needs to read all 20 GB from disk. What might work better is if there's a text editor (or a Notepad++ plugin?) which can search inside .gz files without uncompressing them to disk. The 700 MB of .gz files can easily fit into system cache and I assume that uncompressing each file in memory would be much faster than reading an uncompressed file from disk. I guess my alternative is to only work with a couple of gigs at a time so that all of it has a chance of being cached, but that would be pretty inconvenient. Thanks for suggestions.

Original source