How to load a big text file efficiently in python

file, file-io, full-text-search, python, search

Solution

iterate over each line of the file, without storing it. This will make for program memory Efficient.

with open(filname) as f:
   for line in f:
      if "search_term" in line:
           break

Problem

I have a text file containing 7000 lines of strings. I got to search for a specific string based upon few params. Some are saying that the below code wouldn't be efficient (speed and memory usage). ``` f = open("file.txt") data = f.read().split() # strings as list ``` - First of all, if don't even make it as a list, how would I even start searching at all? - Is it efficient to load the entire file? If not, how to do it? - To filter anything, we need to search for that we need to read it right! A bit confused

Original source