How to find the byte position of specific line in a file
bash, command-line, linux
Solution
From @chepner's comment on my other answer:
position = 0 # or wherever you left off last time
try:
with open('myfile.txt') as file:
file.seek(position) # zero in base case
for line in file:
position = file.tell() # current seek position in file
# process the line
except:
print 'exception occurred at position {}'.format(position)
raise
Problem
What's the fastest way to find the byte position of a specific line in a file, from the command line? e.g. ``` $ linepos myfile.txt 13 5283 ``` I'm writing a parser for a CSV that's several GB in size, and in the event the parser is halted, I'd like to be able to resume from the last position. The parser is in Python, but even iterating over `file.readlines()` takes a long time, since there are millions of rows in the file. I'd like to simply do `file.seek(int(command.getoutput("linepos myfile.txt %i" % lastrow)))`, but I can't find a shell command to efficiently do this. Edit: Sorry for the confusion, but I'm looking for a non-Python solution. I already know how to do this from Python.