Python (3) readline returns empty string regardless of what is in the file

file-io, python, python-3.x

Solution

`getNumEvents` consumes the entire file, so when you go around reading it, its pointer is already at the end. Instead, reset the file pointer with `seek`, like this:

def runFile(eventFile, channelRefFileName, serialPorts, timeScale, maxPEs):
    timer0 = threading.Timer(.0001, turnOffLED, [0, 0, 0, 1])
    numEvents = getNumEvents(eventFile)
    eventFile.seek(0, 0)
    eventCount = 0
    for linein in eventFile:
        if linein == '': print('EOF')
        ....

Problem

I have a Python (3) program that reads some data in from a file, gets the relevant parts and spits it out to various other places. The problem is that `file.readline()` has started refusing to get any lines from any file. It just returns an empty string as if it was at the end of the file. The files are not the problem, I checked that with a quick script (read file, print line by line). The possible solution (although I don't know why) is I used 2to3 to convert the code to python3, but looking at the diff it printed I don't see any modifications to the file I/O stuff. Here is the program: ``` import threading import time def runFile(eventFile, channelRefFileName, serialPorts, timeScale, maxPEs): timer0 = threading.Timer(.0001, turnOffLED, [0, 0, 0, 1]) numEvents = getNumEvents(eventFile) eventCount = 0 while 1: linein = eventFile.readline() #if linein == '' or '\n' or '\r\n' or '\r': break #above checks if the end of file has been reached, below is debugging if linein == '': print('EOF') if linein == '\n': print('linefeed') if linein == '\r\n': print('carrige return/ linefeed') if linein == '\r' : print('carrige return') if linein == 'New Event\n': print('New Event', eventCount, ': file is', (eventCount/numEvents)*100, '% done.') eventCount += 1 #insert event sleep time here continue linein = linein.split(' ') del(linein[::2]) linein[2] = linein[2][:-1] linein[1] = float(linein[1]) if linein[1] < 0: linein[1] = linein[1] * (-1) channel = channelLookup(linein[0], channelRefFileName) #serialPorts[channel[0]].write(str(channel[1],linein[2])) if timer0.isAlive: #use backup timer timer1 = threading.Timer(0.005, turnOffLED, [serialPorts, channel[0], channel[1], 0]) timer1.start() else: #use primary timer timer0 = threading.Timer(0.005, turnOffLED, [serialPorts, channel[0], channel[1], 0]) timer0.start() print('Sleeping:', linein[1]*timeScale) time.sleep(linein[1]*timeScale) def channelLookup(channel, channelRefFile): channelRef = open(channelRefFile, 'r') for line in channelRef: if channel in line: linein = line break linein = linein.split('-') channelRef.close() return linein[1:] def turnOffLED(serialPorts, arduino, pmt, bs): if bs: return -1 #serialPorts[arduino].write(str(pmt,0)) def getNumEvents(eventFile): i = 0 for lines in eventFile: if lines == 'New Event\n': i += 1 return i ``` The above is run by: ``` import hawc import datetime timeScale = 0.001 #timeScale of 0.000001 results in runtime of 0:06:52.858136 #a timeScale of 0.001 is 9:28:34.837992 eventSleep = 0 maxPEs = 1000 serialPorts = [0, 1, 2, 3, 4] channelRefFileName = 'Data/hawc-arduino_channel_lookup' st = datetime.datetime.now() print('Start time is', st) eventFile = open('Data/events.txt', 'r') hawc.runFile(eventFile, channelRefFileName, serialPorts, timeScale, maxPEs) eventFile.close() et = datetime.datetime.now() print('End time is', et) print('File has run for', et-st) print('--------Done---------') ``` And here is some sample data that the program might handle (one file might be about 500000 lines): ``` Channel: 447 Time: 121.263 PEs: 2263.38 Channel: 445 Time: 118.556 PEs: 1176.54 Channel: 448 Time: 120.384 PEs: 1159.59 Channel: 446 Time: 122.798 PEs: 983.949 Channel: 499 Time: 129.983 PEs: 762.07 ```

Original source