Iterating through a pandas dataframe

pandas, python

Solution

Here's another take

df['group'] = (df.condition == False).astype('int').cumsum().shift(1).fillna(0)

df
             date    long     lat condition  group
2/5/2013 19:45:00  39.940 -86.159      True      0
2/5/2013 19:50:00  39.940 -86.159      True      0
2/5/2013 19:55:00  39.940 -86.159     False      0
2/5/2013 20:00:00  39.777 -85.995     False      1
2/5/2013 20:05:00  39.775 -85.978      True      2
2/5/2013 20:10:00  39.775 -85.978      True      2
2/5/2013 20:15:00  39.775 -85.978     False      2
2/5/2013 20:20:00  39.940 -86.159      True      3
2/5/2013 20:25:00  39.940 -86.159     False      3

df['result'] = df.groupby(['group']).date.transform(lambda sdf: 5 *len(sdf))

df
             date    long     lat condition  group result
2/5/2013 19:45:00  39.940 -86.159      True      0     15
2/5/2013 19:50:00  39.940 -86.159      True      0     15
2/5/2013 19:55:00  39.940 -86.159     False      0     15
2/5/2013 20:00:00  39.777 -85.995     False      1      5
2/5/2013 20:05:00  39.775 -85.978      True      2     15
2/5/2013 20:10:00  39.775 -85.978      True      2     15
2/5/2013 20:15:00  39.775 -85.978     False      2     15
2/5/2013 20:20:00  39.940 -86.159      True      3     10
2/5/2013 20:25:00  39.940 -86.159     False      3     10

Problem

I have a pandas dataframe where one column represents if the location value in another column changed in the row below it. As an example, ``` 2013-02-05 19:45:00 (39.94, -86.159) True 2013-02-05 19:50:00 (39.94, -86.159) True 2013-02-05 19:55:00 (39.94, -86.159) False 2013-02-05 20:00:00 (39.777, -85.995) False 2013-02-05 20:05:00 (39.775, -85.978) True 2013-02-05 20:10:00 (39.775, -85.978) True 2013-02-05 20:15:00 (39.775, -85.978) False 2013-02-05 20:20:00 (39.94, -86.159) True 2013-02-05 20:30:00 (39.94, -86.159) False ``` So, what I want to do is go row by row through this dataframe and check for the rows with `False`. And then (may be add another column) which has total 'continuous' time spent in that place. The same place can be visited again like in the example above. In that case it is taken to be as a separate condition. So, for the above example, something like: ``` 2013-02-05 19:45:00 (39.94, -86.159) True 0 2013-02-05 19:50:00 (39.94, -86.159) True 0 2013-02-05 19:55:00 (39.94, -86.159) False 15 2013-02-05 20:00:00 (39.777, -85.995) False 5 2013-02-05 20:05:00 (39.775, -85.978) True 0 2013-02-05 20:10:00 (39.775, -85.978) True 0 2013-02-05 20:15:00 (39.775, -85.978) False 15 2013-02-05 20:20:00 (39.94, -86.159) True 0 2013-02-05 20:25:00 (39.94, -86.159) False 10 ``` I would then plot a histogram of these 'continuous' time spent using the hist() function per day. How would I get the second dataframe from the first by iterating through the dataframe? I'm new to python and pandas and the real datafile is huge so, I would need something reasonably efficient.

Original source