expand time ranges into more steps of smaller incr

Question

I have a text file with time stamps and labels like this :

0.000000    14.463912   tone
14.476425   16.891247   noise
16.891247   21.232923   not_music
21.232923   23.172289   not_music
23.172289   29.128018   not_music

If I specify a step size of 1 second. I want this list to explode into time frames of 1 second long duration but still carry the nearest label. How do I explode the time ranges into smaller steps but with accurate labels?

for example if my step were 1 second, then the first line would become ~ 14 lines like :

0.0 1.0 tone
1.0 2.0 tone 
.  
.  
.
13.0 14.0 tone 
[14.0 , 14.46] and [14.47, 15.0] #fall in a grey zone , don't know 
what to do
15.0 16.0 noise

So far I have managed to read in the text file and store them in a list like:

my_segments =[]
for line in open('./data/annotate.txt', 'rb').readlines():
            start, end, label = line.split("\t")
            start = float(start)
            end = float(end)
            label = label.strip()
            my_segments.append((start, end, label))

# print my_segments
for i in range(len(my_segments)):
    print my_segments[i]

I looked at https://stackoverflow.com/a/18265979/4932791 by @Jared which details how to create a range between two numbers with a given step size using numpy. like so :

>>> numpy.arange(11, 17, 0.5)
array([ 11. ,  11.5,  12. ,  12.5,  13. ,  13.5,  14. ,  14.5,  15. ,
        15.5,  16. ,  16.5])

Unable to figure out how to do something similar on a range of ranges.

Pseudocode/algorithm I managed to come up with is :

• step 1- take a step size,
• step 2- assign step size to a left_variable and a right_variable corresponding to the step size
  step 3- move this step like window over the each range and check if the step falls within the range or not, If it does then assign it the corresponding label.
• step 4- now update the left and right by 1 step.
• step 5- repeat from step 3 till end of file is reached.

I think to handle edge cases, I should reduce step size to 0.25 seconds or something like that and put a condition if the current step has atleast 40 or 50% overlap then I assign the label accordingly.

Update : my non working solution :

sliding_window = 0
#st,en = [0.0,1.0]
jumbo= []
for i in range(len(hold_segments)):
    if sliding_window > hold_segments[i][0] and sliding_window+1 < hold_segments[i][1]:
        jumbo.append((sliding_window,sliding_window+1,hold_segments[i][2]))
        sliding_window=sliding_window+1
        print hold_segments[i][2]

Answer 1

With pandas that's quite straight-forward, assuming you've loaded your data into a dataframe called df such as:

df

               value        tag
index
0.000000   14.463912   ringtone
14.476425  16.891247      noise
16.891247  21.232923  not_music
21.232923  23.172289    music_B
23.172289  29.128018    music_A


df = df.reindex(
    [i + 0.5 for i in range(math.floor(df.index.min()), math.ceil(df.value.max()))], 
    method='pad'
)

Then restore the ranges with:

(df.index, df.value) = (df.index - 0.5, df.index + 0.5)

       value        tag
index
0.0      1.0   ringtone
1.0      2.0   ringtone
2.0      3.0   ringtone
3.0      4.0   ringtone
4.0      5.0   ringtone
5.0      6.0   ringtone
6.0      7.0   ringtone
7.0      8.0   ringtone
8.0      9.0   ringtone
9.0     10.0   ringtone
10.0    11.0   ringtone
11.0    12.0   ringtone
12.0    13.0   ringtone
13.0    14.0   ringtone
14.0    15.0      noise
15.0    16.0      noise
16.0    17.0      noise
17.0    18.0  not_music
18.0    19.0  not_music
19.0    20.0  not_music
20.0    21.0  not_music
21.0    22.0    music_B
22.0    23.0    music_B
23.0    24.0    music_A
24.0    25.0    music_A
25.0    26.0    music_A
26.0    27.0    music_A
27.0    28.0    music_A
28.0    29.0    music_A
29.0    30.0    music_A

Answer 2

I hope with the comments it is clear what the code does. Works also well for non-integer stepsize

from __future__ import division
import numpy as np

my_segments = [
    (0, 14.46, "ringtone"),
    (14.46, 16.89, "noise"),
    (16.89, 21.23, "not_music"),
]


def expand(segments, stepsize):
    result = []
    levels = [x[0] for x in segments] + [segments[-1][1]] #0, 14.46, 16.89, 21.23
    i = 0   # tracks the index in segments that we need at the current step
    for step in np.arange(0, levels[-1], stepsize):

        # first check if the index needs to be updated
        # update when the next level will be reached at the next 'stepsize / 2'
        # (this effectively rounds to the nearest level)
        if i < len(levels) - 2 and (step + stepsize / 2) > levels[i+1]:
            i += 1

        # now append the values
        result.append((step, step + stepsize, segments[i][2]))

    return result

stepsize = 0.02
print len(expand(my_segments, stepsize))
print my_segments[-1][1] / stepsize

>>> 1062  # steps are rounded up
>>> 1061.5