What happened when `pop`-ing an element while `for

Question

This question already has answers here:

Modifying list while iterating [duplicate] (7 answers)

Closed 6 years ago.

Code:

arr = [ i for i in xrange(10) ]

for i in arr:
  if i in arr:
    print i
    arr.pop(0)

print arr

And the output:

$ python2.7 ts.py 
0
2
4
6
8
[5, 6, 7, 8, 9]

Why is this the result? Shouldn't it be []?

Answer 1

Let me show you what happens in the code:

# Initial position
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# ^
# i

# Remove first
 [1, 2, 3, 4, 5, 6, 7, 8, 9]
# ^
# i

# Move next
 [1, 2, 3, 4, 5, 6, 7, 8, 9]
#    ^
#    i

# Remove first
 [2, 3, 4, 5, 6, 7, 8, 9]
#    ^
#    i

# Move next
 [2, 3, 4, 5, 6, 7, 8, 9]
#       ^
#       i

# And so on...

 [4, 5, 6, 7, 8, 9]
#             ^
#             i

# Remove first
 [5, 6, 7, 8, 9]
#             ^
#             i

# Move next
# Oops, the end of the list

# The result:

[5, 6, 7, 8, 9]

Let's see how does it work under the hood. Firstly, we need an iterator:

# for i in arr:

In [30]: it = iter(arr)

In [31]: it
Out[31]: <listiterator at 0x130f9d0>

And we will call next(it) until it rises a StopIteration exception. So, let's do it:

In [32]: i = next(it)

In [33]: i
Out[33]: 0

Wow, we got the first element from the list! Let's see, what will happen if we try to pop element with zero index:

# if i in arr:
#     print i
#     arr.pop(0)

In [34]: i in arr
Out[34]: True

In [35]: print i
0

In [36]: arr.pop(0)
Out[36]: 0

In [37]: arr
Out[37]: [1, 2, 3, 4, 5, 6, 7, 8, 9]

Okay, element has popped. Move to next loop iteration:

In [38]: i = next(it)

In [39]: i
Out[39]: 2

Hmm... Seems right, we got the second element. Let's pop the first again!

In [40]: i in arr
Out[40]: True

In [41]: print i
2

In [42]: arr.pop(0)
Out[42]: 1

In [43]: arr
Out[43]: [2, 3, 4, 5, 6, 7, 8, 9]

Let's see at third iteration:

In [44]: i = next(it)

In [45]: i
Out[45]: 4

I guess, it's clear now, that loop will have 5 iterations. In each iteration you will remove the first element. And, because of it, you will skip odd elements while iterating.

Answer 2

It is not recommended to modify the sequence(or mapping) while you iterate it. It will mess interal pointer.

For example, following code will never end.

arr = [1,2,3]
for i in arr:
    print i
    arr.append(i)

According to for statement - NOTE:

Note: There is a subtlety when the sequence is being modified by the loop (this can only occur for mutable sequences, i.e. lists). An internal counter is used to keep track of which item is used next, and this is incremented on each iteration. When this counter has reached the length of the sequence the loop terminates. This means that if the suite deletes the current (or a previous) item from the sequence, the next item will be skipped (since it gets the index of the current item which has already been treated). Likewise, if the suite inserts an item in the sequence before the current item, the current item will be treated again the next time through the loop. This can lead to nasty bugs that can be avoided by making a temporary copy using a slice of the whole sequence, e.g.,

for x in a[:]:
   if x < 0: a.remove(x)

Answer 3

Apparently what you whanted is:

for i in range(len(arr)):
    arr.pop(0)

Because, as @falsetru mentioned, changing the sequence during the iteration is not recommended, but in this example your for loop is based on a constant value, i.e. the length of arr. Each pop() will remove the first element and the list will be progressively emptied.

Answer 4

Updating a Sequence while Iterating has some unexpected results, which is why it is never recommended. The following graphic depicts how the variable i changes every time you iterate while popping from the list

var    Instruction                    <--------- arr -------------> 
 i                                   [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
 |     for i in arr                   ^
 |____________________________________|
 |                                    |
 |                                    V
 |     arr.pop(0)                    [1, 2, 3, 4, 5, 6, 7, 8, 9]
 |     
 |     for i in arr                  [1, 2, 3, 4, 5, 6, 7, 8, 9]
 |                                       ^
 |_______________________________________|
 |_______________________________________|
 |                                       |
 |                                       V
 |     arr.pop(0)                    [2, 3, 4, 5, 6, 7, 8, 9]
 |     
 |     for i in arr                  [2, 3, 4, 5, 6, 7, 8, 9]
 |                                          ^
 |__________________________________________|
 |__________________________________________|
 |                                          |
 |                                          V
 |     arr.pop(0)                    [3, 4, 5, 6, 7, 8, 9]
 |     
 |     for i in arr                  [3, 4, 5, 6, 7, 8, 9]
 |                                             ^
 |_____________________________________________|
 |_____________________________________________|
 |                                             |
 |                                             V
 |     arr.pop(0)                    [4, 5, 6, 7, 8, 9] 
 |     
 |     for i in arr                  [4, 5, 6, 7, 8, 9]
 |                                                ^
 |________________________________________________|
 |________________________________________________|
 |                                                |
 |                                                V
 |     arr.pop(0)                    [5, 6, 7, 8, 9] 

Answer 5

It's easier to see whats going on by adding an enumerate:

for index,i in enumerate(arr):
    if i in arr:
        print(index,i)
        arr.pop(0)
print arr

outputs:

(0, 0)
(1, 2)
(2, 4)
(3, 6)
(4, 8)
[5, 6, 7, 8, 9]

Popping changes the length of the array but the index for the iterator isn't updated to reflect this.