python over-writing my list elements

Question

Fairly new to python, very new to python classes. Question is a bit involved. Most appreciative of your patience:

I have a class "Star". It is simple. Attributes x, v, and mass. Another class, Galaxy, has an attribute "stars" which is just a list of star objects:

class Galaxy:

    numstars=0.
    stars=[]

    def __init__(self,numin,xes,vees,masses):
        self.numstars=numin
        for n in range(numin):
            self.stars.append(Star(xes[n],vees[n],masses[n]))

Galaxy also has an attribute function called time_stepper. Suffice it to say time_stepper just updates all the elements of "stars" and then returns "stars":

def time_stepper(self,dt):
    self.velstep(dt)
    self.xstep(dt)
    self.velstep(dt)
    return(self.stars)

Now, I'm trying to drive this thing and store the various updates of "stars" in a list called "history":

gal=Galaxy(#stuff#)
history=[]
for n in range(100):
    history.append(gal.time_stepper(.1))

Finally, my question: In each iteration of this loop, the new element of "stars" is added to "history", but ... and here it is ... all the previous elements of history are over-written and given the same values as the newest element of history! So what is going on? I've run into things about python lists that I didn't understand before, but I thought I finally had it nailed down. Apparently not. Thanks for your help.

Addendum: Thanks to everyone for your help. Didn't expect that many helpful replies and especially so soon. My problem was that I was assuming these two pieces of code were essentially the same. First:

>>> a=[]
>>> b=[1,2,3]
>>> a.append(b)
>>> b=[4,5,6]
>>> a.append(b)
>>> a
[[1, 2, 3], [4, 5, 6]]

Second:

>>> a=[]
>>> b=[1,2,3]
>>> a.append(b)
>>> b[:]=(4,5,6)
>>> b
[4, 5, 6]
>>> a.append(b)
>>> a
[[4, 5, 6], [4, 5, 6]]

And whoops! They aren't. So in code 1, I guess, b is "re-pointed" to a completely new memory location while a[0] continues to point the the old b. In the second, the memory at b is "edited" and a[0] is still pointing to that location. After the append, a[1] is also pointing to that location. Do I have it now?

I'm very new to python and am still figuring out the "pythonic" philosophy. But to me, having a reassignment of pointers done straightforwardly, but a "deep copy" done in a more complicated way is sort of backwards from the way I usually want to do things. Can anyone enlighten me? Thanks again.

Answer 1

This is because the stars list is mutable - it is a changeable variable. When you call stars.append, it does not create a new list - it simple edits the existing list. And when you call history.append(x), python does not create a fresh, clean copy of x - it assumes you want the "real" x to be placed in the history array.

If you'd like to copy the state of the list at each iteration, there are several options (see How to clone a list in python?).

The copy module will do the trick. It offers both 'shallow' and 'deep' copies - roughly speaking, deep copies attempt to also copy any other variables they find inside an object (for example, lists containing other lists), while shallow ones just go one layer down:

Here's shallow copies with copy:

import copy

history=[]
for n in range(100):
    history.append(copy.copy(gal.time_stepper(.1)))

And deep copies:

import copy

history=[]
for n in range(100):
    history.append(copy.deepcopy(gal.time_stepper(.1)))

Slicing the array will work, since it always shallow copies as an intermediate step:

history = []
for n in range(100):
    history.append(gal.time_stepper(.1)[:])

You can also call list on the existing list - this typecast-esque operation always returns a new list object (again, shallowly):

history = []
for n in range(100):
    history.append(list(gal.time_stepper(.1)))

Answer 2

It is not changing or over writing all of the values, but rather all of the elements in the history point to the same array object. You need to assign by value as opposed to reference by using a copy of the array.

Answer 3

I think it is worth noting that if you have more than 1 galaxy, they'll share the same stars. This could lead you to believe that you're overwriting your stars when you really aren't...

I'm guessing you would probably be better with an __init__ that looks like:

def __init__(self,numin,xes,vees,masses):
    self.stars = []
    self.numstars = numin
    for n in range(numin):
        self.stars.append(Star(xes[n],vees[n],masses[n]))

Here I've shifted the class attribute stars to be an instance attribute. Now each instance will have it's own stars list instead of sharing one stars list with all of the other galaxies in the universe.

As others have noted, your history list is going to suffer from a similar problem (You have multiple references to the same list). However, the fix really depends on what you do in self.velstep and self.xstep. If you modify the Star objects in place, then a simple (shallow) list copy won't do you any good (e.g. gal.time_stepper(0.1)[:]). (You'll create a new list, but it will hold the same stars which are constantly being updated). In that case, you'll want copy.deepcopy when you append to your history list:

history=[]
for n in range(100):
    history.append(copy.deepcopy(gal.time_stepper(.1)))