Use of None in Array indexing in Python

Question

I am using the LSTM tutorial for Theano (http://deeplearning.net/tutorial/lstm.html). In the lstm.py (http://deeplearning.net/tutorial/code/lstm.py) file, I don't understand the following line:

c = m_[:, None] * c + (1. - m_)[:, None] * c_

What does m_[:, None] mean? In this case m_ is the theano vector while c is a matrix.

Answer 1

This question has been asked and answered on the Theano mailing list, but is actually about the basics of numpy indexing.

Here are the question and answer https://groups.google.com/forum/#!topic/theano-users/jq92vNtkYUI

For completeness, here is another explanation: slicing with None adds an axis to your array, see the relevant numpy documentation, because it behaves the same in both numpy and Theano:

http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html#numpy.newaxis

Note that np.newaxis is None:

import numpy as np
a = np.arange(30).reshape(5, 6)

print a.shape  # yields (5, 6)
print a[np.newaxis, :, :].shape  # yields (1, 5, 6)
print a[:, np.newaxis, :].shape  # yields (5, 1, 6)
print a[:, :, np.newaxis].shape  # yields (5, 6, 1)

Typically this is used to adjust shapes to be able to broadcast to higher dimensions. E.g. tiling 7 times in the middle axis can be achieved as

b = a[:, np.newaxis] * np.ones((1, 7, 1))

print b.shape  # yields (5, 7, 6), 7 copies of a along the second axis

Answer 2

I think the Theano vector's __getitem__ method expects a tuple as an argument! like this:

class Vect (object):
    def __init__(self,data):
        self.data=list(data)

    def __getitem__(self,key):
        return self.data[key[0]:key[1]+1]

a=Vect('hello')
print a[0,2]

Here print a[0,2] when a is an ordinary list will raise an exception:

>>> a=list('hello')
>>> a[0,2]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: list indices must be integers, not tuple

But here the __getitem__ method is different and it accepts a tuple as an argument.

You can pass the : sign to __getitem__ like this as : means slice:

class Vect (object):
    def __init__(self,data):
        self.data=list(data)

    def __getitem__(self,key):
        return self.data[0:key[1]+1]+list(key[0].indices(key[1]))

a=Vect('hello')
print a[:,2]

Speaking about None, it can be used when indexing in plain Python as well:

>>> 'hello'[None:None]
'hello'