Using Numpy to create Yahoo finance price table

Question

Without using matplotlib finance module. I like to get the url data into a numpy array. where I can to column heading to do math. Like:

prices = r.adj_close

From: http://matplotlib.sourceforge.net/examples/pylab_examples/finance_work2.html

except I dont want to use the:

fh = finance.fetch_historical_yahoo(ticker, startdate, enddate)
# a numpy record array with fields: date, open, high, low, close, volume, adj_close)

r = mlab.csv2rec(fh); fh.close()
r.sort()

Using manually create the url:

        url = http://ichart.yahoo.com/table.csv?a=2&c=2011&b=30&e=7&d=7&g=d&f=2011&s=msft&ignore=.csv

        f = urllib.urlopen(url)
        fr = f.read()



        hdata = np.asarray(fr, dtype='object')
        prices = hdata.adj_close
        print prices

Answer 1

It is also possible to use S10 to tell numpy that the first entity is a string with length 10. This way, you don't need to use lambda.

data = np.loadtxt(f, dtype={'names': ('dtime', 'open', 'high','low','close','volume','aclose'), 'formats': ('S10', '<f8', '<f8','<f8','<f8','i','<f8')}, 
                             delimiter="," )

i=integer, <f8 =0.256, f8=0.25600001298, S10="MM-DD-YYYY"

For more info on f, f8, u4, S, u8 etc, visit this link.

Answer 2

If you don't want to load pylab for time string conversion, you can use the mktime function as a lambda:

import numpy as np
import urllib
import time 
url = "http://ichart.yahoo.com/table.csv?a=2&c=2011&b=30&e=7&d=7&g=d&f=2011&s=msft&ignore=.csv" 
f = urllib.urlopen(url) 
title = f.readline().strip().split(",") 
data = np.loadtxt(f, dtype={'names': ('dtime', 'open', 'high','low','close','volume','aclose'),
                            'formats': ('u4', 'f8', 'f8','f8','f8','u4','f8')}, 
                     delimiter=",", 
                     converters={0: lambda y:int(time.mktime(time.strptime(y,'%Y-%m-%d')))})

Answer 3

use numpy.loadtxt() to load csv:

import numpy as np
import pylab as pl
import urllib
url = "http://ichart.yahoo.com/table.csv?a=2&c=2011&b=30&e=7&d=7&g=d&f=2011&s=msft&ignore=.csv"
f = urllib.urlopen(url)
title = f.readline().strip().split(",")
data = np.loadtxt(f, dtype=np.float, delimiter=",", converters={0: pl.datestr2num}))

the first column is date, so use pylab.datestr2num to convert it to number.