Log Into FT.com with Python's requests

Question

I'm trying to write a script which will log into my FT.com account and pull down some data for me.

The first hurdle is obviously logging in. I'm using Pythons Requests library for this.

Here is my code:

#!/usr/bin/env python
import requests


# log into FT website
log_in_url = 'https://accounts.ft.com/login'
my_email = '[e-mail]'
my_pass = 'mypassword'
data = {'email': my_email,
        'password': my_pass,
}

with requests.Session() as session:
    log_in_request = session.post(log_in_url, data=data)
    # prints log-in page with error about contacting cus. service
    print log_in_request.content

It just redirects me back to the login page. It doesn't say 'incorrect details' or anything (they're definitely correct). It just won't let me log in. I suspect something is wrong with my POST request - like I'm missing headers or cookies or something.

Anybody know what might be going on?

Cheers,

Dean

Answer 1

If you copy original content or scrape non-negligible amounts of data from the ft website, you will rightly be blocked as you will be violating their T&Cs (not to mention copyright law).

The FT is partly funded by advertisements. If you bypass these by using Python you are eroding a legimitimate revenue source.

I use this script to pull my watchlists into Excel, which I used to do using copy/paste. I do not use it for analytics or to read articles. I read the FT via the browser generating a fair degree of advert-viewing traffic daily.

Please do not abuse this. If you do, it is likely to result in every paying subscriber being blocked from doing even tiny amounts of navigating in Python.

from robobrowser import RoboBrowser

browser = RoboBrowser(history=True)

url='https://accounts.ft.com/login?location=https%3A%2F%2Fwww.ft.com%2F'

browser.open(url)
form=browser.get_form()
form['email']=""
form['password']=""
browser.session.headers['Referer'] = url
browser.submit_form(form)

There is an FT Developer Programme if you wish to perform Datamining and so on: https://developer.ft.com/