Controlling a browser using Python, on a Mac

Question

I'm looking for a way to programatically control a browser on a Mac (i.e. Firefox or Safari or Chrome/-ium or Opera, but not IE) using Python.

The actions I need include following links, checking if elements exist in a page, and submitting forms.

Which solution would you recommend?

Answer 1

Try mechanize, if you don't actually need a browser.

Example:

import re
import mechanize

br = mechanize.Browser()
br.open("http://www.example.com/")
# follow second link with element text matching regular expression
response1 = br.follow_link(text_regex=r"cheese\s*shop", nr=1)
assert br.viewing_html()
print br.title()
print response1.geturl()
print response1.info()  # headers
print response1.read()  # body

br.select_form(name="order")
# Browser passes through unknown attributes (including methods)
# to the selected HTMLForm.
br["cheeses"] = ["mozzarella", "caerphilly"]  # (the method here is __setitem__)
# Submit current form.  Browser calls .close() on the current response on
# navigation, so this closes response1
response2 = br.submit()

Answer 2

Might be a bit restrictive, but py-appscript may be the easiest way of controlling a Applescript'able browser from Python.

For more complex things, you can use the PyObjC to achieve pretty much anything - for example, webkit2png is a Python script which uses WebKit to load a page, and save an image of it. You need to have a decent understanding of Objective-C and Cocoa/etc to use it (as it just exposes ObjC objects to Python)

Screen-scaping may achieve what you want with much less complexity.

Answer 3

You can use selenium library for Python, here is a simple example (in form of unittest):

#!/usr/bin/env python3
import unittest
from selenium import webdriver

class FooTest(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Firefox()
        self.base_url = "http://example.com"

    def is_text_present(self, text):
        return str(text) in self.driver.page_source

    def test_example(self):
        self.driver.get(self.base_url + "/")
        self.assertTrue(self.is_text_present("Example"))

if __name__ == '__main__':

    suite = unittest.TestLoader().loadTestsFromTestCase(FooTest)
    result = unittest.TextTestRunner(verbosity=2).run(suite)

Answer 4

Several Mac applications can be controlled via OSAScript (a.k.a. AppleScript), which can be sent via the osascript command. O'Reilly has an article on invoking osascript from Python. I can't vouch for it doing exactly what you want, but it's a starting point.