Automating Chrome

问题:

I've seen some wild things happen when I paste some urlencoded Javascript into a URL on Firefox and Chrome. Is it possible to use this technique to tell Chrome to visit a URL and then save it as a file? I'm trying to automate Chrome, and Selenium looked extremely daunting.

EDIT: Unfortunately, I forgot to be more clear here. Let me explain. Things like wget, curl, etc. won't work because I have to get through logins in some of these scripts. And I've looked at iMacros, but found that I can't get them to run from command line except on Windows, unless I pay for the $499 package. Some other advantages of GCEs are that the dev platform is free and open to some degree, and they are cross-platform. (I use Linux.)

EDIT: At this point, I'm learning about Google Chrome Extensions. It appears they are easy to build and will let me (I think) tell the browser to open a new tab, go to a page, manipulate the DOM on that page (such as populating some fields and logging in), and then manipulate the DOM on the response page. GCEs don't let you do File I/O, so they are not like Firefox Extensions with XPCOM, but you can get around that by using AJAX to send data to a backend script (like a PHP script on a LAMP server) to save that data.

EDIT: By the way, and this is slightly off-topic (but I add to clarify) those "wild things" in Javascript that I mentioned were when you manipulate the DOM creating a URL that looks like so:

javascript:(function(){...your URL-encoded Javascript here...})();

回答1:

You can use Python to automate web tasks using pywebkitgtk. It is a Python binding for WebKitGtk, which uses the WebKit engine, the same engine as chrome.

Thanks to this blog post, pywebkitgtk - Execute JavaScript from Python, I made a subclass of webkit.WebView to make these tasks easier.

import gtk
import webkit
import json

class WebView(webkit.WebView):
    def eval_script(self, script):
        self.execute_script('oldtitle=document.title;document.title="!!!!";document.title=JSON.stringify(eval(' + json.dumps(script) + '));')
        result = json.loads(self.get_main_frame().get_title())
        self.execute_script('document.title=oldtitle;')
        return result
    def wait_for_load(self):
        handle = None
        def load_status_cb(view, frame):
            if frame == view.get_main_frame():
                self.disconnect(handle)
                gtk.main_quit()
        handle = self.connect('load-finished', load_status_cb)
        gtk.main()

I added the function called eval_script which is like execute_script, but you could get the results of the function as Python objects. You just need to make sure that what you are evaluating is JSON-serializable.

Also, I added a wait_for_load function which is pretty self-explanatory.

To set up a UI, you first have to create a window, a scrolled window, and a web view.

# window
window = gtk.Window()
window.set_default_size(800, 600)

# scroll view
scroll_view = gtk.ScrolledWindow()
scroll_view.props.hscrollbar_policy = gtk.POLICY_AUTOMATIC
scroll_view.props.vscrollbar_policy = gtk.POLICY_AUTOMATIC

# web view
web_view = WebView()

# events
window.connect('delete-event', lambda window, event: gtk.main_quit())

# show
scroll_view.add(web_view)
window.add(scroll_view)
window.show_all()

Then you can start automating things! For example, this code loads StackOverflow's login page, click the Facebook login button, fills in the username and password (in this case "test"). Finally, it shows the login button text.

# the script is here
web_view.open('http://www.stackoverflow.com/users/login')
web_view.wait_for_load()

web_view.execute_script('openid.signin("facebook")')
web_view.wait_for_load()

web_view.execute_script('document.querySelector("#email").value = "test"')
web_view.execute_script('document.querySelector("#pass").value = "test"')

print "Login's button text is:", web_view.eval_script('document.querySelector("#buttons input[type=\\"submit\\"]").value')

In my case, the Facebook's interface was in Thai language, and I could see the login's button text.

Login's button text is: เข้าสู่ระบบ

You can also have it actually click the submit button, just by calling click() on that element. (Note: click() works for button elements, not on links)

web_view.execute_script('document.querySelector("#buttons input[type=\\"submit\\"]").click()')
web_view.wait_for_load()

You will notice that after all the scripts are finished, the application closes itself without waiting.

If you want to keep the application running after it finish all the scripts in there, you need to add the last line:

gtk.main()

Also, if you remove the window.show_all() line and the last gtk.main() line. Then your app will work without a GUI. (Note: You still need a display server.)

Right now, we don't have good pywebkitgtk docs yet, so you have to look at WebKitGtk's documentation instead. Good luck.

回答2:

The solution appears to be to make one's own Google Chrome Extension (GCE). It is easy to learn within about 4 hours if you know how to do slightly advanced Javascript stuff, and is very powerful. I can use the Tabs API to create a new tab and go to a specific URL. I can then inject jQuery into that URL and make it manipulate the DOM or do anything we normally can do with jQuery. I can't do file I/O, but there are two workarounds. One, I can force the browser to download a file from a remote location, and I can send data from the current page back up to a remote server via jQuery's $.get() or $.post() calls.

回答3:

You could try iMacros for Chrome. It is a pretty easy to use automation system.

1. Open iMacros
2. Click Record.
3. Go about you browsing routine.
4. Click stop.

I don't think it can get any easier than that. The scripts it saves are in plain text so you can edit them for some fine grain control if need be.

回答4:

Definitely check out Watir! I find it extremely straightforward. It works just as easily with Selenium as it does with Chrome, IE, or Firefox. Though the version for Chrome is not yet officially supported, they claim it is usable. I, myself have only used it for Selenium, IE, and Firefox.

Also, Watir easily integrates with Cucumber, if you're looking for cutting-edge BDD (Behavior-Driven Development). It's just Ruby code, is open source, and hosted on gitHub. Enjoy!

回答5:

You can checkout http://qaagent.com. This is an easy way to automate some web related task