After experimenting with scripts on facebook.com, I noticed that for a couple of minutes it didn't allow me to connect to facebook.com. This happened more than one time so I suspect that Facebook doesn't like to mess with their code using userscripts.
The Stack Overflow question Can a website know if I am running a userscript? answers the multiple ways a website can detect scripts.
Now I want to know how I can fool a website to not detect any scripts I may be running.
Is there a way to know what my userscript HTML changes trigger on the website?
For Firefox and Chrome there is no magic bullet to prevent detection, it is a process and an art...
The 3 principle ways that a site would detect a script are:
Out of sequence or too-frequent AJAX requests.
Changes to the page's code (HTML, global JS, or even CSS).
Unnatural mouse or keyboard action.
See "Can a website know if I am running a userscript?" for more information.
Some countermeasures:
For AJAX requests that your script makes:
- Use Wireshark, or similar, to analyze the traffic of normal AJAX calls. Make sure that you send the same type of calls, in the same order.
- Make sure that your requests show the same headers and data as the page's.
- Do not send AJAX requests faster than the page does normally or faster than a user could reasonably trigger.
- Consider adding random delays to your script's requests, so that they are not uniformly timed.
- Beware of constantly changing fields in the page's requests. Be sure that you know how to generate the correct value; don't just cut and paste.
For changes that you make to the page's code:
The page can handle these either entirely locally, with it's own javascript, or it can also send back status or code fragments to the server.
Block and replace the page's javascript. Tools like NoScript, RequestPolicy, firewalls and proxies can be used to block the JS.
Then use Greasemonkey to add your edited version of the page's JS. Greasemonkey will run even if all of a page's JS is disabled.
If the page phones status home, and that status changes in response to your script's changes, Intercept and replace those messages. (This is not needed if you used the previous method.) You will need a custom firewall, proxy, or router to do this.
For events that you trigger (mouse clicks, filling out fields, etc.):
- If the page checks for this, give it what it looks for. For example, instead of a
click
event, you may need a mouseover
, mousedown
, mouseup
, mouseout
sequence.
- If that is not enough, then the countermeasures are the same as for changes made to the page's code (replace JS, intercept status messages).
Important!
In the rare event that you are battling a site that tries to thwart userscripts, it is like a war. The webmaster can adapt to what you are doing, and the environment can constantly change. So scripting for such a site is an ongoing process.
Scripts are executed on user's side - your side. Therefore site's server code cannot know if you run anything or not.
There's two things to note however:
- Site can deliver some script to be run on your computer that will inspect its environment and will report back if necessary. Those are most often targeted to detect some specific script that site owner don't want to have in its environment.
- Site author can inspect your requests and analyze if they don't match patterns that would match regular communication from site supplied forms or AJAX. Missing keys or keys not matching hardcoded order in query, headers that identify automation clients or missing browser headers and things like that. Hitting throttling limits if you do requests automatically much faster than human user should fits this category too.
Otherwise if you generate exactly the same responses and don't leave traces in globally accessible environment, it is impossible for site to tell if you're using userscript with either server or client side checks.