URL Shortener Spam

问题:

Lately I'm starting to receive spam URLs into my forum software via URL shortener. In the past, they are easy to block by domain, but I can't exactly block the domain for URL shortener since I'm seeing some valid usage as well.

Any best practice on how to manage these spam URLs (and spammers that are using it)?

回答1:

You could also do something like http://www.untiny.me/ when a post is submitted. Basically, send an HTTP request and capture the "HTTP 301" response. You can then examine the URL you're being redirected to and deny/allow based on your policy.

For example:

C:\>wget -O NUL http://tiny.pl/htk
--2011-01-24 21:47:33--  http://tiny.pl/htk
Resolving tiny.pl... 79.96.141.187
Connecting to tiny.pl|79.96.141.187|:80... connected.
HTTP request sent, awaiting response... 301           <====
Location: http://www.google.com [following]           <====
--2011-01-24 21:47:34--  http://www.google.com/

Relevant lines marked by <====

回答2:

Some form of CAPCHA might be a good idea. If you required registration prior to allowing a user to post, stick it in your registration form. Otherwise, maybe stick it wherever the user posts from as a requirement prior to accepting a post.

回答3:

You could allow the shorteners that only point back to their own sites, like Twitter's or Github's. Another way is suggest your users to use http://9to.one , I developed the logic that will block spammer to shorten their link. You could try to shorten one of those link to test.