Tips for debugging .htaccess rewrite rules

Question

Many posters have problems debugging their RewriteRule and RewriteCond statements within their .htaccess files. Most of these are using a shared hosting service and therefore don't have access to the root server configuration. They cannot avoid using .htaccess files for rewriting and cannot enable a RewriteLogLevel" as many respondents suggest. Also there are many .htaccess-specific pitfalls and constraints are aren't covered well. Setting up a local test LAMP stack involves too much of a learning curve for most.

So my Q here is how would we recommend that they debug their rules themselves. I provide a few suggestions below. Other suggestions would be appreciated.

1. Understand that the mod_rewrite engine cycles through .htaccess files. The engine runs this loop:

   do
     execute server and vhost rewrites (in the Apache Virtual Host Config)
     find the lowest "Per Dir" .htaccess file on the file path with rewrites enabled
     if found(.htaccess)
        execute .htaccess rewrites (in the user's directory)
   while rewrite occurred
   

   So your rules will get executed repeatedly and if you change the URI path then it may end up executing other .htaccessfiles if they exist. So make sure that you terminate this loop, if necessary by adding extra RewriteCond to stop rules firing. Also delete any lower level .htaccess rewrite rulesets unless explicitly intent to use multi-level rulesets.

2. Make sure that the syntax of each Regexp is correct by testing against a set of test patterns to make sure that is a valid syntax and does what you intend with a fully range of test URIs. See answer below for more details.

3. Build up your rules incrementally in a test directory. You can make use of the "execute the deepest .htaccess file on the path feature" to set up a separate test directory (tree) and debug rulesets here without screwing up your main rules and stopping your site working. You have to add them one at a time because this is the only way to localise failures to individual rules.

4. Use a dummy script stub to dump out server and environment variables. (See Listing 2)If your app uses, say, blog/index.php then you can copy this into test/blog/index.php and use it to test out your blog rules in the test subdirectory. You can also use environment variables to make sure that the rewrite engine in interpreting substitution strings correctly, e.g.

   RewriteRule ^(.*) - [E=TEST0:%{DOCUMENT_ROOT}/blog/html_cache/$1.html]
   

   and look for these REDIRECT_* variables in the phpinfo dump. BTW, I used this one and discovered on my site that I had to use %{ENV:DOCUMENT_ROOT_REAL} instead. In the case of redirector looping REDIRECT_REDIRECT_* variables list the previous pass. Etc..

5. Make sure that you don't get bitten by your browser caching incorrect 301 redirects. See answer below. My thanks to Ulrich Palha for this.

6. The rewrite engine seems sensitive to cascaded rules within an .htaccess context, (that is where a RewriteRule results in a substitution and this falls though to further rules), as I found bugs with internal sub-requests (1), and incorrect PATH_INFO processing which can often be prevents by use of the [NS], [L] and [PT] flags.

Any more comment or suggestions?

Listing 1 -- phpinfo

<?php phpinfo(INFO_ENVIRONMENT|INFO_VARIABLES);

Answer 1

If you're creating redirections, test with curl to avoid browser caching issues. Use -I to fetch http headers only. Use -L to follow all redirections.

Answer 2

If you planning on writing more than just one line of rules in .htacesss,
don't even think about trying one of those hot-fix methods to debug it.

I have wasted days on setting multiple rules without feedback from LOGs, only to finally giving up on one.
I got Apache on my PC, copied the whole site to HDD, and had the whole rule-set sorted out using logs real fast.
Then I reviewed my old rules which been working, I saw they are not really doing what was desired. A time bomb for a little different address.

There are so many pit falls in rewrite rules, it's not a straight logic thing at all.
You can get Apache up and running in ten minutes, it's 10MB, good license, *NIX/WIN/MAC ready, even without install.
Also, check the header lines of your server and get the same version of Apache from archive if it's old. My OP is still on 2.0, many things are not supported.

Answer 3

Make sure you use the percent sign in front of variables, not the dollar sign.

It's %{HTTP_HOST}, not ${HTTP_HOST}. There will be nothing in the error_log, there will be no Internal Server Errors, your regexp is still correct, the rule will just not match. This is really hideous if you work with django / genshi templates a lot and have ${} for variable substitution in muscle memory.

Answer 4

(Similar to Doin idea) To show what is being matched, I use this code

$keys = array_keys($_GET);
foreach($keys as $i=>$key){
    echo "$i => $key <br>";
}

Save it to r.php on the server root and then do some tests in .htaccess
For example, i want to match urls that do not start with a language prefix

RewriteRule ^(?!(en|de)/)(.*)$ /r.php?$1&$2 [L] #$1&$2&...
RewriteRule ^(.*)$ /r.php?nomatch [L] #report nomatch and exit

Answer 5

Don't forget that in .htaccess files it is a relative URL that is matched.

In a .htaccess file the following RewriteRule will never match:

RewriteRule ^/(.*)     /something/$s

Answer 6

One from a couple of hours that I wasted:

If you've applied all these tips and are only going on 500 errors because you don't have access to the server error log, maybe the problem isn't in the .htaccess but in the files it redirects to.

After I had fixed my .htaccess-problem I spent two more hours trying to fix it some more, even though I simply had forgotten about some permissions.