mod_rewrite NE flag - When is it helpful to encode

Question

I've been looking at the [NE] (noescape) flag in mod_rewrite. After some thought I couldn't figure out a situation when I would NOT want to use the flag. Meaning, it seems most helpful to keep the flag enabled in almost every RewriteRule. Not invoking this flag has caused me problems in a few circumstances.

Most of the rules that I deal with are HTTP redirects ([R]), rather than passing through.

Would someone shed some light as to when it is helpful to have mod_rewrite encode the URL?

Is it generally good practice to enable this flag, or use the default behavior of allowing mod_rewrite escape these special characters? Why?

Answer 1

The [NE] flag is extremely useful when you are adding the request url as part of - let's say - an authorization signature.

I just had a bug where authorization was working with .htaccess off but not with it on. It turned out the cause was that redirection was url encoding the items that ended up in the php $_GET parameter. To solve the bug I changed:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*[^/0-9])$ $1/ [R=301,L]

to

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*[^/0-9])$ $1/ [NE,R=301,L]

(the authorization signature is composed of many things, one of these is the request url)

Answer 2

If you look at the source code for mod_rewrite, you'll notice that it sets a proxy-nocanon flag if noescape is enabled.

In the revision where that line was first added, it also included this comment:

make sure that mod_proxy_http doesn't canonicalize the URI, and preserve any (possibly qsappend'd) query string in the filename for mod_proxy_http:proxy_http_canon()

Following on from that, if you read the mod_proxy documentation, you'll see the following mention of nocanon:

Normally, mod_proxy will canonicalise ProxyPassed URLs. But this may be incompatible with some backends, particularly those that make use of PATH_INFO. The optional nocanon keyword suppresses this, and passes the URL path "raw" to the backend. Note that may affect the security of your backend, as it removes the normal limited protection against URL-based attacks provided by the proxy.

I'm may be mistaken, but that implies to me that the use of nocanon in mod_proxy (and by extension noescape in mod_rewrite) has potential security ramifications. That would explain why it is disabled by default, even thought it seems like it would be more useful to have it enabled in most cases.