ISAPI_Rewrite 2.5 documentation

Introduction

ISAPI_Rewrite is a powerful regular expressions-based URL manipulation engine. It acts mostly like Apache's mod_Rewrite, but it is designed especially for Microsoft Internet Information Server. If you ever wanted to change your web site's URL scheme, this product is for you!

Some key benefits of ISAPI_Rewrite:

• Speed

  ISAPI_Rewrite is extremely fast and highly scalable solution. It is written by using only pure C/C++ code, Win32 API and ISAPI. It uses intelligent configuration cache mechanism. All work is done just in one stage and there are no recursively requests or any other operations that may take a long time.

• Security

  ISAPI_Rewrite is designed for operation in a shared environment. It can serve as many sites as you have. ISP and hosting providers can safely permit their users to configure ISAPI_Rewrite and be sure that any configuration changes will affect only local users environment. ISAPI_Rewrite can even solve many security problems, for example, block an access to some folders or file extensions or create more complex rules.

• Power

  Flexibility and power of ISAPI_Rewrite come form its regular expression nature. With regular expressions you don't need to write a thousands check strings. The comparison and replace of URLs can be done with a few string patterns. So, ISAPI_Rewrite can do many things that cannot be done using other technology solutions available for IIS. See examples section for more information.

Main concept

ISAPI_Rewrite provides a rule-based rewriting engine to rewrite requested URLs on the fly. It supports virtually unlimited number of the rules and an unlimited number of attached rule conditions to provide a really flexible and powerful URL manipulation mechanism (Really a config file size is forcibly limited to 2Mb to prevent possible config parsing overhead). The URL manipulations can depend on tests for HTTP headers, Server variables, Request-URI, method and version information of a client request.

This program operates on Request-URI (path information plus query string) and HTTP headers as it described in RFC 2068 both in server (global) or site context. The result of operation can lead to rewriting, proxiing, redirection, or blocking the original request. For example if client requests resource as http://www.somesite.com/path/file.ext?parameter=value ISAPI_Rewrite will operate on the part marked in red. In addition ISAPI_Rewrite can rewrite, create or remove any other HTTP header of the request.

The rewriting engine loops through the ruleset rule by rule (RewriteRule directives). The particular rule is applied only if it matches against URI and all corresponding conditions (RewriteCond directives) matches against their test strings. ISAPI_Rewrite uses MATCH algorithm. It means that pattern expression is NOT searched in the string, it is matched against the whole input string. If rule is applied ISAPI_Rewrite continues to loop through the ruleset with new URI until the last rule will be processed.

Rewriting will cause server to continue request processing with new URI as if it has been the originally requested by a client. New URI can include query string section (following question mark) and may direct to any files, script calls, program invocations etc.

Proxiing causes the result URI to be internally forced as a proxy request and immediately (i.e., rewriting rule processing stops here) put through the ISAPI extension that handles proxy requests. You have to make sure that the substitution string is a valid URI including protocol, host etc. or proxy will return an error.

Redirection will cause server to send immediate response to client with redirect instruction (HTTP response code 302 with Location header), providing result URI as a new location. You can use absolute links (that is required by RFC 2068) in redirect instruction to redirect request to different host, port and protocol. Redirect instruction always causes rewriting engine to stop the processing sequence.

Rules are processed in the order as they appear in configuration file. ISAPI_Rewrite processes server-level (global) rules first and then it processes an individual virtual site rules if specified. There are no recursively requests or subsequent rollbacks in processing order, so you will never get into an infinite loop.

ISAPI_Rewrite saves original path info + query string before any manipulation on the URL in HTTP header named X-Rewrite-URL. Then it can be retrieved in ASP with Request.ServerVariables("HTTP_X_REWRITE_URL").

Whenever you use parentheses in regular expression patterns, back-references are internally created which can be used withing the format string (using $N syntax) or withing the other patterns (using \N syntax). The references are global for the entire RewriteRule directive and corresponding RewriteCond directives. Sub matches are numbered from up to down and from left to right beginning with the first RewriteCond directive (if such is exists) corresponding to the RewriteRule directive.

To simplify rules and strengthen server security it is strongly recommended that you disable parent paths in the IIS settings.

Installation

Automatic installation

Commonly you install ISAPI_Rewrite by running one of the installation packages and following instructions in the wizard. Installation program will copy files and register COM-objects, ISAPI filters and ISAPI extensions automatically. No other manual steps required to complete installation.

Manual installation

Only Full version of ISAPI_Rewrite supports manual installation. You will need to download distinct manual installation package containing all required files. You could install the whole features or only a part of them.

Minimal installation includes only the filter. You should copy ISAPI_Rewrite.dll file to the target machine and register it as an ISAPI filter using IIS MMC snap-in. ISAPI_Rewrite.dll could be registered either globally or at a particular site level (but not both). Filter will try to load main configuration file (httpd.ini) from the folder where it is located. Site level configurations are loaded from the site's roots.

It is also recommended to register message source dll providing text desciptions of ISAPI_Rewrite's events to the Event Viewer. This could be done by launching provided regmsg.vbs script from the folder where RewriteMsg.dll will be located.

To enable monitoring of a site root path changes register COM server mtbnotif.dll with a command regsvr32 mtbnotif.dll.

Proxy module requires WinHTTP 5.1 to be installed on the server. It is already present in the Windows Server 2003, in the Windows XP since SP1 and in the Windows 2000 since SP3. To install it on the earlier versions of the Windows 2000 and Windows XP or on the NT4 you could either use an automatic installation or download and install SOAP Toolkit 3 SDK from http://download.microsoft.com/download/2/e/0/2e068a11-9ef7-45f5-820f-89573d7c4939/soapsdk.exe. On the Windows Server 2003 you should place file rwhelper.dll.manifest near the rwhelper.dll module. Do not install manifest file on the other OSes. Configuration of the proxy module is described in the Configuration Utility section of this manual.

To register ISAPI_Rewrite use ISRWConfig.exe utility.

Lite version limitation

Lite and Full versions of ISAPI_Rewrite shares the same rewriting engine and source code. But Lite version doesn't support for per-site configuration and all relative features, only global rules are processed. In addition Lite version does not include proxiing engine. The following directives is not supported in Lite version: RewriteProxy, EnableRewrite, DisableRewrite, EnableConfig, DisableConfig, CacheClockRate. The following flags is not supported in Lite version: P - proxy.

Special notes for the IIS6

These special notes concern new features of the Internet Information Server 6.0 built-in into the Windows 2003 Server and limitations imposed by those features upon the ISAPI_Rewrite functionality.

The main difference of the IIS6 from it's ancestors is a new default process model called Worker Process Isolation (WPI) mode. Also IIS6 could operate in the IIS5-compatibility mode (which have no effect on the ISAPI_Rewrite's functionality) it's main advantages could be achived only in the WPI mode.

In the WPI mode virtual web sites or even individual web applications are running inside an Application Pools. And each application pool is served by one or more isolated worker processes w3wp.exe. It looks like High isolation mode in the IIS5 but there exists one significant difference - filters are not running inside the inetinfo.exe process anymore. They running inside a worker processes as an usual applications.

It means that there could be multiple instances of a single filter (one instance for each worker process). Nevertheless this is not a problem for the ISAPI_Rewrite. But if, for example, two web applications http://mysite/app1 and http://mysite/app2 are running in different application pools than rewriting of URLs from /app1 to the /app2 will be prohibited in spite of both applications belong to the same web site http://mysite. Only redirect or proxy could be safely used in this case. Nevertheless, usually this restriction does not have any significant impact on the ISAPI_Rewrite usage since in most cases the whole web site contains a single web application served by a single application pool.

Configuration

Permissions required to run ISAPI_Rewrite

On Windows NT4, Windows 2000, Windows XP and Windows 2003 in IIS5 compatibility mode filter runs in the inetinfo.exe process under the System account. Thus System account should be given at least Read access to all ISAPI_Rewrite dlls and all httpd.ini files. We also recommend to give System account Modify permissions on all folders with httpd.ini files. It will allow creation of httpd.parse.errors files containing config files parsing errors. Additional permissions may be required for the proxy module. Since it could be running in the Pooled or High-isolated application modes, accounts of the IIS shared pool and high isolation pools should be given Read permissions to the rwhelper.dll. By default IWAM_<ComputerName> is used for all pools. Pool account could be found in the corresponding COM+ application settings using COM+ Administration MMC snap-in.

On Windows 2003 in native IIS mode both filter and proxy run in the w3wp.exe worker process corresponding to an application pool hosting particular web application. By default w3wp process runs under the "Network Service" account. But since it could be changed it would be better to find used account in the application pool settings with a help of IIS MMC snap-in. All application pools accounts should be given at least Read permissions to all ISAPI_Rewrite dlls, rwhelper.dll.manifest and all httpd.ini files. We also recommend to give those accounts Modify permissions on the directories containing httpd.ini files to allow httpd.parse.errors files creation.

Configuration file format

There are two types of configuration files - global (server-level) and individual (site-level) files. The global configuration file should be named httpd.ini and should appear in the ISAPI_Rewrite installation directory. The shortcut of this file is provided through the start menu. The individual configuration files should be named httpd.ini and could appear in physical root directories of virtual sites. Both file types formats are the same and it is the standard Windows INI file braked by sections. The only section allowed in this version of ISAPI_Rewrite is [ISAPI_Rewrite]. All directives should be placed in this section and each directive should be placed on a separate line. Any text outside this section will be ignored.

httpd.ini file example:

[ISAPI_Rewrite]

# This is a comment

# 300 = 5 minutes
CacheClockRate 300
RepeatLimit 20

# Block external access to the httpd.ini and httpd.parse.errors files
RewriteRule /httpd(?:\.ini|\.parse\.errors) [F,I,O]

# Block external access to the Helper ISAPI Extension
RewriteRule .*\.isrwhlp [F,I,O]

# Some custom rules
RewriteCond Host: (.+)
RewriteRule (.*) /$1$2 [I]

When ISAPI_Rewrite parses configuration file it creates error log file named httpd.parse.error in the same directory where parsed file is located.

RewriteCond directive

The RewriteCond directive defines a rule condition. Precede a RewriteRule or RewriteHeader or RewriteProxy directive with one or more RewriteCond directives. The following rule applied only if its pattern matches the current state of the URI and if these additional conditions apply too.

• TestVerb

  Specifies verb that will be matched against regular expression.

  TestVerb=(URL | METHOD | VERSION | HTTPHeaderName: | %ServerVariable) where:

  • URL - returns Request-URI of client request as described in RFC 2068 (HTTP 1.1);
  • METHOD - returns HTTP method of client request (OPTIONS, GET, HEAD, POST, PUT, DELETE or TRACE);
  • VERSION - returns HTTP version;
  • HTTPHeaderName - returns value of the specified HTTP header. HTTPHeaderName can be any valid HTTP header name. Header names should include the trailing colon ":". If specified header does not exists in a client's request  TestVerb is treated as empty string.

    HTTPHeaderName = Accept: Accept-Charset: Accept-Encoding: Accept-Language: Authorization: Cookie: From: Host: If-Modified-Since: If-Match: If-None-Match: If-Range: If-Unmodified-Since: Max-Forwards: Proxy-Authorization: Range: Referer: User-Agent: Any-Custom-Header:

  For more information about HTTP headers and their values refer to RFC 2068.

  • ServerVariable - returns value of the specified Server Variable. For examlpe, SERVER_PORT. Complete list of the server variables could be found in the IIS documentation. Variable name should be prefixed with % sign.
• CondPattern

  The regular expression to match TestVerb.

• [Flags]

  Flags is a comma-separated list of the following flags:
  
  

  • O (nOrmalize)

    Normalizes string before processing. Normalization includes removing of an URL-encoding, illegal characters, etc. This flag is useful with URLs and URL-encoded headers.

RewriteRule directive

The RewriteRule directive is the real rewriting workhorse. The directive can occur more than once. Each directive defines one single rewriting rule. The definition order of these rules is important, because this order is used when applying the rules at run-time.

• Pattern

  Specifies regular expression that will be matched against Request-URI. See regular expression syntax section for more information.

• FormatString

  Specifies format string that will generate new URI. See format string syntax section for more information.

• [Flags]

  Flags is a comma-separated list of the following flags:

  • I (ignore case)

    Indicates that characters are matched regardless of case. This flag affects RewriteRule directive and all corresponding RewriteCond directives.

  • F (Forbidden)

    Stops the rewriting process and sends 403 Forbidden response to a client. Note that FormatString is useless in this case and could be set to any non-empty string.

  • L (last rule)

    Stop the rewriting process here and don't apply any more rewriting rules. Use this flag to prevent the currently rewritten URI from being rewritten further by following rules.

  • N (Next iteration)

    Forces rewriting engine to modify rule's target and restart rule checking from the beginning (all modifications are saved). Number of restarts is limited by the value specified in the RepeatLimit directive. If this number is exceeded N flag will be simply ignored.

  • P (force proxy)

    Forces the result URI to be internally forced as a proxy request and immediately (i.e., rewriting rule processing stops here) put through the ISAPI extension that handles proxy requests. You have to make sure that the substitution string is a valid URI including protocol, host etc. or proxy will return an error.

  • R (explicit redirect)

    Force server to send immediate response to client with redirect instruction, providing result URI as a new location. Redirect rule is always the last rule.

  • RP (permanent redirect)

    Almost the same as the [R] flag but issues 301 (moved permanently) HTTP status code instead of 302 (moved temporary).

  • U (Unmangle Log)

    Log the URL as it was originally requested and not as the URL was rewritten.

  • O (nOrmalize)

    Normalizes string before processing. Normalization includes removing of an URL-encoding, illegal characters, etc. This flag is useful with URLs and URL-encoded headers.

RewriteHeader directive

The RewriteHeader directive is more general variant of RewriteRule directive and it is designed to rewrite not only the URL part of client request, but any HTTP header. This directive can be used to rewrite, create or delete any HTTP headers, or even change method of the client request.

• HeaderName 

  Specifies a HTTP header that will be rewritten. Possible values are the same as for the TestVerb parameter in the RewriteCond directive. Thus, RewriteRule directive is a synonym to the RewriteHeader URL Pattern Format [Flags]

• Pattern

  Specifies regular expression that will be matched against specified header. See regular expression syntax section for more information.

• FormatString

  Specifies format string that will generate new header value. See format string syntax section for more information.

• [Flags]

  Flags is a comma-separated list of the following flags:

  • I (ignore case)

    Indicates that characters are matched regardless of case. This flag affects RewriteHeader directive and all corresponding RewriteCond directives.

  • F (Forbidden)

    Stops the rewriting process and sends 403 Forbidden response to a client. Note that FormatString is useless in this case and could be set to any non-empty string.

  • L (last rule)

    Stop the rewriting process here and don't apply any more rewriting rules.

  • N (Next iteration)

    Forces rewriting engine to modify rule's target and restart rule checking from the beginning (all modifications are saved). Number of restarts is limited by the value specified in the RepeatLimit directive. If this number is exceeded N flag will be simply ignored.

  • R (explicit redirect)

    Force server to send immediate response to client with redirect instruction, providing new URI as a new location. Redirect rule is always the last rule.

  • RP (permanent redirect)

    Almost the same as the [R] flag but issues 301 (moved permanently) HTTP status code instead of 302 (moved temporary).

  • U (Unmangle Log)

    Log the URL as it was originally requested and not as the URL was rewritten.

  • O (nOrmalize)

    Normalizes string before processing. Normalization includes removing of an URL-encoding, illegal characters, etc. This flag is useful with URLs and URL-encoded headers.

To remove header, format string pattern should generate an empty string. For example this rule will remove user agent information from the client request:

RewriteHeader User-Agent: .* $0

And this rule will add Old-URL header to the request, providing a Request-URL as a header value:

RewriteCond URL (.*)
RewriteHeader Old-URL: ^$ $1

This last example will direct all WebDAV requests to the /webdav.asp script by changing request method:

RewriteCond METHOD OPTIONS
RewriteRule (.*) /webdav.asp?$1
RewriteHeader METHOD OPTIONS GET

RewriteProxy directive

Forces the result URI to be internally forced as a proxy request and immediately (i.e., rewriting rule processing stops here) put through the ISAPI extension that handles proxy requests. This allow IIS to act as a proxy server and reroute requests to other sites and servers.

• Pattern

  Specifies regular expression that will be matched against Request-URI. See regular expression syntax section for more information.

• FormatString

  Specifies format string that will generate new URI. See format string syntax section for more information. For this directive FormatString should generate a valid URL (including protocol, host, etc.) or ISAPI_Rewrite proxy module will return an error.

• [Flags]

  Flags is a comma-separated list of the following flags:

  • D (Delegate security)

    Proxy module will try to login on the remote server with the credentials of currently impersonated user. In case of built-in IIS authentification it will be credentials of the user that sends original request.

  • C (use Credentials)

    Proxy module will try to login on a remote server with the credentials specified in the URL or basic authentification headers. With this flag you can use http://user:password@host.com/path/ syntax for the URLs.

  • F (Follow redirects)

    By default ISAPI_Rewrite will try to map redirect instructions (301,302,etc.) returned by a remote server into the local server namespace. If remote server returns a redirect pointing to the location somewhere on that server, ISAPI_Rewrite will modify this redirect instruction to point to the local server name. This will avoid for user to see the real (internal) server name.

    Use F flag to force the proxy module to internally follow redirect instructions returned from the remote server. Use this flag if you don't need to receive redirects from remote server at all. There is a redirect limit in WinHTTP settings to prevent infinite redirection loops.

  • I (ignore case)

    Indicates that characters are matched regardless of case. This flag affects RewriteProxy directive and all corresponding RewriteCond directives.

  • U (Unmangle Log)

    Log the URL as it was originally requested and not as the URL was rewritten.

  • O (nOrmalize)

    Normalizes string before processing. Normalization includes removing of an URL-encoding, illegal characters, etc. This flag is useful with URLs and URL-encoded headers.

CacheClockRate directive

This directive can appear only in global configuration context. If this directive is found in a site-level context it will be ignored and an error message will be written to a httpd.parse.errors file.

ISAPI_Rewrite caches every configuration file at first time it is loaded. Using this directive you can specify period of inactivity of particular site when it's configuration will be purged from cache. By setting this parameter big enough you can force ISAPI_Rewrite to never recycle its cache. Remember that any changes to configuration files update cache immediately after the next request regardless of this interval.

• Interval

  Specifies time of inactivity (in seconds) when particular configuration will be purged from cache. The default value is 3600 (1 hour).

EnableConfig and DisableConfig directives

Enables or disables site-level configurations for a selected site or changes the defaults. By default site-level configurations are enabled. This directive can appear only in global configuration context. If found in a site-level context it will be ignored and an error message will be written to a httpd.parse.errors file.

• SiteID

  Numeric metabase identifier of a site

• Site name

  Name of the site as it appears in the IIS console.

If used without parameter this directives turn default settings to enable/disable configuration processing.

Example:

The following will enable configuration only for the site with ID=1 (tipically it is Default Web Site) and a site nemed "My site":

DisableConfig
EnableConfig 1
EnableConfig"My site"

In the following example configurations are enabled for the site named "Some site" because explicit settings overrides the default settings.

EnableConfig"Some site"
DisableConfig

EnableRewrite and DisableRewrite directives

Enables or disables rewriting for a selected site or changes the defaults. By default rewriting is enabled. This directive can appear only in global configuration context. If found in a site-level context it will be ignored and an error message will be written to a httpd.parse.errors file.

• SiteID

  Numeric metabase identifier of a site

• Site name

  Name of the site as it appears in the IIS console.

RepeatLimit directive

This directive could appear both in global and in site-level configuration files. If it will appear in the global configuration file it will change the global limit for all sites. If this directive will appear in a site-level configuration file it will change a limit for this site only and this limit could not exceed the global limit.

ISAPI_Rewrite allows loops while processing rules (see the description of the N flag of the RewriteRule and RewriteHeader directives). This directive allows to limit the maximum number of possible loops. It could be set to zero or one to disable looping.

• Limit

  Specifies a maximum number of allowed loops. The default value is 32.

RFStyle directive

The RFStyle directive allows control over redirect and forbidden responses style in a Full version of ISAPI_Rewrite. The default Old style is to issue those responses directly from the filter. This method is fast and requires no additional configuration. But in this case original requests could not be logged by IIS. New style forces filter to issue redirect and forbidden responces through a proxy module. This technique makes possible to log redirected and forbidden requests. But this method would require manual configuration in many cases. Details of the proxy module configuration could be found in the following chapter.

Configuration Utility

• Trial page.

  This page will be shown only if you have installed ISAPI_Rewrite in a trial mode. It allows you to view product's trial status and to enter a registration code.

• Settings page.

  This page contains edit boxes for the following parameters:

  • Helper URL

    This parameter affects a way of communication between the filter and the proxy module. This could be either a file extension prefixed with a dot (like .isrwhlp) or an absolute URI (like /isrwhlp/rwhelper.dll).

    In the first case that extension will be appended to the original request URI and proxy module will be invoked via a script map. The default extension ".isrwhlp" is added to the global script map during the installation process. If you have changed this extension or if your application does not inherit global script map settings you should manually add a required entry to the script map. It should have the following parameters:

    Executable:	An absolute path to the rwhelper.dll in the short form
    Extension:	Desired extension (.isrwhlp is default)
    Verbs radio button:	All Verbs
    Script engine checkbox:	Checked
    Check that file exists checkbox:	Unchecked

    We have created a WSH script proxycfg.vbs which could simplify helper extension registration in a script maps. It's located in the ISAPI_Rewrite installation folder and could be run from command line in this way:

    cscript proxycfg.vbs [-r] [MetabasePath]

    • Optional -r parameter forces script to register extension in the metabase recursively.
    • Optional MetabasePath parameter allows specification of the first metabase key to process. By default it is "/localhost/W3SVC".

    To register extension in all existing script maps you could invoke script with the following command string:
    

    cscript proxycfg.vbs -r

    In the second case (absolute URI) you should provide an URI like /vfolder/rwhelper.dll for a value of the 'Helper URL'. You should also map an ISAPI_Rewrite's installation folder as a choosen virtual folder (/vfolder) to an each site which would use proxiing and enable executables launching for these virtual folders.

    Note: As it was reported by our customers, IIS 5 (and may be IIS 4) has a problem with a long directory names in the Helper URL. So, it is strongly recommended to use short directory names.

    Both methods have advantages and drawbacks. The second one allows you to isolate proxy application from other applications but this method could fail under IIS6. The first one should work fine under IIS6 but the proxy module will be invoked in a context of an application targeted by an original request URL.

  • Worker threads limit

    This parameter limits maximum number of the worker threads in the proxy extension threads pool. Default value 0 means that this limit will be equal to the number of processors multiplied by 2.

  • Active threads limit

    This parameter limits maximum number of the concurrently running worker threads. This value could not be greater than "Worker threads limit". Default value 0 means that this limit will be equal to the number of processors.

  • Queue size

    This parameter defines the maximum number of request which could be queued for processing by the single instance of the proxy module. You could increase this parameter if you will ever see "Queue timeout expired" message in the Application event log.

  • Queue timeout

    This parameter defines the maximum wait time for placing new request into the internal request queue. It becomes actual when the number of queued request reaches it's maximum defined by the "Queue size" parameter. You could increase this parameter if you will ever see "Queue timeout expired" message in the Application event log.

  • Connect timeout

    Specifies connect timeout for the proxy module in milliseconds. Default value is 60000 (60 sec).

  • Send timeout

    Specifies send timeout for the proxy module in milliseconds. Default value is 30000 (30 sec).

  • Receive timeout

    Specifies receive timeout for the proxy module in milliseconds. Default value is 30000 (30 sec).

• About page.

  It contains copyright information and a link to the ISAPI_Rewrite's web site.

Regular expression syntax

Literals

All characters are literals except: ".", "*", "?", "+", "(", ")", "{", "}", "[", "]", "^" and "$". These characters are literals when preceded by a "\". A literal is a character that matches itself.

Wildcard

The dot character "." matches any single character except null character and newline character. 

Repeats

A repeat is an expression that is repeated an arbitrary number of times. An expression followed by "*" can be repeated any number of times including zero. An expression followed by "+" can be repeated any number of times, but at least once. An expression followed by "?" may be repeated zero or one times only. When it is necessary to specify the minimum and maximum number of repeats explicitly, the bounds operator "{}" may be used, thus "a{2}" is the letter "a" repeated exactly twice, "a{2,4}" represents the letter "a" repeated between 2 and 4 times, and "a{2,}" represents the letter "a" repeated at least twice with no upper limit. Note that there must be no white-space inside the {}, and there is no upper limit on the values of the lower and upper bounds. All repeat expressions refer to the shortest possible previous sub-expression: a single character; a character set, or a sub-expression grouped with "()" for example.

Examples:

• "ba*" will match all of "b", "ba", "baaa" etc.
• "ba+" will match "ba" or "baaaa" for example but not "b".
• "ba?" will match "b" or "ba".
• "ba{2,4}" will match "baa", "baaa" and "baaaa".

Non-greedy repeats

Non-greedy repeats are possible by appending a '?' after the repeat; a non-greedy repeat is one which will match the shortest possible string.

For example to match html tag pairs one could use something like:

"<\s*tagname[^>]*>(.*?)<\s*/tagname\s*>"

In this case $1 will contain the text between the tag pairs, and will be the shortest possible matching string. 

Parenthesis

Parentheses serve two purposes, to group items together into a sub-expression, and to mark what generated the match. For example the expression "(ab)*" would match all of the string "ababab". All sub matches marked by parenthesis can be back referenced using \N or $N syntax. It is permissible for sub-expressions to match null strings. Sub-expressions are indexed from left to right starting from 1, sub-expression 0 is the whole expression.

Non-Marking Parenthesis

Sometimes you need to group sub-expressions with parenthesis, but don't want the parenthesis to spit out another marked sub-expression, in this case a non-marking parenthesis (?:expression) can be used. For example the following expression creates no sub-expressions:

"(?:abc)*" 

Alternatives

Alternatives occur when the expression can match either one sub-expression or another, each alternative is separated by a "|". Each alternative is the largest possible previous sub-expression; this is the opposite behaviour from repetition operators.

Examples:

• "a(b|c)" could match "ab" or "ac".
• "abc|def" could match "abc" or "def". 

Sets

A set is a set of characters that can match any single character that is a member of the set. Sets are delimited by "[" and "]" and can contain literals, character ranges, character classes, collating elements and equivalence classes. Set declarations that start with "^" contain the compliment of the elements that follow.

Examples:

Character literals:

• "[abc]" will match either of "a", "b", or "c".
• "[^abc] will match any character other than "a", "b", or "c".

Character ranges:

• "[a-z]" will match any character in the range "a" to "z".
• "[^A-Z]" will match any character other than those in the range "A" to "Z".

Character classes

Character classes are denoted using the syntax "[:classname:]" within a set declaration, for example "[[:space:]]" is the set of all whitespace characters. The available character classes are: 

There are some shortcuts that can be used in place of the character classes:

• \w in place of [:word:]
• \s in place of [:space:]
• \d in place of [:digit:]
• \l in place of [:lower:]
• \u in place of [:upper:] 

Collating elements

Collating elements take the general form [.tagname.] inside a set declaration, where tagname is either a single character, or a name of a collating element, for example [[.a.]] is equivalent to [a], and [[.comma.]] is equivalent to [,]. ISAPI_Rewrite supports all the standard POSIX collating element names, and in addition the following digraphs: "ae", "ch", "ll", "ss", "nj", "dz", "lj", each in lower, upper and title case variations. Multi-character collating elements can result in the set matching more than one character, for example [[.ae.]] would match two characters, but note that [^[.ae.]] would only match one character. 

Equivalence classes

Equivalenceclassestakethegeneralform[=tagname=] inside a set declaration, where tagname is either a single character, or a name of a collating element, and matches any character that is a member of the same primary equivalence class as the collating element [.tagname.]. An equivalence class is a set of characters that collate the same, a primary equivalence class is a set of characters whose primary sort key are all the same (for example strings are typically collated by character, then by accent, and then by case; the primary sort key then relates to the character, the secondary to the accentation, and the tertiary to the case). If there is no equivalence class corresponding to tagname, then [=tagname=] is exactly the same as [.tagname.].

To include a literal "-" in a set declaration then: make it the first character after the opening "[" or "[^", the endpoint of a range, a collating element, or precede it with an escape character as in "[\-]". To include a literal "[" or "]" or "^" in a set then make them the endpoint of a range, a collating element, or precede with an escape character. 

Line anchors

An anchor is something that matches the null string at the start or end of a line: "^" matches the null string at the start of a line, "$" matches the null string at the end of a line. 

Back references

A back reference is a reference to a previous sub-expression that has already been matched, the reference is to what the sub-expression matched, not to the expression itself. A back reference consists of the escape character "\" followed by a digit "1" to "9", "\1" refers to the first sub-expression, "\2" to the second etc. For example the expression "(.*)\1" matches any string that is repeated about its mid-point for example "abcabc" or "xyzxyz". A back reference to a sub-expression that did not participate in any match, matches the null string. In ISAPI_Rewrite all back references are global for entire RewriteRule and corresponding RewriteCond directives. Sub matches are numbered up to down and left to right beginning from the first RewriteCond directive of the corresponding RewriteRule directive, if there is one.

Forward Lookahead Asserts

There are two forms of these; one for positive forward lookahead asserts, and one for negative lookahead asserts:

• "(?=abc)" matches zero characters only if they are followed by the expression "abc".
• "(?!abc)" matches zero characters only if they are not followed by the expression "abc".

Word operators

The following operators are provided for compatibility with the GNU regular expression library.

• "\w" matches any single character that is a member of the "word" character class, this is identical to the expression "[[:word:]]".
• "\W" matches any single character that is not a member of the "word" character class, this is identical to the expression "[^[:word:]]".
• "\<" matches the null string at the start of a word.
• "\>" matches the null string at the end of the word.
• "\b" matches the null string at either the start or the end of a word.
• "\B" matches a null string within a word.

Escape operator

The escape character "\" has several meanings.

• The escape operator may introduce an operator for example: back references, or a word operator.
• The escape operator may make the following character normal, for example "\*" represents a literal "*" rather than the repeat operator.

Single character escape sequences:

The following escape sequences are aliases for single characters:
 

Miscellaneous escape sequences:

The following are provided mostly for perl compatibility, but note that there are some differences in the meanings of \l \L \u and \U: 

What gets matched?

The regular expression will match the first possible matching string, if more than one string starting at a given location can match then it matches the longest possible string. In cases where their are multiple possible matches all starting at the same location, and all of the same length, then the match chosen is the one with the longest first sub-expression, if that is the same for two or more matches, then the second sub-expression will be examined and so on. Note that ISAPI_Rewrite uses MATCH algorithm. The result is matched only if the expression matches the whole input sequence. For example:

• RewriteCond URL ^/somedir/.* #will match any request to somedir directory and subdirectories, while
• RewriteCond URL ^/somedir/ #will match only request to the root of the somedir.

Special note about "pathological" regular expressions

ISAPI_Rewrite uses a very powerful regular expressions engine Regex++ written by Dr. John Maddock. But as any real thing it's not ideal: There exists some "pathological" expressions which may require exponential time for matching; these all involve nested repetition operators, for example attempting to match the expression "(a*a)*b" against N letter a's requires time proportional to 2^N. These expressions can (almost) always be rewritten in such a way as to avoid the problem, for example "(a*a)*b" could be rewritten as "a*b" which requires only time linearly proportional to N to solve. In the general case, non-nested repeat expressions require time proportional to N², however if the clauses are mutually exclusive then they can be matched in linear time - this is the case with "a*b", for each character the matcher will either match an "a" or a "b" or fail, where as with "a*a" the matcher can't tell which branch to take (the first "a" or the second) and so has to try both.

Boost 1.29.0 Regex++ could detect "pathological" regular expressions and terminate theirs matching. When a rule fails ISAPI_Rewrite sends "500 Internal Server error - Rule Failed" status to a client to indicate configuration error. Also the failed rule is disabled to prevent performance losses.

Format string syntax

In format strings, all characters are treated as literals except: "(", ")", "$", "\", "?", ":".

To use any of these as literals you must prefix them with the escape character \

The following special sequences are recognized: 

Grouping:

Use the parenthesis characters ( and ) to group sub-expressions within the format string, use \( and \) to represent literal '(' and ')'. 

Sub-expression expansions:

The following perl like expressions expand to a particular matched sub-expression:

Conditional expressions:

Conditional expressions allow two different format strings to be selected dependent upon whether a sub-expression participated in the match or not:

?Ntrue_expression:false_expression

Executes true_expression if sub-expression N participated in the match, otherwise executes false_expression.

Example: suppose we search for "(while)|(for)" then the format string "?1WHILE:FOR" would output what matched, but in upper case.

Escape sequences:

The following escape sequences are also allowed:
 

Examples

Emulating host-header-based virtual sites on a single site

For example you have registered two domains www.site1.com and www.site2.com.  Now you can create two different sites using single physical site. Add the following rules to your httpd.ini file:

[ISAPI_Rewrite]

#Fix missing slash char on folders
RewriteCond  Host:  (.*)
RewriteRule  ([^.?]+[^.?/]) http\://$1$2/ [I,R]

#Emulate site1
RewriteCond  Host:  (?:www\.)?site1\.com
RewriteRule  (.*)   /site1$1 [I,L]

#Emulate site2
RewriteCond  Host:  (?:www\.)?site2\.com
RewriteRule  (.*)   /site2$1 [I,L]

Now just place your sites in /site1 and /site2 directories.

Or you can use more generic rules:

[ISAPI_Rewrite]

#Fix missing slash char on folders
RewriteCond  Host:  (.*)
RewriteRule  ([^.?]+[^.?/]) http\://$1$2/ [I,R]

RewriteCond  Host:  (www\.)?(.+)
RewriteRule  (.*)   /$2$3

The directory names for sites should be like /somesite1.com, /somesite2.info, etc.

Using loops (Next flag) to convert request parameters

Suppose you wish to access physical URLs like http://www.myhost.com/foo.asp?a=A&b=B&c=C using requests like http://www.myhost.com/foo.asp/a/A/b/B/c/C and the number of parameters may vary from one request to another.

There exists at least two possible solutions. You could simply add a separate rule for each possible number of parameters or you could use a technique demonstrated by the following example.

[ISAPI_Rewrite]
RewriteRule (.*?\.asp)(\?[^/]*)?/([^/]*)/([^/]*)(.*) $1(?2$2&:\?)$3=$4$5 [N,I]

This rule will extract one parameter from request URL, append it to the end of the request string and restart rules processing from the beginning. So it will loop until all parameters will be moved to the right place (or until the RepeatLimit will be exceeded).

There also exist many variations of this rule with different separator characters. For example, to use URLs like http://www.myhost.com/foo.asp~a~A~b~B~c~C the following rule could be implemented:

[ISAPI_Rewrite]
RewriteRule (.*?\.asp)(\?[^~]*)?~([^~]*)~([^~]*)(.*) $1(?2$2&:\?)$3=$4$5 [N,I]

Running servers behind IIS

Assume we have internet server running IIS and several corporate servers running other platform. These servers are not directly accessible from the internet but only from our corporate network. Here is a simple example how to map another server into the IIS site’s namespace using proxy flag:

[ISAPI_Rewrite]
RewriteProxy /mappoint(.+) http\://sitedomain$1 [I,U]

Moving sites from UNIX to IIS

This rules can help change the URL from /~username to /username and /file.html to /file.htm. It can be useful if you just moved your site from UNIX to IIS and keep getting hits to the old pages from search engines and other external pages.

[ISAPI_Rewrite]

#redirecting to update old links
RewriteRule (.*)\.html $1.htm
RewriteRule  /~(.*)  http\://myserver/$1 [R]

Moving site location

Many webmasters asked for a solution to the following problem: They want to redirect all requests to one web server to the another web server. Such problems usually arise when you need to establish a newer web server which will replace the old one over time. The solution is to use ISAPI_Rewrite on the old web server:

[ISAPI_Rewrite]

#redirecting to update old links
RewriteRule  (.+)  http\://newwebserver$1 [R]

Browser-dependent content

It is sometimes necessary to provide browser-dependent content at least for important top-level pages, i.e. one has to provide a full-featured version for the Internet Explorer, a minimum-featured version for the Lynx browsers and an average-featured version for all others.

We have to act on the HTTP header "User-Agent". The sample code does the following: If the HTTP header "User-Agent" contains "MSIE", the target foo.htm is rewritten to foo.IE.htm. If the browser is "Lynx" or "Mozilla" of version 1 or 2 the URL becomes foo.20.htm. Other browsers receive page foo.32.html. All this is done by the following ruleset:

[ISAPI_Rewrite]

RewriteCond  User-Agent:  .*MSIE.*
RewriteRule  /foo\.htm  /foo.IE.htm  [L]

RewriteCond  User-Agent:  (?:Lynx|Mozilla/[12]).*
RewriteRule  /foo\.htm  /foo.20.htm  [L]

RewriteRule  /foo\.htm  /foo.32.htm  [L]

Dynamically generated robots.txt

robots.txt is a file that search engines use to discover URLs that should or should not be indexed. But creation of this file for large sites with lot of dynamic content is a very complex task. Have you ever dreamed about dynamically generated robots.txt? Let's write robots.asp script:

<%@ Language=JScript EnableSessionState=False%>
<%

//The script must return plain text
Response.ContentType="text/plain";

/*
Place generation code here
*/

%>

Now make it robots.txt using single rule:

[ISAPI_Rewrite]

RewriteRule  /robots\.txt  /robots.asp

Making search engines to index dynamic pages

Content of the site stored in XML files. There is /XMLProcess.asp file that processes XML files on server and returns HTML to end user. URLs to the documents have a form of:
http://www.mysite.com/XMLProcess.asp?xml=/somdir/somedoc.xml
But many popular search engines will not index such documents because URLs contain question mark (document is dynamically generated). ISAPI_Rewrite can competely eliminate this problem.

[ISAPI_Rewrite]

RewriteRule  /doc(.*)\.htm  /XMLProcess.asp\?xml=$1.xml

Now to access documents use URL like http://www.mysite.com/doc/somedir/somedoc.htm. Search engines will never know that physically there is no somedoc.htm file and content is dynamically generated.

Negative expressions (NOT)

Sometimes you need to apply rule when some pattern not matches. In this case you may use so called Forward Lookahead Asserts in regular expressions.

For example you need to move all users not using Internet Explorer to the other location:

[ISAPI_Rewrite]
# Redirect all non Internet Explorer users
# to another location
RewriteCond  User-Agent: (?!.*MSIE).*
RewriteRule  (.*) /nonie$1

Dynamic authentification

For example we have some members area on the site and we need password-protect files in this area but we don't like to use built-in server security. In this case it is possible to create ASP script (call it proxy.asp) that will proxy all requests to the members area and check for required permissions. Here is a simple template for this page where you can put your own authorisation code:

<%@ Language=JScript EnableSessionState=False%>
<%
function Authorize()
{

  //Check if the user is authorized to view a resource here
  //Return true if user has a required permission, otherwise return false

  return true;
}
if(!Authorize())
{
  //Redirect to the login page
  Response.Redirect("http://mysite.com/LoginPage.asp?ref="+Request.QueryString.Item);
  Response.End()
}
var WinHttpReq = new ActiveXObject("WinHttp.WinHttpRequest.5");
WinHttpReq.Open(Request.ServerVariables("REQUEST_METHOD").Item, Request.QueryString.Item, true);
var headers=String(Request.ServerVariables("ALL_RAW")).split("\n");
for(i=0; i<headers.length && headers[i]; i++)
{
  header = headers[i].match(/([\w-\.]+):\s*([ \S]*)/);
  if(header)
    WinHttpReq.SetRequestHeader(header[1],header[2]);
}
if(lngCount = Request.TotalBytes)
{
  var data=Request.BinaryRead(lngCount);
  WinHttpReq.Send(data);
} else {
  WinHttpReq.Send();
}
if(!WinHttpReq.WaitForResponse(15))
{
  WinHttpReq.Abort();
  Response.Status="408 Request Timeout";
} else {
  Response.Status = "" + WinHttpReq.Status + " " + WinHttpReq.StatusText;
  headers=String(WinHttpReq.GetAllResponseHeaders()).split("\n");
  for(i=0; i<headers.length && headers[i]; i++)
  {
    header = headers[i].match(/([\w-\.]+):\s*([ \S]*)/);
    if(header)
      Response.AddHeader(header[1],header[2]);
  }
  Response.Write(WinHttpReq.ResponseText);
}
%>

Now we need to configure ISAPI_Rewrite to proxy requests through this page:

[ISAPI_Rewrite]
# Proxy all requests through proxy.asp
RewriteRule  /members(.+)  /proxy.asp\?http\://mysite.com/members$1

Blocking inline-images (stop hot linking)

Assume we have some pages with inlined GIF graphics under http://www.mysite.com/. These graphics are nice, so others directly incorporate them via hyperlinks to their pages. We don't like this practice because it adds useless traffic to our server.

While we cannot 100% protect the images from inclusion, we can at least restrict the cases where the browser sends a HTTP Referer header.

[ISAPI_Rewrite]
RewriteCond  Referer: .+
RewriteCond  Referer: (?!http://(www\.)?mysite\.com).*
RewriteRule  .*\.(gif|jpg)  /404.asp [I,L]

Feature history

ISAPI_Rewrite version 2.5 build 46:

• Fixed possible proxy crash on new-style forbid (regression).
• Improved redirects and forbids RFC compliance.

ISAPI_Rewrite version 2.5 build 45:

• Fixed bug with DisableRewrite directive introduced in build 44.

ISAPI_Rewrite version 2.5 build 44:

• Reduced memory consumption by a process executing filter.
• Added advanced error logging to the installation.
• Fixed incorrect loading of the "Helper URL" parameter from registry in the Configuration Utility.

ISAPI_Rewrite version 2.5 build 43:

• Proxy module will change Location header of 3xx responses to point to the proxiing server if it originally points to a proxied server.
• Fixed memory leak and WinHTTP handles leak in the proxy module in case of response timeout.
• Added description of Permissions required to run ISAPI_Rewrite to the documentation.

ISAPI_Rewrite version 2.4 build 42:

• Fixed problem with incorrect WinHTTP timeouts setting for a request in the proxy module.

ISAPI_Rewrite version 2.4 build 41:

• Fixed problem with long querystrings in the proxy module.

ISAPI_Rewrite version 2.3 build 39:

• Incorporated Regex++ fix for non-greedy patterns.

ISAPI_Rewrite version 2.3 build 38:

• Compiled with new Regex library from boost 1.31.0. Matching speed significantly increased.
• Fixed RXTest utility crash on "pathological" regular expressions.

ISAPI_Rewrite version 2.3 build 37:

• Fixed problem with old-style HTTPS redirects and forbids.

ISAPI_Rewrite version 2.2 build 32:

• Added Permanent Redirect [RP] flag to the RewriteRule/RewriteHeader directives
• Added possibility to configure Connect, Send and Receive timeouts for the proxy module through the configuration utility.

ISAPI_Rewrite version 2.2 build 30:

• Proxy module was redesigned and rewritten to use IIS's asynchronous I/O functions.
• Threads pooling strategy was changed:
  • "Threads Spawn Threshold" parameter was replaced with 2 new parameters - "Worker threads limit" and "Active threads limit".
• Added possibility to automatically fix registration keys broken by Outlook to the Configuration Utility.
• Proxy module uses WinHTTP 5.1.

ISAPI_Rewrite version 2.1 build 28:

• Fixed serveral problems related to a new protection system:
  • "Invalid Access to Memory Location" in the proxy module.
  • Some per-site configs may not work.
  • Filter may not work when installed at a site level.
• Added possibility of a manual product installation.
• Proxy module now returns more friendly error messages.

ISAPI_Rewrite version 2.1 build 24:

• Moved to new trial protection system. Fixed several bugs with trial check.
• New directive RewriteProxy brings more flexible proxy features.
• Fixed bug with EnableRewrite/DisableRewrite.
• Proxy module now sends redirect responses to the client.
• Fixed several bugs in the documentation.

ISAPI_Rewrite version 2.0.1 build 22:

• Added script for manual proxy module registration in a script maps (full version only). Look into the Configuration Utility section for details.
• Add new config directive RFStyle for controling of redirect and forbid responses style (full version only). Redirect and forbidden now could be issued in the same way as in 1.3 (old style) or in a way of 2.0 (new style). Default is an old way.

ISAPI_Rewrite version 2.0 build 21:

• Changed "pathological" rules protection. Failed rules are not diabled anymore.

ISAPI_Rewrite version 2.0 build 20:

• Now ISAPI_Rewrite supports proxy throughput. Use [P] flag in RewriteRule or RewriteHeader directive to proxy request. Check the Configuration Utility section of the documentation for details of the proxy module configuration.
• New directives EnableRewrite and DisableRewrite which can be used to enable or disable rewriting globally or for each site separately.
• New directives EnableConfig and DisableConfig intended to enable or disable site level configurations for a selected site.
• Now ISAPI_Rewrite Full has a trial period.
• New helper ISAPI extension intended to serve proxiing, redirecting and blocking requests.
• Using new Regex++ version from Boost 1.29.0 with more bug fixes and new features.

ISAPI_Rewrite version 1.3 build 16:

• Introduced some modifications to the Regex++ regular expressions engine to overcome a problem with "pathological" rules requiring exponential time for processing. Now time to process a single rule is limited to half-second. If a rule fails to complete in this time a processing finishes and ISAPI_Rewrite sends "500 Internal Server error" to a client to indicate configuration error.
• Added new N (Next) flag to the RewriteRule and RewriteHeader directives. It makes possible to organize loops while processing rules.
• Added RepeatLimit directive to limit the number of possible loops.
• Added F (Forbidden) flag to the RewriteRule and RewriteHeader directives. It forces to send 403 Forbidden response to a client if a positive match detected.
• Added O (nOrmalize) flag to the RewriteRule, RewriteHeader and RewriteCond directives. It points out that checked string first should be normalized (i.e. URL encoding, illegal characters, etc removed).
• Added a possibility to check ServerVariables with RewriteCond directive. It could be done using %ServerVariable instead of a header name.
• Improved configuration parsing process error logging. Now error messages contain line numbers.

ISAPI_Rewrite version 1.2 build 14 (Full version only):

• Fixed a problem introduced in the Full version 1.1 build 11. Configuration flag CacheClockRate was incorrectly parsed. And after the first cache cleanup inetinfo.exe process began to consume 99% of a CPU time.

ISAPI_Rewrite version 1.2 build 13:

• Added new flag U (Unmangle Log). Now ISAPI_Rewrite can log URL as it was originally requested.

ISAPI_Rewrite version 1.1 build 11:

• Fixed a problem with the truncation of the last character of a configuration file.
• Fixed several shortcomings with documentation and default configuration files.
• Included additional optimisation for the Internet Information Server 6.0.
• ISAPI_Rewrite now adds custom header with original URL information to the client request, so the original URL can be retrieved in the server script.
• New RewriteHeader directive now allows to rewrite not only the URL part of the client request, but any other HTTP header or even method and version information.

Copyright ©  2002 - 2004, Helicon Tech