Troubleshooter: Configuring RPC over HTTP Behind an ISA Server Firewall

We're trying to set up remote procedure call (RPC) over HTTP on our production server. Although this configuration works in the test lab, we can't get it to work on our production network, which is behind a Microsoft Internet Security and Acceleration (ISA) Server 2000 firewall that's off-loading Secure Sockets Layer (SSL) processing. In this production environment, why are our users receiving the message HTTP/1.0 401 Anonymous requests or requests on unsecure channel are not allowed?

By default, the RPC over HTTP proxy server doesn't allow anonymous connections. However, if you use ISA Server to perform SSL bridging (in which case the remote client uses SSL to talk to ISA Server, and ISA Server initiates a new, separate SSL session to the proxy), RPC over HTTP will work. If you try to use ISA Server to terminate the SSL session without bridging, you run into the problem you've described. To make this configuration work in your production environment, you need to add the AllowAnonymous entry (of type REG_DWORD) to the HKEY_LOCAL_MACHINESOFTWAREMicrosoftRpcRpcProxy registry subkey and set its value to 1. When this entry is missing or set to 0, the RPC proxy service disallows anonymous connections. To make your setup work behind an ISA Server firewall in Web publishing mode, you must add this value and configure anonymous access on the RPC proxy virtual directory. Note that you probably shouldn't use this value with hardware SSL off-loading because it allows unencrypted and unauthenticated connections to the RPC proxy server.