Sign up for the ITPro Today newsletter
Stay on top of the IT universe with commentary, news analysis, how-to's, and tips delivered to your inbox daily.
15 Tips for Troubleshooting VPN Connections
Troubleshooting a VPN is complex because the data travels through many links. Check out these troubleshooting tips tailored to specific VPN connection problems.
Paula Sharick
March 8, 2000
18 Min Read
A few server tips and a bucketful of client techniques
You can construct a VPN in a myriad of ways, but constructing your VPN is just your first step. After you construct the VPN, you have to troubleshoot it.
A minimal VPN implementation has a RAS PPTP server connected to the Internet, a client connected to the Internet, and a PPTP connection between the server and the client. As long as ISP service or Internet connectivity is available, clients can connect to your server or LAN from anywhere in the world. However, most VPNs aren't as simple as a connected server and client. More often, the VPN server is on a routed LAN segment, often behind a firewall, and the client connection uses an ISP network, which also employs routers and firewalls.
You can build a PPTP server as a standalone server or as a domain controller in a couple of steps: You install RAS and the PPTP protocol and configure PPTP ports the same way you configure dial-up connections. Windows NT client setup is equally straightforward: You load PPTP and configure the PPTP connection to locate the PPTP server over the Internet. With such a simple setup, you might assume that the VPN connection will function properly the first time. However, administrators spend a fair amount of time troubleshooting before they successfully deploy a new VPN.
Troubleshooting a VPN, like troubleshooting any WAN connectivity problem, is complex because the data travels through many links before it arrives at its destination. For example, data typically flows from the client to an ISP's router, through a firewall, across the ISP's network, maybe across additional ISPs' networks, to the company's router, to a firewall or proxy server, and finally to the destination PPTP server.
When a client connects to an ISP (this connection uses the Point-to-Point Protocol—PPP—portion of the VPN connection), the ISP assigns the client a TCP/IP address, a DNS server address, and a default gateway. When the client initiates a PPTP connection, that action creates a second TCP/IP session (this session is the tunnel portion of the VPN connection), which is embedded inside the first session and provides packet encryption and encapsulation. When the client connects successfully, the VPN server assigns the client a second IP address, a second DNS server address, frequently a WINS server, and another default gateway. At each link in the connection, something can go wrong. Knowing the common configuration and connectivity problems and having a troubleshooting procedure to follow will help you debug your VPN connections.
VPN Server Recommendations
If possible, start with an NT server that has a minimum number of services installed and limit the protocols to TCP/IP and PPTP. You'll save time if you update your server with service packs before you try to debug client connections. NT 4.0 Service Packs 5 (SP5) and SP6a correct numerous problems with PPTP connections, including performance problems related to fragmented packets, dropped connections, and refused connections. I have four more recommendations to help you keep the server configuration simple and straightforward for troubleshooting purposes.
Configuring a multihomed server. If your PPTP server has two network cards, one for the LAN and one for the WAN, leave the gateway field on the LAN adapter blank (don't enter zeros; leave it blank). In the gateway field of the WAN network interface, enter the TCP/IP address that your ISP provides; the gateway address usually points to a router at your ISP. You need the blank gateway so that the server can route network packets to the client. Leaving the LAN gateway blank is standard practice when you configure a server with multiple network adapters. For test purposes, I recommend you manually enter the TCP/IP address and WINS server address for the LAN NIC (instead of using DHCP to assign these values).
Configuring RAS. When you install RAS, configure only as many VPN ports as you truly need to support active client connections. Although each RAS server can support 256 concurrent connections (assuming you have the bandwidth for all this activity), you might need only 40 concurrent connections to support your mobile users. Next, configure the server to assign client addresses from a static address pool, rather than assigning addresses from a DHCP server. If you configure RAS to assign client addresses from a static address pool, clients inherit the DNS and WINS settings from the RAS server. If your RAS server can browse the network, clients should also be able to browse the network with the same settings.
If you prefer DHCP, verify that DHCP scope option 44 (WINS/NetBIOS name server) points to the WINS server and that scope option 6 shows the address of your DNS server. When you don't define these options, you almost guarantee problems with client browsing.
Enabling PPTP filtering. Configuring and testing a VPN server that resides outside your firewall is easier than testing a server inside your firewall because avoiding the firewall removes one link in the test-and-debug chain. If you aren't running your server in a highly secure environment, you can place the server outside the firewall and restrict incoming VPN traffic to PPTP packets only. To enable PPTP filtering, right-click Network Neighborhood, select Properties and Protocols, double-click TCP/IP Protocols, and select the WAN adapter and Advanced. Then, select the Enable PPTP Filtering check box. When you enable PPTP filtering, the server will refuse all non-PPTP requests. I've tested this feature, and it's an effective method for restricting incoming sessions to PPTP-only connections. PPTP filtering has one important side effect: When you enable filtering, LAN clients can't use the RAS server's WAN connection to browse the Internet because filtering disables incoming HTTP and FTP traffic.
If you want the VPN server to restrict incoming packets to PPTP and host an Internet-accessible Web site, you can make a Registry modification that lets other packets through the filtered interface to the local system only. Go to the HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesRASPPTPF Parameters Registry key, and add the value entry AllowPacketsForLocalMachine of REG_DWORD data type 1. When you make this modification, you expose the RAS server to the Internet yet restrict incoming connections to the VPN server, so remote clients can't see any other resources on your network.
Using firewall ports. Before you place a VPN server behind a firewall, verify that your firewall software accepts PPTP packets. Sometimes firewall software packages (including some versions of Check Point Software Technologies' FireWall-1) don't accept PPTP connections when you configure the firewall with Network Address Translation (NAT—for information about NAT, see Zubair Ahmad, "Windows 2000's Network Address Translation," February 2000). In this situation, the client's attempt to connect to the RAS server produces the error message Event ID 721 PPP remote peer not responding. When you place a VPN server behind your firewall, be sure to enable IP protocol 47 (Generic Routing Encapsulation—GRE) and TCP port 1723. The connection uses port 1723 for general housekeeping, such as PPTP tunnel creation, maintenance, and termination. Port 47 passes tunneled data between the client and the server (including the GRE protocol), and you also need TCP port 1723 [established] if you're supporting RAS server-to-server VPN connections.
Before you try to connect a VPN client, verify the server's TCP/IP settings on both NICs and make sure your RAS server can perform all typical network operations (e.g., browse the LAN, connect to LAN resources, connect to the Internet, browse the Internet). Then, enable dial-up permission for your test account. You might also want to enable PPP logging for your initial test.
Client Troubleshooting
To operate successfully, a PPTP client must properly maintain two sets of TCP/IP stack settings: one for the ISP and Internet connection and one for the VPN server connection. The client's routing table must also have two entries: one that directs network packets to the ISP for Internet browsing and one that points to the VPN server interface for LAN browsing. When the stack settings are incorrect, clients experience problems. In general, NT clients maintain separate TCP/IP stack settings, but Windows 95 clients typically have trouble with stack settings when the clients have a network card and a modem. After establishing a PPTP connection, the Win9x default gateway might still point to the ISP, which prevents the client from successfully browsing the LAN. Let's take a look at the five most common client connectivity problems.
Client can't connect to the PPTP server. The first problem you might encounter is the client's inability to connect to the PPTP server. You need to check for three possible causes of this problem.
Establishing VPN server Internet connectivity. After you configure the client, you need to verify that the VPN server has a connection to the Internet. The easiest way to verify a connection is to use the server's TCP/IP address and ping the server from the client. (If your PPTP server is behind a firewall and the firewall blocks Internet Control Message Protocol—ICMP—ping messages, this technique won't work.) If the ping gives you the message request timed out, something is amiss with the server's Internet connection. If the server responds by address, you can enter the TCP/IP address in the phone number field of the DUN entry to establish the PPTP session. Although it's less friendly than a Fully Qualified Domain Name (FQDN), this technique works fine when you know the server's address.
Remember that a server with a dial-up connection is likely to get a different address each time the server connects to the ISP. To connect by address, you must know the address the ISP has assigned to the server each time the server makes a dial-up connection. More commonly, your RAS server will have a permanent address, which eliminates one small variable in the connection process.
If the server responds by address, ping it by name. If the server doesn't respond by name, one of two situations is likely: The server might not have a registered domain name, or your ISP DNS server might be down or not working properly.Checking PPTP filtering. With PPTP filtering enabled on the server, you might see the message Error 678: There is no answer, or Error 650: The Remote Access Server is not responding. Disable PPTP filtering on the server (Net Stop RASPPTPF), and see whether you can establish a nonfiltered connection.
If you can connect with filtering disabled, check the server's filter settings. If you disable UDP ports 137 and 138 or TCP port 139, NetBIOS packets can't pass through the network. You also need to enable these ports on all firewalls and routers that are between the client and the server for unicast (point-to-point) traffic.Filtering the GRE protocol. If the server responds by address and name but you still can't connect, your ISP's routers or internal routers or firewalls might be filtering GRE packets. To establish a PPTP tunnel, the client and server exchange GRE packets, and some ISPs disable external GRE packets because the ISPs use GRE internally to manage routers. Although GRE filtering is uncommon, it will prevent a PPTP connection, so make sure you have IP protocol 47 (GRE) and TCP port 1723 enabled at both ends of the VPN connection. You can identify GRE filtering with the Microsoft Network Monitor or similar network sniffer tool. For more information about monitoring PPTP packets during a VPN connection, see "Related Articles in Previous Issues."
Client can connect but can't log on. The second connectivity problem you might encounter is when a connected client can't log on. You need to check three possible causes for this problem.
Configuring domain and server accounts. You can configure your RAS server as a domain controller or a standalone system. If you configure the server as a domain controller, make sure the user's domain account has dial-in permission. If the server isn't a domain controller, RAS by default authenticates client credentials against the local SAM. A user can authenticate to a standalone server in two ways: with a local account on the RAS server or with a Registry edit that forces the server to authenticate credentials against the domain SAM. In all cases, the account you supply must have permission to dial in.
Configuring computer accounts. If the client is an NT workstation or server, the computer must have an account in the domain. If the client is a new system, create the new computer account in Server Manager before you test the connection. If the client system already has an account on the network but has been disconnected for a week or more, the computer account password might not be synchronized with the server. Every computer account has a hidden password that the PDC resets automatically; if a system is offline for a long time, the account password can be different on the PDC and the client. Ordinarily, you can delete and re-add the account to correct this problem.
Negotiating client authentication. A RAS server can use three different authentication protocols to authenticate PPTP users. In increasing order of security, these protocols are Password Authentication Protocol (PAP), which is clear text; Challenge Handshake Authentication Protocol (CHAP), which is encrypted and hashed; and Microsoft Challenge Handshake Authentication Protocol (MSCHAP), which is encrypted and double-hashed with a checksum. The authentication protocols that the client and server negotiate for logon depend on the encryption settings you select when you configure the server's incoming VPN ports and the client PPTP connection network settings. The following options are available on the server and the client:
Allow any authentication, including clear text. The server will authenticate clients with the protocol that the client requests (e.g., PAP, CHAP, MSCHAP).
Require encrypted authentication. The server will authenticate with MSCHAP, Data Encryption Standard (DES), or Shiva PAP (SPAP).
Require Microsoft encrypted authentication. The server will authenticate only with MSCHAP.
Microsoft introduced MSCHAP V2, a more secure version of MSCHAP, after SP3. You can make a Registry edit on the server and on Windows clients to force clients to authenticate only with MSCHAP V2. However, when you make this modification, clients that don't support MSCHAP V2 (which is a proprietary Windows protocol) can't log on successfully. Thus, this change can prevent UNIX and Macintosh systems from logging on to your VPN server.
To obtain troubleshooting information about logon failures, enable logon auditing in User Manager and try the connection again. You can get a good picture of the roadblock when you look at the records in NT Event Viewer's Security Log. You can see if the username is invalid, if the password has expired, if the computer lacks a valid account, or if no VPN ports are available.
When a user does log on successfully, the application event log records the date and time of the logon. You'll find another event in the event log that records the user's logoff time and the duration of the session.
Client can log on but can't browse the LAN. You can also encounter a situation in which the client has logged on but can't browse the LAN. To begin to troubleshoot this problem, make sure you set the workgroup to the target NT domain name on all Win9x clients. Next, you might not want your clients to browse if you have more than 15 or 20 nodes visible because browsing a large network over a slow dial-up connection can be extremely frustrating. Predefining or manually mapping Uniform Naming Convention (UNC) connections to needed shares and resources after establishing the PPTP session is far more user-friendly. Finally, you need to understand how the four TCP/IP settings affect your network connection. (For more information about these TCP/IP settings, see the sidebar "Important Client TCP/IP Settings.") When you support users who work at home with a permanent high-speed connection, browsing the LAN remotely is a viable option. After checking these components and reviewing the TCP/IP settings, you can use the following items to troubleshoot the browsing problem.
Check browsing. When you browse the network or even a specific server, you commonly receive the message System error 53 has occurred. The network path was not found. An inability to browse usually means the client can't resolve NetBIOS names. Make sure the client has a WINS server assigned, either statically (in the PPTP connection's Network Settings) or dynamically (use Ipconfig for all clients or Winipcfg for Win9x clients). If the client has no WINS server address, enter the address manually, reconnect, and try browsing again.
Set up the default gateway. Look at the default gateway setting for the PPTP connection, or print out the route table (use the Route Print command). If the gateway still points to the ISP, every client request to browse the LAN goes to the ISP rather than to the VPN connection, and the ISP might block ports required for NetBIOS name broadcasts. Remember, routers and firewalls can prevent the transmission of NetBIOS names unless you enable unicast traffic on UDP ports 137 and 138 and TCP port 139. NetBIOS names are proprietary to Microsoft, and some ISPs might not let this data flow through their infrastructure.
You can manually delete the route in the route table and add a static route to the VPN server's virtual interface. The server's virtual interface is the address assigned to the VPN interface. This address is either the first address in the static address pool or the first available DHCP address in your RAS server configuration.Enable NetBEUI. If you aren't a TCP/IP purist, you can install NetBEUI on the RAS server and remote clients to solve client-browsing problems. Enable NetBEUI for incoming connections on the server's VPN ports, and select NetBEUI in the PPTP connection on the client. Doing so connects the client to the server with NetBEUI over TCP/IP. Try as you might, you can't escape the inherent limitations of the NetBIOS-based NT namespace. This NetBEUI option is probably the easiest way to obtain a fully browsable LAN.
If the client still can't browse, try connecting from the client to a network share. For example, use the Net Use Z: \myservermyshare command. Manually connecting to shares is frequently a good workaround that lets users access files and printers while you're in troubleshooting mode.
If you still can't browse, you need to review the VPN server configuration. Many server problems affect client browsing, but the list of potential problems and solutions is too long to cover in this article. You'll find many excellent pointers if you search for PPTP client browsing and multihomed browsing in the Microsoft Knowledge Base at http://support.microsoft.com/search/default.asp. The search will produce a list of articles that document problems specific to multihomed servers and browsing (e.g., browsers on each NIC don't exchange browse lists), PPTP connections, WINS server location, and more.
RELATED ARTICLES IN PREVIOUS ISSUES |
|---|
You can obtain the following articles from Windows 2000 Magazine's Web Site at http://www.win2000mag.com/articles.KEN MILLER AND RICHARD BRACKETT"Configuring VPNs," December 1999, InstantDoc ID 7529ERIC PEARCE"Managing VPNs with PPTP," January 1999, InstantDoc ID 4695PAULA SHARICK"PPTP Provides Secure Connectivity to Your Corporate Network," March 1999, InstantDoc ID 4877 |
Connected client can't browse the Internet. I've seen this problem several times on Win95 clients that have both a network card and a modem. A client can't browse the Internet while its VPN session is active. This problem occurs in two common scenarios. First, the VPN server might not let remote clients access the Internet when they have a connection. In this case, when you close the VPN connection, the client can browse the Internet because the default gateway reverts to the gateway that the ISP specifies. Second, Win95 might overwrite the ISP gateway with the VPN server-defined gateway when the client connects, so the client has no path to the Internet. You can manually add a static route to the ISP's default gateway by trying the VPN gateway first and the ISP gateway second to solve this problem.
Connected client doesn't appear in Network Neighborhood. I worked with a network engineer who encountered the problem of the client not appearing in Network Neighborhood on the LAN side, even with a fully functional VPN client connection. You configure the client's PPTP connection with TCP/IP only and connect and authenticate to the VPN server. Then the client can browse all LAN resources. When the remote client expands Network Neighborhood, the client shows itself and all other clients in its browse list, but the remote system never appears in Network Neighborhood on the LAN. If you want remote clients to appear on the LAN browse list, you need to install NetBEUI on the RAS server and RAS clients. This peculiarity is a known problem with RAS, but no fix is available at press time.
Prepared at Last
I've covered most of the common VPN configuration and connection problems, and I expect that these tips will help you get your VPN connections up and running. This technology is engaging and popular and will become widely adopted when administrators can take advantage of the standard implementation of tunneling protocols and IP Security (IPSec) in Windows 2000 (Win2K). Remember, start simple and take one step at a time. Then, lean back, put your feet on the desk, and listen to your users rave about this new functionality—it's a great alternative to traffic jams and gridlock.
About the Author
You May Also Like
.jpg?width=700&auto=webp&quality=80&disable=upscale)
