How can I troubleshoot IPSec?

John Savill

April 9, 2000

5 Min Read
ITPro Today logo in a gray background | ITPro Today

A. There are a number of tools available to help you troubleshoot yourIPSec configuration which consist of

  • The IPSec snap-in for policy configuration

  • The event log

  • Group Policy snap-in to set IPSec policies for a GPO

  • The file oakley.log in the %systemroot%debug directory

But we will concentrate on two other tools, netdiag.exe and IPSecmon.exe.

IPSecmon.exe is part of standard Windows 2000 but netdiag.exe is supplied aspart of the support tools (SupportTools) so you will need toinstall these.

IPSecmon.exe is the simplest tool and shows current security associations forthe hosts communicated with over IP and if IPSec is being used (and if it iswhat TYPE of IPSec).

Click here to view image
Clicking the Options button allows the update frequency to be changed. In theexample I have one IPSec association in place using Triple DES.

The meaning of each field is as follows:

Active Associations

The number of active security associations with the computer being monitored.

Confidential Bytes Sent

The total number of bytes sent with Confidentiality, indicating that the packets were sent using the Encapsulating Security Payload (ESP) security protocol (decimal ID 50).

Confidential Bytes Received

The total number of bytes received with Confidentiality, indicating that the packets were sent using the Encapsulating Security Payload (ESP) security protocol (decimal ID 50).

Authenticated Bytes Sent

The total number of bytes sent with the authentication property enabled.

Authenticated Bytes Received

The total number of bytes received with the authentication property enabled.

Bad SPI Packets

The total number of packets for which the Security Parameters Index (SPI) was invalid. This probably indicates that the security association (SA) has expired or is no longer valid.The SPI is a unique identifying value in the SA that allows the receiving computer to select the SA under which a packet will be processed.

Packets Not Decrypted

The total number of packets the receiving IPSec driver was unable to decrypt. This may indicate that the security association (SA) has expired or is no longer valid, authentication did not succeed, or integrity checking did not succeed.

Packets Not Authenticated

The total number of packets that could not be successfully authenticated to the IPSec driver. This may indicate that the security association (SA) has expired or is no longer valid. The information in the security association is required for the IPSec driver to process the packets.It may also indicate that the two computers have incompatible authentication settings. Verify that the authentication method specified for each computer is the same.

Key Additions

The total number of keys that ISAKMP (the ISAKMP/Oakley mechanism) sent to the IPSec driver. This indicates that the ISAKMP Phase II security associations were successfully negotiated.

Oakley Main Modes

The total number of successful security associations established during ISAKMP Phase I. This indicates that the key information exchange was successful. Identities were authenticated and common keying material was established.

Oakley Quick Modes

The total number of successful security associations established during ISAKMP Phase II. This indicates that the negotiation for protection services during the data transfer was successful.

Soft Associations

The total number of ISAKMP Phase II negotiations that resulted in the computers agreeing only to a clear-text data transfer (no encryption or signing of the packets).

Authentication Failures

The total number of times authentication of the computer identities did not succeed. Verify that the authentication method settings for each computer are compatible. This may also indicate that the security association has expired.

Netdiag.exe is a more generic tool that is used to troubleshoot networkconnectivity problems but one of its options is to test IPSec as follows:

C:>netdiag /test:ipsec /v /debug

    Gathering IPX configuration information.
    Opening DeviceNwlnkIpx failed
    Querying status of the Netcard drivers... Passed
    Testing Domain membership... Passed
    Gathering NetBT configuration information.
    Gathering IP Security information

    Tests complete.


    Computer Name: CYPHER
    DNS Host Name: cypher.savilltech.com
    DNS Domain Name: savilltech.com
    System info : Windows 2000 Professional (Build 2195)
    Processor : x86 Family 6 Model 5 Stepping 2, GenuineIntel
    Hotfixes :
        Installed?      Name
           Yes          Q147222
           Yes          Q253562
           Yes          Q253934


Netcard queries test . . . . . . . : Passed

    Information of Netcard drivers:

    ---------------------------------------------------------------------------
    Description: Compaq NC3161 Fast Ethernet NIC
    Device: DEVICE\{9C65E63C-5242-45F8-9685-4A6649E92F35}

    Media State:                     Connected

    Device State:                    Connected
    Connect Time:                    16:34:16
    Media Speed:                     10 Mbps

    Packets Sent:                    25960
    Bytes Sent (Optional):           0

    Packets Received:                150278
    Directed Pkts Recd (Optional):   32265
    Bytes Received (Optional):       0
    Directed Bytes Recd (Optional):  0

    ---------------------------------------------------------------------------
    [PASS] - At least one netcard is in the 'Connected' state.



Per interface results:

    Adapter : Local Area Connection
        Adapter ID . . . . . . . . : {9C65E63C-5242-45F8-9685-4A6649E92F35}

        Netcard queries test . . . : Passed


Global results:


Domain membership test . . . . . . : Passed
    Machine is a . . . . . . . . . : Member Workstation
    Netbios Domain name. . . . . . : SAVILLTECH
    Dns domain name. . . . . . . . : savilltech.com
    Dns forest name. . . . . . . . : savilltech.com
    Domain Guid. . . . . . . . . . : {A225B0B5-8E82-4690-93F2-AA166BFDA773}
    Domain Sid . . . . . . . . . . : S-1-5-21-1614895754-1767777339-1801674531
    Logon User . . . . . . . . . . : Administrator
    Logon Domain . . . . . . . . . : CYPHER


NetBT transports test. . . . . . . : Passed
    List of NetBt transports currently configured:
        NetBT_Tcpip_{9C65E63C-5242-45F8-9685-4A6649E92F35}
    1 NetBt transport currently configured.

IP Security test . . . . . . . . . : Passed
    Directory IPSec Policy Active: 'Server (Request Security)'

IP Security Verbose Test . . . . . : Failed
    Access is denied.



The command completed successfully

About the Author

Sign up for the ITPro Today newsletter
Stay on top of the IT universe with commentary, news analysis, how-to's, and tips delivered to your inbox daily.

You May Also Like