Fiddling with HTML Forms with Fiddler

In "Teaching PowerShell to Click," I showed you how to use the Invoke-WebRequest cmdlet to automate the process of filling out a form on a web page and then capturing the form's result. To accomplish that, however, I skipped over an important step: How did I know how the form wanted its inputs formatted in a query string? I mentioned that you could use a tool such as WireShark or Microsoft Message Analyzer to trace your HTTP traffic, but if you've never used those tools, you'd probably find their learning curves a bit steep. That's why it's nice that Telerik gives away a much simpler web-peeking tool called Fiddler, and this month I'll show you how to install it and put it to work.

Download and Install Fiddler

Telerik offers two versions of Fiddler for the PC: Fiddler 2 for Windows 7 and earlier, and Fiddler 4 for Windows 8.0 and up. I'm working on a Windows 10 machine, so my figures show Fiddler 4, but downloading either version of Fiddler is easy. The process requires an email address, but I haven't gotten any junk mail, and they don't validate it in any case. Now, run the downloaded executable (Fiddler4Setup.exe or Fiddler2Setup.exe), and accept the defaults.

Configure Fiddler for SSL/TLS

On first startup, Fiddler points you to a web page suggesting three initial setup steps: Configure Fiddler for Windows 8, Configure Fiddler to decrypt HTTPS traffic, and Monitor traffic from localhost from IE or .NET. If you’re never going to try to sniff traffic to/from a Windows “modern” app, skip the first step. If you’re not using Internet Explorer (IE) 8 or earlier, skip the last.  Everyone, however, should configure Fiddler for HTTPS, as many forms are secure. The instructions that Fiddler shows in a browser are quite good, so I won’t recapitulate them here.

Now we’re ready to check out the Fiddler UI. When it opens, you’ll see a window with left and right panes, as in Figure 1.

The pane on the right is the "web session" window, and it shows what you're doing with whatever web browsers you have open or even just from background tasks that talk on the web. That’s the cool thing about Fiddler: No matter what kind of web traffic some app somewhere on your system generates, Fiddler shows it to you. That's why there's all that chatter in Figure 1 about sessions going to markminasi-my.sharepoint and so on; Fiddler has detected that my Office 365 apps (e.g., OneDrive for Business) are all talking to my Office 365 storage, which is really Microsoft SharePoint (a web application, basically) in the background. I'm not terribly interested in the SharePoint traffic, so let's get rid of it. Look at the bottom left of Figure 1 and note that you see "Capturing" and, to its immediate right, "All Processes." Click "All Processes," and you'll get four options: All Processes, Web Browsers, Non-Browser, and Hide All. Choose Web Browsers to see just browser-based traffic. The old SharePoint junk is still in the window, however, so press Ctrl+X to clear the web session window, and then we'll be ready to sniff a form.

The basic approach to getting something done in Fiddler is to click one of the lines (sessions) in the left pane, and then see information about that session in the right pane. By default, the right pane is the Statistics pane, not surprisingly. If you right-click a session, you can choose Filter Now, which can remove all traffic from the session's URL, process ID, or any sessions on that given type of browser—so, for example, you could tell Fiddler to ignore all traffic from Chrome. (The page then has a list of filters that you can click on and remove when desired.)

Give it a try with my simple form from last month. Point your browser to minasi.com/addit.htm and, if you cleared your session screen, you'll see something like Figure 2.

Now fill out the form, using "42" and "237" as the addends, and click SUBMIT! You will see a new session line (200 HTTP www.minasi.com /addit-worker.asp…), and if you click that line you'll get stats on it, which aren't very interesting. But click Inspectors and TextView, and you'll get a screen like Figure 3.

Thus, Fiddler showed us that the URI to query was www.minasi.com/addit-worker.asp rather than addit.htm, and that the key to getting the data to addit-worker.asp was to pass in a body that looked like addend1=42&addend2=237&B1=SUBMIT%21 to elicit our result, which would be Sum=279.

I cannot stress strongly enough how useful Fiddler is for anyone trying to troubleshoot or explore just about any web-based conversation. Next month, more forms—see you then!