Inked

LANGUAGES: VB.NET

ASP.NET VERSIONS: 2.0

This article provides an introduction to the Ink APIs for Webdevelopers targeted at Windows XP Tablet PC users. Follow along and you ll learnto build a simple Windows Forms control that is Ink-enabled, embed it into a Webpage, then interact with it.

Although Tablet PCs have a lot of great built-infunctionality, such as handwriting recognition, the Tablet PC SDK enablesdevelopers to tap into the power of Ink to write a lot of interestingapplications. Over the years I ve tried to look at applications that think outof the box of typical data entry to see where Ink can really shine. Inaddition to some of the awesome illustration software, some eye-openingexamples of these are XThink s MathJournal and Physics Illustrator, which cameout of the Microsoft Research Labs. Although handwriting and recognition arethe first things that most people think of, it s the true mobility of thesecomputers that make them stand out. Consider any scenario where a worker uses aclipboard to fill out a form, then hands that to someone else to enter into acomputer. This is the perfect target for an Ink-enabled business application.But there are many other, more interesting things that can be done, such asmarking up images (consider an architect, or perhaps an insurance adjusterinspecting a damaged car). The real estate and medical industries have also embracedthe technology in a very big way.

One thing that many of these applications have in commonis that they are all being built as client-side Windows applications. Thereason for this is that the Tablet PC is dependent on a digitizer that coversthe computer s screen. The digitizer receives the magnetized signals from thestylus and sends those to the operating system, which identifies the strokesand passes them on to the software (internal or customized). The software thenuses the Tablet PC API to interpret and work with the resulting data. WithWindows XP, a special version of the O/S, Windows XP Tablet PC Edition, wasrequired. However, with Windows Vista, the Tablet PC functionality is wrappedinto the base O/S. This article will focus on Windows XP Tablet PC Edition.

Because all of this work is done on the client side,embedding Ink into Web sites has not been an obvious target for developers.Additionally, the challenges of working with ActiveX or other embedded controlsin a Web page have been daunting.

Basic Requirements

You don t need a Tablet PC to do Tablet PC development.Installing the Tablet PC SDK will give your computer most of what you need todevelop and test your applications. MSDN has articles on setting up fordevelopment, as well as developing off-tablet (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/tpcsdk10/lonestar/devcenter/tbidxindex.asp).If you want to test the inking abilities with something finer than what a mousewill render, you can plug into your system an inexpensive digitizer, such asone made by Wacom. Whether you have a Tablet PC or not, you ll need to downloadand install the Tablet PC SDK 1.7, as well as the updates to integrate with .NET2.0 (http://www.microsoft.com/downloads/details.aspx?FamilyId=B46D4B83-A821-40BC-AA85-C9EE3D6E9699&displaylang=en; http://www.microsoft.com/downloads/details.aspx?FamilyID=69640b5c-0ee9-421e-8d5c-d40debee36c2&displaylang=en).The MSDN articles will assist you with this. Users must have a Tablet PC tointeract with your Ink-enabled controls.

Your First Ink-enabled Web Application

Now to the code! The remainder of this article will walkyou through building a simple Ink-enabled Windows Forms control, as well asembedding it into a Web page.

This control will allow users to draw or write into itusing a few colors and a few pen widths, erase strokes, and clear the controlcompletely. It will also have the ability to expose the contents of the controlso you can get the data to the server side for saving into a database orwhatever you choose to do with the data.

It is important to note that you should build as muchlogic into the control as possible, thereby minimizing the necessaryinteraction between the Web page and the control. For example, place controlsfor changing the pen color within the user control. That way it won t benecessary to write JavaScript code to tell the user control to change a color.

In Visual Studio, create a Windows Control Library projectand add to the project a reference to the Microsoft Tablet PC API. A usercontrol is automatically created. We ll work with that one. The first thing todo to this control is give it a border using the control s BorderStyleproperty. Without a border, it s impossible to determine where to ink when thecontrol is embedded in the Web page.

A ToolStrip control is a great place to organize all thebuttons for interacting with Ink. With the buttons, we can tell the control toselect a new color for the pen or change other characteristics of how the inkwill be drawn into the control. Add eight ToolStripButtons to the ToolStrip.Using the properties for each button, change the first four buttons. Eachbutton should have DisplayStyle set to None and the BackGroundColor set to acolor of your choice. The fifth and sixth buttons will be used to set the inkweight to Thick and Thin . The next button will be used to allow the user toerase strokes. Change the DisplayStyle of these buttons to Text and their Textproperties accordingly. Of course, you can use an eraser image instead of text,if you d like. The last ToolTipButton should be for clearing the entirecontrol. Figure 1 shows what the control should look like so far.

Figure 1: The control as it shouldlook so far.

Ink-enabling the User Control

It s time to get Ink capabilities into the control. Thisis done using the Microsoft.Ink.InkOverlay object, which is a component, not acontrol. InkOverlay is instantiated and attached to whichever control you wouldlike to be Ink-enabled. If this were a Windows Form, you may want to Ink-enablea TextBox control. In some cases, we ll want the entire user control Ink-enabled.However, because the ToolStrip control is there, we ll create a panel to fillthe balance of the user control and Ink-enable this panel.

The basic structure of the object model is that InkOverlaycontains an Ink object, which is made up of a collection of strokes. A strokeis defined by the starting point when a user touches the stylus to the screen, tothe ending point when the stylus is lifted from the screen. You can drill down furtherto work with all the points that make up a stroke.

Drop a panel onto the control and set its docking propertyto Fill. In the code, create a variable for the InkOverlay object, and in theLoad method instantiate the object, attach it to the panel, and enable it:

Imports Microsoft.Ink

Public Class UserControl1

 Dim inkO As InkOverlay

 Private SubUserControl1_Load(ByVal sender As Object, _

  ByVal e AsSystem.EventArgs) Handles Me.Load

     inkO = NewInkOverlay(Panel1)

     inkO.Enabled = True

  End Sub

End Class

Thanks to Visual Studio 2005 s UserControl TestContainer,you can run the project and test the drawing ability of the control. If you arenot on a Tablet PC, you ll be able to draw with your mouse although it willbe a bit ragged as the resolution of the mouse is not nearly as fine as that ofa stylus and digitizer.

Formatting the Ink

The Tablet PC API has a surprising amount of functionality.We ll barely scratch the surface in this example, but it s a great place tostart.

The InkOverlay.DefaultDrawingAttributes property containsproperties to control the color and other attributes of the ink. This affectsink to be drawn, not ink that already exists in the control. The ChangeInkColormethod below can be used to change the ink color. Wire up the method to the Clickevents of the four color buttons:

Private Sub ChangeInkColor(ByVal sender As System.Object, _

 ByVal e As System.EventArgs)

 Dim button AsToolStripButton = sender

 inkO.DefaultDrawingAttributes.Color = button.BackColor

End Sub

The thinInk and thickInk Click events change the Width ofthe DefaultDrawingAttributes:

Private Sub thinInk_Click(ByVal sender As System.Object, _

 ByVal e AsSystem.EventArgs) Handles thinInk.Click

    inkO.DefaultDrawingAttributes.Width = 50

 End Sub

 Private SubthickInk_Click(ByVal sender As System.Object, _

 ByVal e AsSystem.EventArgs) Handles thickInk.Click

   inkO.DefaultDrawingAttributes.Width = 150

End Sub

The InkOverlay.EditingMode options are Ink, Delete, andSelect. In the Click event of the Eraser button, we ll toggle the mode from inkto eraser. At the same time, the text of the button will change so that whenthe InkOverlay is in eraser mode, it will be obvious that clicking the buttonwill set the user back to ink mode. Note that there are two ways to erase,controlled by the InkOverlay.EraserMode property. The default mode isStrokeErase, which will remove an entire stroke by touching any point on thestroke. The alternate mode is PointErase, which behaves more like a typicalerase action, removing ink only at the positions the eraser touches:

Private Sub Eraser_Click(ByVal sender As System.Object, _

 ByVal e AsSystem.EventArgs) Handles Eraser.Click

  If inkO.EditingMode =InkOverlayEditingMode.Delete Then

   inkO.EditingMode =InkOverlayEditingMode.Ink

   Eraser.Text ="Erase"

  Else

   inkO.EditingMode =InkOverlayEditingMode.Delete

   Eraser.Text ="Ink"

  End If

 End Sub

The last button on the ToolStrip is used to clear all theink from the control, using the InkOverlay.Ink.DeleteStrokes method. You alsoneed to invalidate the control that is attached to the InkOverlay to force itto redraw itself. Otherwise, even though the strokes no longer exist in the Inkcontrol, they remain displayed on the drawing surface:

Private Sub ClearControl_Click(ByVal sender _

 As System.Object, ByVal eAs System.EventArgs) _

 Handles ClearControl.Click

   inkO.Ink.DeleteStrokes()

    Panel1.Invalidate()

 End Sub

Run the project and check out all the functionality of thecontrol in the UserControl TestContainer (see Figure 2).

Figure 2: Test the control sfunctionality.

Adding Functions to Interact with the Web Page

At this point you could easily embed the control in a Webpage and let the user have some fun drawing. All the functionality built in tothe control will work on the Web page with no additional coding required.However, the user won t be able to do anything more than draw on the page; therewould be no way to save images or send them anywhere, either by e-mail or toserver applications, such as a Web service or a database.

Let s rectify this situation; let s add functionality thatthe Web page can call. For this article, we ll enable the application toextract the Ink data from the Ink control so you can save it as a .bmp file onthe server. Although it is certainly possible to store the Ink data so that itmay be retrieved and pulled back into an Ink-enabled control and editedfurther, that is beyond the scope of this article.

We ll be adding a public function named GetInkData thatcan be called by the host (the Web page); it will return a string to the page.

The Ink class has a Save function that outputs a byte arrayin several formats. One format, InkSerializedFormat (or ISF) persists all themetadata about the Ink object. This is convenient for placing the data into an Inkcontrol at a later date. Another format, GIF, saves the data as a GIF file andadds the ISF metadata, as well. That means if you want to have a true imagefile and the ISF data, you need only save once. The resulting byte array canthen be written out to a GIF file or pulled back into an Ink-enabled control.

The GetInkData function will output the GIF-persisted data;however, for it to be transmitted from the client-side script to the server-sidecode it first must be converted to a Base64 string format. Although theInk.Save function has alternative persistence formats for Base64, they requireadditional manipulation. Therefore, this code will persist to GIF and then usethe .NET conversion method to get the Base64 string. Add the following functionto the Ink control:

Public Funtion GetInkData() As String

 If inkO.Ink.Strokes.Count= 0 Then

 Return  ("empty")

 Else

 Dim inkBytes As Byte() =inkO.Ink.Save(PersistenceFormat.Gif)

 ReturnConvert.ToBase64String(inkBytes)

 End If

End Function

Now the control is complete. But before building theproject, there s one more very important step! The assembly must be visible toCOM. This can be done by opening the Application Tab of the project properties,then clicking the Assembly Information button and checking the Make AssemblyCOM-Visible checkbox.

Compile the project. The resulting DLL is what gets addedto the Web site. Because it will be an embedded resource, you ll actually dothis simply by copying and pasting the DLL file, just as you would an imagefile.

Creating the Host Web Site

Create a new ASP.NET Web site. I have successfully runthis sample using an IIS (HTTP) Web site, a file-based server (File System) Website, and a Web Application project. Note that if you do development in a FileSystem server, it will be necessary to change the user control s class name anytime you modify the user control and recompile the assembly.

Next, add the DLL to the project. If the Web site is inthe same solution as the user control project, you can copy and paste from oneproject to the other. Otherwise, you can use the Add Existing Item method toadd the DLL to the Web site.

Prior to Internet Explorer 7, it was enough to embed thecontrol directly into the HTML of the Web page with the

The object will be embedded using the assembly name andthe strongly typed name of the user control s class. My user control projectwas named aspnetPROInkControl; by default this is the name of the DLL assembly.The class has the default name of the user control (UserControl1). The code forthe JavaScript file looks like this:

document.write('');

Because the control is being embedded when the page isrendered, you won t see it on the Web page at design time. You ll most likelywant to adjust the height and width and determine the placement of the control.I get around this by embedding the control directly at design time using thefollowing snippet in the HTML of my page:

When I am satisfied with this, I modify the JavaScriptfile accordingly, then replace the

This will render the Ink control in the position where thescript tags are placed. The last steps are to call the GetInkData function fromthe client side and send the data to the server.

Place an control on the Web page. Also,place an HTML Hidden input control somewhere on the page. Be sure to make theHidden input control available to the server by setting its runat property toserver. The following code assumes that the form and these controls have theirdefault names of form1, Button1, and Hidden1.

Two things need to happen when the button is clicked. Thefirst is a client-side call to the GetInkData function in the embedded Ink control.This call will also place the resulting Ink in a hidden HTML control. Then thebutton s Click event will read the data from the hidden HTML control and saveit to a GIF file on the server.

Thanks to a new property in the ASP.NET Button control,named OnClientClick, it is possible to fire off both the client-side and server-sideevents with just one click of the button. Change the OnClientClick property toSaveInk. Then add the SaveInk JavaScript function to the source of the Webpage. Place it inside the opening tag:

 function SaveInk() {     form1.Hidden1.value =form1.InkControl.GetInkData(); }

Fortunately, the client-side event is hit first.Therefore, in the server-side Click event function for the button you ll beable to read the value of the Hidden1 control, which will already have the Inkdata. Because the data is in Base64String format, it will be necessary toconvert it back to bytes to create the GIF image file from it. Note that in thecontrol there was a test to ensure that there was data in the Ink control. Ifthere was no data, the control returns empty . The Button1_Click event testsfor the return value of empty before proceeding with the file creation:

Protected Sub Button1_Click(ByVal sender as Object, _

 ByVal e asSystem.EventArgs) Handles Button1.Click

Dim inkString as String

   inkString=Hidden1.Value

 If InkString<>"empty"Then

   Dim bytes() asByte=Convert.FromBase64String(InkString)

   System.IO.File.WriteAllBytes(Server.MapPath("") & _

    "myInkImage.gif",bytes)

End Sub

Now you should be able to run the Web page, draw in it,and save your drawing. One thing to point out is that once you get to thebutton s Click event and have the string that is returned by Hidden1.Value, youcan do whatever you want with the data. Though the example merely saves it toan image file, you could save the data into a database or an XML file, store itin Session for use elsewhere on the site, send it to a Web service, or takesome other action on it. At this point it s simply a string. Figure 3 shows theWeb page with the Ink control embedded in it.

Figure 3: The Web page with the Inkcontrol embedded in it.

Conclusion

Ink-enabled Web sites can be used for fun or as part ofbusiness applications. For example, rather than a drawing palette, considerallowing users to mark-up online documents or to actually use their signatureto sign a document. Be aware that hidden controls are not secure, so anythingthat requires security should be encrypted prior to storing it in the hiddencontrol. The biggest challenge, of course, is to find end users with moreartistic talent than the typical developer, as evidenced by the drawings inthis article.

The source code accompanyingthis article is available for download.

Julia Lerman is anindependent consultant and .NET Mentorwho has been designing and writing software applications for more than 20years. She lives in Vermont,where she runs the Vermont.NET User Group. Julia is well known in the .NETcommunity as an INETA Speaker, .NET MVP, ASPInsider, conference speaker, andprolific blogger. You can read Julia s blog at http://thedatafarm.com/blog.