Sign up for the ITPro Today newsletter
Stay on top of the IT universe with commentary, news analysis, how-to's, and tips delivered to your inbox daily.
Working with Images and Animation on the iPhone and iPad with .NET and C#: Part 1
Take photos and display and animate pictures in your iOS apps using Xamarin's MonoTouch C# development platform
Wallace B. McClure
April 24, 2012
16 Min Read
Multimedia has come a long way during my career in programming. I remember the HP-41C, the first calculator that I used day to day in my geeky teenage years (unfortunately, I've only gotten geekier as I have gotten older). It could only output text and a few beeps. I was in nirvana and thought it just couldn't get any better. Now, everyone in my family has an iPhone and an iPad. With these devices, we can use software and hardware to record and play audio and video, take and display pictures, and modify this content.
In this article (part 1) and the one soon to follow (part 2), I will discuss developing multimedia applications for the iPhone/iPad/iOS with .NET and C# using Xamarin's MonoTouch iOS development platform. Specifically, in the two-part series we'll explore taking pictures, displaying pictures, recording video, and playing video in iOS. In my experience, these are the most common operations performed by iPhone and iPad users. And for good measure, along the way we'll bounce a golf ball around the screen of your iPhone, and we'll do it all in C#.
In this article, we'll look at single images: how to display them, take pictures, rotate images, move images around the screen, and such. Part 2 will focus on more interesting animation, displaying video, recording video, and finally, editing video.
iOS provides a rich set of libraries to perform these operations. Thanks to MonoTouch, these libraries are surfaced for C# developers. These libraries are spread across several different classes within the MonoTouch.UIKit namespace. Now let's get started!
Displaying Pictures
What grandparent doesn't like to get the pictures of their grandchildren taking their first steps? Let's look at how we can display such images in iOS. There are several common ways to load images within iOS:
Images can be loaded from the local project. The advantage to this scenario is that images that are loaded from the local project are guaranteed to be available and the application does not have to go off of the device to retrieve the image.
Images can be pulled from a shared location, such as the photo gallery. A specific image is not guaranteed to be available, but the application does not have to go outside of the device to get the image. The application will need to integrate with the photo gallery, but that is not overly difficult.
Images can be pulled from a remote location (off-device). While this doesn't sound bad, doing so poses some potential challenges. Going off-device typically means we'll use a mobile 3G network. Mobile networks are not guaranteed to be available. Users can be in a location with no network or otherwise have a bad connection. iOS implements a watchdog timer. This timer will kill applications that have locked the UI thread. Add in the bad connection, and you can have a problem. Whenever you go off-device, you need to make sure that you use a background thread (e.g., a thread, a .NET 4 task, or another operation that will not run on the UI thread).
When displaying images in iOS, there are three classes that you will want to familiarize yourself with. These classes are
UIImage. A UIImage class is a high-level mechanism for displaying an image. UIImage objects can be created from files, Quartz image objects, or other raw image data. UIImage objects are immutable. Once an image's properties are set and the image is created, the object can't be changed.
UIImageView. A UIImageView class is the container class for a UIImage. It is responsible for displaying an image.
NSData. NSData contains raw data. We will use it to download and cache images from the Internet.
Displaying images in iOS is amazingly simple. There is a several-step process to display images. Here's the sequence of steps:
The first step when displaying an image is to create a UIImageView on our iOS screen, as shown in Figure 1. This will be what we use to display our images.
Figure 1: Creating a UIImageView
Associated with creating our UIImageView, we'll need to create an outlet so that we can display our image. With MonoDevelop 2.8 and the integration with Xcode 4, we can create the outlet by pressing the Ctrl key, selecting the object that we want to expose in the outlet, and then dragging that into the editor view. The UIImageView is named img. The result is the following code:
@property (nonatomic, retain) IBOutlet UIImageView *img;For loading the image from our project, we'll use this code:
UIImage pic = UIImage.FromFile("./Images/IMGCAT.jpg");img.Image = pic;The result is the image in Figure 2, showing my family's cat Wells sitting at the door.
Figure 2: Initial image in the project
Remote Images
Now that we have seen how to display an image that is within our project, let's look at how we would display an image that is loaded off-device.
Loading off-device images is a somewhat complicated issue. We can load one or more images using the main UI thread of iOS to do so. iOS comes with a watchdog timer that will "watch" applications that are running. If the application hogs the UI thread for too long, the application is killed by iOS. This keeps one rogue application from hogging the system. Along with this, we need to think about mobile networks and their availability. Wireless networks are not always available. We need to think about that as we build our application and load resources. The result of this is that we need to not lock the UI when we go off of the device. We can do this by using a background thread, such as a ThreadPool thread or .NET 4 task. If you use a regular managed thread, you will need to handle some NSAutoReplace operations.
Another thing to remember when pulling data from an off-device location is that network connectivity is precious. Once we pull an image, or any other heavyweight resource across a wireless connection, we need to think about how we can reuse that resource.
Let's look at some code to pull the data down and cache it. In this example, we'll use a thread pool thread. I'll assume that you've set up a UIImageView and an outlet named uiRemoteImage.
The first step is to define and create a Dictionary object. Notice that we are creating a dictionary of NSData objects.The definition looks like this:
private Dictionary imageDict;The dictionary object will be instantiated in the class's constructor.
The call to begin the download is a standard call to starting a ThreadPool thread.
ThreadPool.QueueUserWorkItem( new WaitCallback(LoadRemoteImage), "http://morewally.com/CatImage.jpg");In this situation, the LoadRemoteImage method is a custom method in our code. It will be called, and a string with the URL listed will be passed in.
The final step is to create our LoadRemoteImage method. Some of the things to note in this method are that we are using the Dictionary object to check whether we already have our image downloaded. A reminder is that the Dictionary object is not thread-safe, so we need to lock before we write to it. Another item to note is that we are in a background thread. Because of that, when we try to write to a UI control, we need to have that code execute via the main UI thread. To do that, we need to use InvokeOnMainThread and pass in a delegate that will be used to write to the UI controls. Figure 3 shows the code that does this.
private void LoadRemoteImage(object o){var img = Convert.ToString(o);NSUrl nsUrl = new NSUrl(img);NSData data;if ( !imageDict.ContainsKey(img) ){data = NSData.FromUrl(nsUrl);lock(this){imageDict.Add(img, data);}}else{data = imageDict[img];}InvokeOnMainThread(delegate{uiRemoteImage.Image = new UIImage(data);} );}One thing to note in our code sample is that the dictionary, caching, and locking are not a requirement for loading remotely located images. It is possible to just hand in an NSUrl of an image and load it synchronously. However, the result may not be what you really want performance-wise or application-stability-wise. Figure 4 shows the results of the code in Figure 3.
Figure 4: Result of loading a remote-located image
Picking Images from the Photo Gallery
In iOS, when you take a set of pictures and videos, they are put into the photo gallery, as shown in Figure 5. This lets the user easily see all the pictures he or she has taken, select them, and pull them back up to view.
Figure 5: Viewing an iOS photo gallery
The iOS photo gallery contains all the images stored on the device. The code to open the photo gallery is fairly simple, as shown in Figure 6.
UIImagePickerController picker;…......................if ( UIImagePickerController.IsSourceTypeAvailable(UIImagePickerControllerSourceType.PhotoLibrary) ){picker = new UIImagePickerController();picker.Delegate = new ImageGalleryPickerDelegate(this);picker.SourceType = UIImagePickerControllerSourceType.PhotoLibrary;this.PresentModalViewController(picker, true);}The photo gallery is managed by the UIImagePickerController class. The image picker class will manage user interactions and will deliver the results of these interactions to a delegate object. In this case, the delegate object is one that we have created and overridden some of the methods.
Now let's look at the delegate in the project for our UIImagePickerController class, shown in Figure 7.
public class ImageGalleryPickerDelegate : UIImagePickerControllerDelegate{PhotoGallery _pg;UIImageView imageView;public ImageGalleryPickerDelegate(PhotoGallery pg){_pg = pg;}public ImageGalleryPickerDelegate(){}public void ClearImage(){if ( imageView != null ){_pg.View.WillRemoveSubview(imageView);}}public override void Canceled (UIImagePickerController picker){_pg.DismissModalViewControllerAnimated(true);Console.WriteLine ("Cancelled picking an image");}public override void FinishedPickingMedia (UIImagePickerController picker, NSDictionary info){var originalImage = new NSString("UIImagePickerControllerOriginalImage");UIImage image = (UIImage) info[originalImage];imageView = new UIImageView(new RectangleF(0f, 0f, 320f, 460f));imageView.Image = image;_pg.View.AddSubview(imageView);_pg.DismissModalViewControllerAnimated(true);Console.WriteLine ("Did finish picking media");}}In this example code, we are inheriting from the UIImagePickerControllerDelegate class. The sequence of events is as follows:
The constructor of the method will take the instance of the class that created it. This will allow the delegate to communicate back to the calling class and interface with it through properties and methods that we will use later.
We will override the Canceled method. In this case, we will just go back to our application.
We override the FinishedPickingMedia method. This method is called when the user selects some media. In this method, when the media is selected, a UIImageView is created, programmatically added to the XIB/NIB file, and then the UIImagePickerController is dismissed so the image can be shown. In this specific example, we will select a picture of Wells and then display it on screen, as shown in Figure 8. Notice in the image that after the image is chosen, we can go back into our application and we know the image that was made available to us. There are other things that we can do, but I think this gets the point across.
Figure 8: Displaying media selected from the photo gallery
Taking Pictures
Taking pictures is an important part of any application. Let's look at the sequence of steps to take a picture.
The first step is to set up the UIRequiredDeviceCapabilities key in the info.plist file in the project. This key will document the capabilities required in a device to run an application. For this example, I have set up a "still-camera" requirement, as shown in Figure 9. For more information about using UIRequiredDeviceCapabilities, check out the References and Comments section at the end of this document.
Figure 9: Setting a "still-camera" requirement to enable picture-taking
Applications can programmatically check to make sure that the device that they are running on has the necessary hardware. This is done by calling the .IsSourceTypeAvailable static method on the UIImagePickerController.
The next step is to emulate what we did with taking a single image. In this case, we'll check to see whether the device has a camera. If so, we'll continue on and create a UIImagePickerDelegate and instantiate it, using the code in Figure 10.
var cameraType = UIImagePickerControllerSourceType.Camera;if(UIImagePickerController.IsSourceTypeAvailable(cameraType)){UIImagePickerController picker = new UIImagePickerController();picker.SourceType = cameraType;picker.Delegate = new StillImagePickerDelegate(this);this.PresentModalViewController(picker, true);}else { using (var alert = new UIAlertView("Whoops", "No Camera found", null, "Ok!", null)) { alert.Show(); }}The next step is to create our delegate, using the code in Figure 11.
public class StillImagePickerDelegate : UIImagePickerControllerDelegate{StillPicture _sp;public StillImagePickerDelegate(StillPicture sp){_sp = sp;}public override void Canceled (UIImagePickerController picker){_sp.DismissModalViewControllerAnimated(true);Console.WriteLine ("Cancelled picking an image");}public override void FinishedPickingMedia (UIImagePickerController picker, NSDictionary info){var originalImage = new NSString("UIImagePickerControllerOriginalImage");UIImage image = (UIImage) info[originalImage];image.SaveToPhotosAlbum(delegate(UIImage img, NSError err){Console.WriteLine("Saved!");} );UIImageView imageView = new UIImageView(new RectangleF(0f, 0f, 320f, 460f));imageView.Image = image;_sp.View.AddSubview(imageView);_sp.DismissModalViewControllerAnimated(true);Console.WriteLine ("Did finish picking media");}}Once the camera has taken the image, we will save the images to the gallery. Saving the image to the gallery is performed by the call to .SaveToPhotosAlbum(), which takes a delegate that can provide information on the save. Finally, we'll see another picture that I took of Wells laying in his favorite chair, shown in Figure 12.
Figure 12: Taking a picture and saving it to the photo gallery
Adding Filter Effects to Images
In iOS, Apple has included a library called Core Image. MonoTouch has bound the Core Image library, which is included in the MonoTouch.CoreImage namespace. Core Image is an API that provides a set of image filters that can be applied to video and still images. Core Image for iOS does not currently support creating custom filters. A major feature of Core Image is that it uses programmable graphics hardware whenever possible. This provides for nearly real-time processing.
In this example, let's look at a simple use of Core Image to apply a filter to an image. The steps to perform this are
Load an image. In this example, we'll just load an image from our project.
Create a CIImage from the UIImage. This will be used to interface with our image.
The next step is to create our filter. In this case, we'll set up a CISeptiaTone filter, set up a series of its properties, and then create the resulting image.
Finally, we will create our output image and display it to the user, as shown in Figure 13.
var uiimage = UIImage.FromFile("./Images/IMGCAT.jpg");var ciimage = new CIImage(uiimage);var hueAdjust = new CIHueAdjust();hueAdjust.Image = ciimage;hueAdjust.Angle = 2.094f;var output = hueAdjust.OutputImage;var context = CIContext.FromOptions(null);var cgimage = context.CreateCGImage (output, output.Extent);var ui = UIImage.FromImage (cgimage);img.Image = ui;Finally, out of this code, you can see the effect of filtering in the images shown in the table in Figure 14.
Figure 14: Applying a filter to an image
Animation
Generally speaking, animation is the rapid display of images. This creates the illusion of movement. These images can be either 2D or 3D. With computers, animation can be performing using a variety of mechanisms. We'll start off by looking at the simple animation features of the UIImageView. In part 2 of this article, we'll look at some more complicated animation using the NSTimer and the UIView animation.
Simple Animation of Images
There are instances where our applications have a limited set of images. We may need to animate between these images. Fortunately for developers, it is fairly easy to animate between a set of images in iOS. The general steps to perform this are
Load all the images that need to be loaded. This should be done once when the view is loaded, or in a similar location.
Create an array of these images. This will be assigned to the .AnimationImages property of a UIImageView.
The .AnimationDuration property will need to be set. This will be the number of seconds that each image is displayed before the UIImageView loads the next image.
The .AnimationRepeatCount will be used to set the number of times the array of images will be iterated through. By setting this value to -1, the images will be continually displayed in a loop.
The final step to begin the animation is to call the .StartAnimation() method. This will begin animating the images.
When the user moves away from this view, the application should be a good citizen and not continue to take up CPU and memory resources. Therefore, a call to the UIImageView's /StopAnimation() method should be set up at some point in the app. In our example, we will do this in the ViewDidUnload(). For your application, this should probably be done in another location.
Let's take a look at our code, shown in Figure 15.
public override void ViewDidLoad (){base.ViewDidLoad ();this.Title = "Images";var image1 = UIImage.FromFile("./Images/IMGCAT-1.png");var image2 = UIImage.FromFile("./Images/IMGCAT-2.png");var image3 = UIImage.FromFile("./Images/IMGCAT-3.png");var image4 = UIImage.FromFile("./Images/IMGCAT-4.png");imageView.AnimationImages = new [] {image1, image2, image3, image4};imageView.AnimationDuration = 4;imageView.AnimationRepeatCount = -1;imageView.StartAnimating();}public override void ViewDidUnload (){base.ViewDidUnload ();imageView.StopAnimating();}The output of our code is the image shown in Figure 16. In this example, the code will rotate Wells as he waits to go out the door. Thankfully, gravity keeps Wells from falling on his head. Somewhere in all this there is a joke about cats, butter, falling, and landing on their feet but butter-side down all at the same time.
Figure 16: Result of image-animation code
More Multimedia to Come
OK, so that's it for part 1. We've looked at displaying an image, using the camera, and how to code some basic animation using the UIImageView. In part 2, we'll look at some more complicated animation, watching and recording movie/video, and editing a movie. Stay tuned!
Wallace B. "Wally" McClure is an ASPInsider, member of the national INETA Speakers Bureau, author of seven programming books, and a partner in Scalable Development.
Email: [email protected]
Twitter: @wbm
Blog: www.morewally.com
Website: www.aspnetpodcast.com
[sidebar]
References and Comments:
This code was written and verified with iOS 5.01, MonoTouch 5.2.x, and MonoDevelop 2.8.6.4.
Chris Hardy wrote the Multimedia Chapter in our book, Professional iPhone Programming with MonoTouch and .NET/C# (Wrox). He has been a great friend and resource over the years.
Martin Bowling was helpful regarding a few of the finer points of the UIImagePickerController.
Two additional books, by Mike Bluestein and Bryan Costanich , respectively, are also worthwhile MonoTouch references. There is no competition amongst authors in my mind (we're fighting over too few peanuts). Mike and Bryan have been great friends and resources over the past few years as I have jumped head first into the iPhone and Android.
More information on topics discussed in this article:
More Dev Pro articles on iOS development:
About the Author
You May Also Like
