Introduction
A few months ago I received a Mac Mini and an iPhone 4S and got task to develop an augmented reality (AR) application for iOS, focusing on location services and GPS positioning.
A basic idea was to create an application that could display Points of Interest (POI) locations on the screen while video camera is active, and continously track POI locations as they change. Apple's push notification infrastructure was to be used to receive information about POI location change. The final goal is to be able to get a visual indication of the position of other people running the application on their mobile devices overlayed on the iPhone's camera output. To be able to do this, our mobile devices exchange information with a server that runs on top of the ASP.NET Web API framework. To make things more interesting, the location information can be sent from Android devices - one of my posts will cover basics of programming with Mono for Android. Please be aware of the fact that our company is not related in any way with the Mono Project - Mono Software is just a happy user of some of their products.
As AR development was a new experience for me, I figured that it would be a good idea to share my newly acquired knowledge with the rest of our community in a series of blog posts on our site. There is simply too much ground to cover in a single blog post, so please stay tuned for next articles in this series.
I set up my development environment using all latest tools, including Xamarin's MonoDevelop 3.0.3 with MonoTouch 5.1 framework and XCode 4.3.3 running on Mac OSX 10.7.4 Lion. My iPhone runs iOS 5.1.1.
We are using C# a a primary development language in Mono Software, and although Objective-C was a viable option, we tried MonoTouch, quickly fell in love with it and never looked back.
Preparing the Environment
Since I was working in a VS.NET / Windows environments for years now, developing on Mac was quite annoying, at least for the first couple of days. Different shortcuts, OS layout, installation processes – all that needs some time to settle down with a new user.
Installing tools on Mac
There are a lot of information on installing these tools, but things didn't work as described on many occasions, so I’ll try to give you a few hints to ease the process.
There is no good way to test sensor capabilities in the iOS simulator provided with MonoDevelop, and some of them are not testable at all, so we used an iPhone 4S 16GB and upgraded it to iOS 5.1 right after unpacking. During the installation process your AppleId must be created, and (if you want to develop for iPhone), an iOS developer account needs to be created at the same time. This account will be used for installing XCode.
The next step was to install MonoDevelop and MonoTouch, and it didn't go smoothly as expected. My version of MonoDevelop requires XCode 4.3 (I’ll describe using XCode with MonoDevelop later). So I went to the Apple store and downloaded the latest XCode. To my surprise, I was greeted with the message that XCode 4.3. requires Mac OS X 10.7 Lion. It took some time, but it payed of, and I installed OS X Lion, XCode 4.3.2 and MonoDevelop 3.0.3.2, in that order.
Installing Certificates
To be able to deploy application to your iPhone, besides developer certificate and key, you also need certificate and provisioning profile for the application. Our application will use push notifications, so we need a push provisioning profile.
I found a great post with step by step instructions for installing these certificates, profiles and configuring XCode for deployment on the iPhone, provided by Matthijs Hollema: http://www.raywenderlich.com/3443/apple-push-notification-services-tutorial-part-12 . Since I wanted to build MVC 4 WebApi application for sending push notifications to APNS (Apple Push Notification Server), some time was spent configuring Windows to allow me to send messages to APNS. At the end, it was all fairly simple. As Matthiys said in his post, you need to pack your certificate and key in a p12 file and then import that file in Windows certificate store.
Creating an iOS Application
We will be using MonoDevelop for writing application code, building and deploying, and XCode will be used for drawing the application user interface.
View And Controllers - Main View
We are developing an iOS application in MonoDevelop using the MVC approach, which means that for every screen we want to display, a View part and a Controller part are created. Our view part contains layout data and is designed in XCode, while Controller is a class which inherits from UIViewController and contains all logic related to the view. We use controller to show, update and remove views. Every view can be container for other subviews, and every subview can be container for other subviews, and so on. So when we are talking about controls we are thinking of subviews contained within another view.
For simplicity's sake, our application will have only one main view and only one controller. Since we will add all controls programmatically, we don’t need to do anything with our view using the XCode designer. Main view will display POI locations with camera preview in the background. It will also contain a button for connecting to our server application and labels for displaying number of POIs monitored. We also want to provide the ability to lock user's location and enable or disable each of his sensors. Here is how this view should look like in the landscape mode:
Creating the solution
We will start by creating a new Single View Application project using MonoDevelop and rename the default controller to iPhoneAppViewController.cs (and .xib). This class will contain all our application logic. I refactored a big part of this logic into helper classes that I will describe in my other posts. We will use all those helper classes in this project - they are located in the Classes folder. Note that a zip file that is attached to this post contains a complete solution that can be used in your scenarios. Here is how it looks in the Solution Explorer:
Along with our helper classes and the main application controller there are folders Controls and Settings. Controls folder holds two classes with custom controls we will use in our view and the Settings folder contains ApplicationSettings.xml and ApplicationSettingsGenerator.tt - the XML file contains settinsg for our application and the T4 template file is used to generate C# representation of the XML file.
Configuring the solution
After setting the project properties, we need to set a few configuration parameters in the ApplicationSettings.xml: sensor update intervals, field of view settings and the application server URL that will be used to exchange data with the server - more on that in my next post. After you finish editing the XML file, just open the T4 template, save it and the ApplicationSettings class will be (re)generated. Here is the sample configuration I used:
<?
xml
version
=
"1.0"
encoding
=
"UTF-8"
?>
<
Configuration
>
<
LocationMonitor
>
<
ApplicationPurpose
>SampleApp location monitor</
ApplicationPurpose
>
</
LocationMonitor
>
<
MotionMonitor
>
<
GyroUpdateInterval
>0.1</
GyroUpdateInterval
>
<
AccelerometerUpdateInterval
>0.1</
AccelerometerUpdateInterval
>
<
DeviceMotionUpdateInterval
>0.1</
DeviceMotionUpdateInterval
>
</
MotionMonitor
>
<!-- Landscape mode
<
FOV
>
<!-- For iPhone in radians (float)2.0 * (float)Math.Atan (58.5 / 2.0 / 56.5);
// For iPhone 4S in radians (float)2.0 * (float)Math.Atan (4.592 / (2 * 4.28)); -->
<
Horizontal
>0.98475913484564</
Horizontal
>
<!-- For iPhone in radians (float)2.0 * (float)Math.Atan (21.5 / 2.0 / 56.5);
// For iPhone 4S in radians (float)2.0 * (float)Math.Atan (3.450 / (2 * 4.28)); -->
<
Vertical
>0.76624413062396</
Vertical
>
</
FOV
>
<
Notification
>
<
ApplicationServerUrl
>
</
ApplicationServerUrl
>
</
Notification
>
</
Configuration
>
Implementing the main controller
If we look in the
iPhoneApplicationViewController.cs, the first two regions are
Fields region with declarations of the fields used in our controller and the Controls region with declarations of UI controls mapped to the view.
Dependency injection is used to pass the reference to the application delegate to the main controller:
#region Constructor
public
iPhoneAppViewController (AppDelegate appDelegate) :
base
(
"iPhoneAppViewController"
,
null
)
{
app = appDelegate;
HandleMotionEvents ();
}
#endregion
Constructor is also used to hook up to the motion events of a device. This way, all changes of yaw and roll values will trigger the update of the main view.
void
HandleMotionEvents ()
{
app.DeviceListenerShared.YawChanged += yaw => {
DrawFriends ();
UpdateDebugInfo ();
UpdateSensorsState ();
};
app.DeviceListenerShared.RollChanged += roll => {
DrawFriends ();
UpdateDebugInfo ();
UpdateSensorsState ();
};
}
We are overriding different event methods of a view and use them to update the appearance of our controls. Some of these methods are
ViewDidLoad() where controls are initialized, status bar is hidden and the device is subscribed for notifications.
DidRotate() contains code for setting the heading orientation and view layout updates. All of them basically contains calls to other helper methods with and no real logic, and the complete code can be found in the
Overrides region of
iPhoneAppViewController.cs in the attached solution.
Going further, when a device changes its orientation, the location manager
HeadingOrientation property must be updated, and
SetHeadingOrientation() is called to perform that task:
private
void
SetHeadingOrientation (UIInterfaceOrientation toInterfaceOrientation)
{
if
(app.DeviceListenerShared !=
null
)
return
;
if
(toInterfaceOrientation.Equals (UIInterfaceOrientation.LandscapeLeft))
app.DeviceListenerShared.LocationManager.HeadingOrientation = CLDeviceOrientation.LandscapeLeft;
else
if
(toInterfaceOrientation.Equals (UIInterfaceOrientation.LandscapeRight))
app.DeviceListenerShared.LocationManager.HeadingOrientation = CLDeviceOrientation.LandscapeRight;
else
if
(toInterfaceOrientation.Equals (UIInterfaceOrientation.Portrait))
app.DeviceListenerShared.LocationManager.HeadingOrientation = CLDeviceOrientation.Portrait;
else
if
(toInterfaceOrientation.Equals (UIInterfaceOrientation.PortraitUpsideDown))
app.DeviceListenerShared.LocationManager.HeadingOrientation = CLDeviceOrientation.PortraitUpsideDown;
}
The
DrawPOIs() method calculates the coordinates for each POI, alter coordinates for the POIs that are already on the screen, tries to find the existing control for the POI in the current view and if there is one updates its location, or otherwise create a new control for the POI. The same method finds the POI that is closest to the center of the screen to display the information about it.
public
void
DrawPOIs ()
{
btnData.SetTitle (String.Format (
"POIs: {0}"
, app.LocationsToDraw.Count.ToString ()), UIControlState.Normal);
var cameraLocation =
new
CLLocationCoordinate2D (app.DeviceListenerShared.Latitude, app.DeviceListenerShared.Longitude);
MKMapPoint currentCoord = MKMapPoint.FromCoordinate (cameraLocation);
var friends = app.LocationsToDraw;
int
nearestX = 0;
bool
nearestChanged =
false
;
bool
nearestOffScreen =
false
;
Circle nearestControl =
null
;
foreach
(var friend
in
friends) {
Circle btn =
null
;
btn =
this
.View.ViewWithTag (friend.Id)
as
Circle;
Point screenCoord = Helper.GetScreenPoint (
friend,
currentCoord, (
float
)app.DeviceListenerShared.Altitude,
(
float
)app.DeviceListenerShared.Yaw,
(
float
)app.DeviceListenerShared.Roll,
(
int
)
this
.View.Bounds.Height, (
int
)
this
.View.Bounds.Width,
(
float
)ApplicationSettings.FOV.Horizontal,
(
float
)ApplicationSettings.FOV.Vertical,
app.DeviceListenerShared.LocationManager.HeadingOrientation
);
//Find nearest POI
nearestChanged =
false
;
if
(Math.Abs (240 - screenCoord.X) < 25
&& Math.Abs (240 - screenCoord.X) < Math.Abs (240 - nearestX)) {
nearestX = screenCoord.X;
nearestPOI = friend;
nearestControl = btn;
nearestChanged =
true
;
}
UIColor color = GetAlwaysOnScreenAndColor (
ref
screenCoord,
ref
nearestChanged,
ref
nearestOffScreen);
int
circleSize = 10;
if
(btn ==
null
) {
btn =
new
Circle (screenCoord.X, screenCoord.Y, circleSize, 2, color);
btn.Tag = friend.Id;
btn.SetTitle (friend.Id.ToString (), UIControlState.Normal);
this
.View.AddSubview (btn);
}
else
{
btn.MoveTo (screenCoord.X, screenCoord.Y, color);
}
}
if
(nearestControl !=
null
) {
if
(nearestOffScreen)
nearestControl.SetTitleColor (UIColor.Yellow, UIControlState.Normal);
else
nearestControl.SetTitleColor (UIColor.Blue, UIControlState.Normal);
}
}
Main controller additionally contains logic for subscribing to notifications. This is done by sending a device token to our .NET WebAPI service (again, I will present more info on the server-side logic in one of the next articles). This token is used by APNS to identify iOS devices. Subscribing to the WebAPI service is done by using
AsyncCallback methods to keep UI more responsive. We are using
System.Net.WebRequest to send JSON data with the token to the server.
private
void
SendDeviceToken ()
{
if
(!String.IsNullOrEmpty (NSUserDefaults.StandardUserDefaults.StringForKey (
"deviceToken"
))) {
string
lastDeviceToken = NSUserDefaults.StandardUserDefaults.StringForKey (
"deviceToken"
)
.Replace (
" "
, String.Empty).Replace (
"<"
, String.Empty).Replace (
">"
, String.Empty);
string
url = ApplicationSettings.Notification.ApplicationServerUrl.Subscribe;
webRequest = WebRequest.Create (url)
as
WebRequest;
System.Json.JsonObject jsonNotif =
new
System.Json.JsonObject ()
{{
"Token"
, lastDeviceToken}};
messageBody = jsonNotif.ToString ();
webRequest.ContentLength = messageBody.Length;
webRequest.Method =
"POST"
;
webRequest.ContentType =
"application/json"
;
btnConnect.SetTitle (
"Connecting..."
, UIControlState.Normal);
btnConnect.SetTitleColor (UIColor.Yellow, UIControlState.Normal);
Console.WriteLine (
"Sending token..."
);
webRequest.BeginGetRequestStream (
new
AsyncCallback (GetRequestStreamResponse),
null
);
}
}
private
void
SendDeviceTokenResponse (IAsyncResult ar)
{
webRequest.EndGetResponse (ar);
if
(ar.IsCompleted) {
Console.WriteLine (
"Token sent."
);
}
else
{
Console.WriteLine (
"Token not sent!"
);
}
}
private
void
GetRequestStreamResponse (IAsyncResult ar)
{
Console.WriteLine (
"Waiting response..."
);
StreamWriter stOut =
new
StreamWriter (webRequest.EndGetRequestStream (ar), System.Text.Encoding.ASCII);
stOut.Write (messageBody);
stOut.Close ();
webRequest.BeginGetResponse (
new
AsyncCallback (SendDeviceTokenResponse),
null
);
Console.WriteLine (
"Token sending in progress..."
);
}
Using the sample application
Sample application should be pretty simple and easy to use. It is designed to support landscape interface orientation as a primary working mode, but will also work in the portrait mode. After application starts, its main (and only) view is displayed.
I have placed sensor-related buttons in the top left part of the main window. These buttons have three states: red color is used if a sensor is unavailable, yellow color is used if a sensor is not active but available, and green color is used if a sensor is active. Sensors are (from left to right): device motion sensor, magnetometer, gyroscope, accelerometer and heading. Clicking on the each button will toggle its state.
The bottom section of the view displays the current server connection state, a button for locking the device location and a button with number of POIs monitored. You can click on this button for testing purposes, and
Mono office location will be added as a POI.
The right section of the screen displays debug information. Touching this label will toggle its visibility state. Debug information include (in this order): latitude, longitude, altitude, yaw, roll, heading accuracy, horizontal accuracy, vertical accuracy, sensors state, reference frame and, if a POI is focused, distance to the POI and its altitude.
Complete solution is attached in a zip file, feel free to use it anyway you like.