# Knowledge is power. We love to share it.

News related to Mono products, services and latest developments in our community.

## iOS Augmented Reality with some trigonometry

### Augmented Reality and Points Of Interest

Augmented Reality (AR) is extending user's experience of the world around him using computer generated objects. Its goal is to add information and meaning to a real object or place by introducing virtual objects into the camera view.

There are several concepts that a developer must grasp when preparing to write an Augmented Reality app: OpenGL, translations, rotations, scaling etc. To understand basic OpenGL principles, some working knowledge of analytic geometry is needed. In this post I will try to describe these concepts, as they are used in our demo app which is presented in my first article on AR and iOS.

The goal is simple: we want to display user's Points Of Interest (POI) when he points the device in any direction. Of course, this must happen in real time, and POIs must be placed intuitively indicating their real location and other properties. POIs will be displayed using simple colored circles, as we don't want to complicate things and use a third-party graphics library for the job - just simple custom controls and some mathematics. I already posted sample code containing instructions for creating custom controls with MonoTouch. In that post I've shown how to create a custom control with button functionality that has a clear background and a circle in the middle. We want to place this control on screen over the POI location.

### Basic Trigonometry

To draw a control on the screen, we need to calculate screen coordinates for each POI. To determine which POI should be made visible and displayed on the screen - relative to the direction in which the device is pointed - we will observe the slope (or gradient) of a line.

As Wikipedia says , "slope is normally described by the ratio of the "rise" divided by the "run" between two points on a line." Using the slope of a line which is defined by 2 points - the location of our device and the location of the each POI - we will calculate the inclination in radians. Here are more details on the slope, inclination and other important concepts.

### Input Data

We will use latitude, longitude and altitude as our location input data along with yaw and roll values to determine device orientation towards the ground. Yaw and roll (and pitch also) are represented by Euler angles given to us by iOS, to represent the device orientation (attitude) along its center. This functionality is part of the CoreMotion framework which is available in iOS 4.0 and above, and all available iPhone sensors are combined together to give most accurate value. Here you can see how data from the sensors can be collected.

iOS 5.0 introduces the reference frame for attitude samples. We are using yaw data collected using the *CMAttitudeReferenceFrameXArbitraryCorrectedZVertical *frame. This constant is rather CPU expensive, but we want the most accurate and the most relevant data we can get. This constant also uses magnetometer to improve the long-term yaw accuracy.

Yaw value can range from -PI to PI for full horizontal circle and north is at zero. Be sure to set the heading orientation when user rotates the device. We want our yaw values to always be positive and use it as inclination in our calculation.

`// Heading correction`

`float`

`inclinationX;`

`if`

`(yaw < 0F) {`

` `

`inclinationX = yaw + TwoPI;`

`} `

`else`

`{`

` `

`inclinationX = yaw;`

`}`

`//Heading correction`

`float`

`inclinationY = (`

`float`

`)Math.Abs (roll) - HalfPI;`

`if`

`(inclinationY <= 0.0)`

` `

`inclinationY += TwoPI;`

*MetersBetweenMapPoints*which calculates distance between two MapPoints. We can use

*MKMapPoint.FromCoordinate*to create our

*MapPoints*based on the location data.

*MetersBetweenMapPoint*is available in iOS 4.0 and never versions.

`var cameraPoint = MKMapPoint.FromCoordinate(`

`new`

`CLLocationCoordinate2D(cameraLatitude, cameraLongitude));`

`var poiPoint = MKMapPoint.FromCoordinate(`

`new`

`CLLocationCoordinate2D(poiLatitude, poiLongitude));`

`float`

`distanceAB = (`

`float`

`)MKGeometry.MetersBetweenMapPoints(cameraPoint, poiPoint);`

### Calculating Coordinates

We are displaying POIs on top of the video preview image, so we need to know which POIs will fit into the field of view of the camera. I found the exact values for the iPhone's camera field of view here: http://stackoverflow.com/questions/3594199/iphone-4-camera-specifications-field-of-view-vertical-horizontal-angle.

Armed with this knowledge we can calculate values for the coefficients for x and for y coordinates. Using the law of cosines and knowing the inclination of a line for x and y, we can calculate the distance between two points on the unit circle.

One point on the unit circle represents the inclination of our POI, while the second point represents the inclination of a device respective to the camera heading.

`#region Coordinate X`

`// Heading correction`

`float`

`inclinationX;`

`if`

`(yaw < 0F) {`

` `

`inclinationX = yaw + TwoPI;`

`} `

`else`

`{`

` `

`inclinationX = yaw;`

`}`

` `

`float`

`distanceX = (`

`float`

`)Math.Sqrt ((`

`float`

`)2 - (`

`float`

`)2.0 * (`

`float`

`)Math.Cos (inclinationX - inclinationXPOI));`

` `

`if`

`(inclinationX < inclinationXPOI) {`

` `

`distanceX = -distanceX;`

`}`

`if`

`(inclinationX <= TwoPI && inclinationX >= (3 * HalfPI) && inclinationXPOI >= 0 && inclinationXPOI < (HalfPI)) {`

` `

`distanceX = -distanceX;`

`}`

` `

`screenKoefX = distanceX;`

`screenX = Convert.ToInt32 (((`

`float`

`)viewWidth * screenKoefX));`

`#endregion`

` `

`#region Coordinate Y`

`float`

`inclinationYPOI = (`

`float`

`)Math.Atan ((`

`float`

`)(poiAltitude - altitude) / distanceAB);`

`if`

`(inclinationYPOI <= 0.0)`

` `

`inclinationYPOI += TwoPI;`

` `

`//Heading correction`

`float`

`inclinationY = (`

`float`

`)Math.Abs (roll) - HalfPI;`

`if`

`(inclinationY <= 0.0)`

` `

`inclinationY += TwoPI;`

` `

`screenKoefY = (`

`float`

`)Math.Sqrt (((`

`float`

`)2 - (`

`float`

`)2.0 * (`

`float`

`)Math.Cos (inclinationYPOI - inclinationY)));`

`if`

`(inclinationYPOI < inclinationY) {`

` `

`screenKoefY = -screenKoefY;`

`}`

`if`

`(inclinationYPOI <= TwoPI && inclinationYPOI >= (3 * HalfPI) && inclinationY >= 0 && inclinationY <= (HalfPI)) {`

` `

`screenKoefY = -screenKoefY;`

`}`

`if`

`(inclinationY <= TwoPI && inclinationY >= (3 * HalfPI) && inclinationYPOI >= 0 && inclinationYPOI <= (HalfPI)) {`

` `

`screenKoefY = -screenKoefY;`

`}`

`screenY = Convert.ToInt32 (viewHeight * screenKoefY);`

`#endregion`

After this, we just need to multiply the width and height of our view with the coefficients and we have values expressed in right measurement units - note that they are positive for POIs in the top half of the view and negative in the lower half. We simply need to add the half of the screen height to our y the coordinate and half of the screen width to the x coordinate, and we'll get our coordinates ready to be displayed on the screen. Note that this code snippet is valid only for the landscape device orientation, but it should be pretty simple to enable it to work in the portrait mode also. Just be careful about FOV.

`return`

`new`

`Point (`

` `

`Convert.ToInt32 ((viewWidth / 2.0) + screenX),`

` `

`Convert.ToInt32 ((viewHeight / 2.0) - screenY)`

`);`

You can find more info about iPhone field of view (FOV) here.

What values are you getting for yaw?

Did you set your heading orientation correctly?

This Delegate Method Invoked As The Device Heading Changes

- (void)locationManager:(CLLocationManager *)manager didUpdateHeading:(CLHeading *)newHeading

{

CMDeviceMotion *currentDeviceMotion = MotionManager.deviceMotion;

CMAttitude *currentDeviceAttitude = currentDeviceMotion.attitude;

double Roll = currentDeviceAttitude.roll;

double Pitch = currentDeviceAttitude.pitch;

double Yaw = currentDeviceAttitude.yaw;

double Magnetic_Heading = newHeading.magneticHeading;

double True_Heading = newHeading.trueHeading;

}

The Problem Is That, How To Calculate The Value Of inclinationXPOI ????????

This Is My Code

// Distance Between User & POI

CLLocationCoordinate2D Coordinate_2 = CLLocationCoordinate2DMake([ObjDeal.Latitude floatValue],[ObjDeal.Longitude floatValue]);

CLLocation *Location_2 = [[CLLocation alloc] initWithCoordinate: Coordinate_2 altitude:1 horizontalAccuracy:1 verticalAccuracy:-1 timestamp:nil];

distanceAB = [Location_2 distanceFromLocation:Location_1];

// Inclination Of X

if (Yaw < 0)

{

InclinationX = Yaw + 2 * PI;

}

else

{

InclinationX = Yaw;

}

// Inclination Of Y

InclinationY = (double)fabs(Roll) - ((1/2)*PI);

if (InclinationY <= 0)

{

InclinationY = InclinationY + 2 * PI;

}

// Inclination Of POI X

InclinationPOIX = ????????????????????????????????????

// Inclination Of POI Y

InclinationPOIY = (double)atan((double)(([ObjDeal.Altitude doubleValue] - Altitude) / distanceAB));

if (InclinationPOIY <= 0)

{

InclinationPOIY = InclinationPOIY + 2 * PI;

}

// Screen Cofficient X

ScreenKoefX = (double)sqrt((double)2.0 - 2.0 * cos(InclinationX - InclinationPOIX));

if (InclinationX < InclinationPOIX)

{

ScreenKoefX = -ScreenKoefX;

}

if (InclinationX <= (2* PI) &&

InclinationX >= (3 * 1/2 * PI) &&

InclinationPOIX >= 0 &&

InclinationPOIX < (1/2*PI))

{

ScreenKoefX = -ScreenKoefX;

}

// Screen Cofficient Y

ScreenKoefY = sqrt ((2.0 - 2.0 * cos (InclinationPOIY - InclinationY)));

if (InclinationPOIY < InclinationY)

{

ScreenKoefY = -ScreenKoefY;

}

if (InclinationPOIY <= (2* PI) &&

InclinationPOIY >= (3 * 1/2 * PI) &&

InclinationY >= 0 &&

InclinationY <= (1/2*PI))

{

ScreenKoefY = -ScreenKoefY;

}

if (InclinationY <= (2* PI) &&

InclinationY >= (3 * 1/2 * PI) &&

InclinationPOIY >= 0 &&

InclinationPOIY <= (1/2*PI))

{

ScreenKoefY = -ScreenKoefY;

}

// Screen X

ScreenX = ViewWidth * ScreenKoefX;

// Screen Y

ScreenY = ViewHeight * ScreenKoefY;

// Exact Point

Point = CGPointMake((ViewWidth / 2 + ScreenX), (ViewHeight / 2 + ScreenY));

Pleas Help Me AS Soon As You Can

Thanks

float inclinationXPOI = (float)Math.Atan(rise / run) - HalfPI;

if (run < 0.0)

inclinationXPOI += OnePI;

if (inclinationXPOI < 0.0)

inclinationXPOI += TwoPI;

You can find complete method for calculating screen coordinates in attached sample in class Helper.cs. Method is called GetScreenPoint and code you are missing is located in lines 46 to 51 of this class.

If other devices are far away, like 500 m or more, then objects should be displayed accurately but on smaller distances there can be errors since GPS sensors are returning coordinates with 2 m tolerance and GPS data are intentionally sent wrong from server.

This code fetches most accurate sensors values by using CMAttitudeReferenceFrameXArbitraryCorrectedZVertical constant.

In http://www.mono-software.com/blog/post/Mono/187/Building-an-augmented-reality-app-for-iPhone-with-MonoTouch/ post you can find complete application using this code.

as i move the device direction towards the particular location, the target objects are being displayed, it is good,

but the problem is that, same objects are being displayed in the opposite direction, or with Pi / 2 phase difference.

what i have to do, kindly guide me, now i am using altitude using CMAttitudeReferenceFrameXArbitraryCorrectedZVertical as u said.