Introduction

The idea

The project is inspired by our childhood game, Chicken Invaders.

We used AR technology to bring Chicken Invaders to life.

Using Microsoft HoloLens & Unity AR technology we succeeded in our mission, and below are the results.

Tools and Technology

Microsoft HoloLens 1

With HoloLens, Microsoft takes augmented reality to a whole new level, by combining cutting-edge optics and sensors to take 3D holograms into the real world.

Mixed Reality Toolkit

MixedRealityToolkit is an application framework providing scripts and components for accelerating the development of mixed reality applications.

Unity

Unity is a cross-platform game engine developed by Unity Technologies. Our chosen graphic framework is Unity AR Foundation.

Game Process

Environment Scanning

As the game begins, a scanner recognizes elements in its surroundings.

Wall detection

Upon finishing the scan, our algorithm filters the objects and detects vertical walls in the scanned environment.

Portal
Placement

As soon as the walls have been detected, our algorithm determines which walls are best suited for portal placement.

Game

ON!

After all preparations are made, the game begins and is managed by our Game Manager.

Key Challenges

1. Environment scanning and wall detection

We encountered many problems with environment scanning and wall detection during the development of the project. For example, tables, walls, and floors were not recognized.

Our problems were due to incompatibility between the code of Mixed Realty ToolKit (MRTK) that was designed for Microsoft's HoloLens 2 hardware and our HoloLens 1.

Our solution involved refactoring some code to fit HoloLens 1 hardware (of course, many HoloLens 2 features wouldn't work on HoloLens 1, but it satisfied our goal of recognizing walls in our environment.

2. Performance

At the time of its launch in March 2016, the HoloLens 1's CPU, GPU, and memory were of high quality, but today they are considered low-end technology. While developing the game, we encountered a few performance problems caused by graphic elements, including high GPU and CPU consumption, low rendering, etc.

We resolved our rendering and performance issues by using low polygon assets. Additionally, we reduced the number of objects displayed on the screen by "killing" objects after some time (for example, removing eggs and baked chickens) or stopping chicken generation once there are a certain number of chickens. In addition, we stopped scanning once our requirements were met, which decreased our CPU consumption.

3. Android Development

When porting to Android Platform, we encountered difficulties detecting walls and objects. These issues were caused by the differences between Unity AR Foundation and MRTK. As well as providing an infrastructure, the MRTK also creates a 3D environment mesh utilizing the sensors and cameras of the HoloLens.

However, when using Unity AR Foundation, the infrastructure uses only one camera (the main camera) and does not make use of other sensors and cameras (such as depth sensor) even if they are present in the device, and Unity AR Foundation cannot detect plain surfaces (this is a known issue).

Due to these limitations, a picture frame, TV, sofa, or non-plain wall will sometimes be selected as a portal location by our algorithm.