News Ticker

Controlling the DJI Phantom 3 Professional drone with an Oculus Rift VR

Design and development of generic framework for VR-headset-enabled drone control

Currently a large variety of relatively low-cost drones, all with differing sensor abilities and control methods, are coming to market. These drones can be remote controlled by an operator, and their sensor input can be retrieved while doing so. Most drones are controlled by a smartphone or a remote controller. Sensor in- put, like camera footage and altitude information, is visible in the vendor’s mobile application. The emergence of virtual reality headsets open new possibilities for controlling these drones and visualizing their sensor information. Using a VR head- set for controlling the drone can give a real “I’m there’ experience as they allow to control the direction of the drone’s camera simply by moving the head. This is meant to mimic the natural motion of the head, so looking up and down will move the camera in those directions.

 

Due to the variety in drones and VR headsets, regarding the control methods and sensor abilities, a software framework that can provide the necessary abstractions is required. The framework will need to provide abstractions for dealing with different types / brands of drones on the one hand, and different Virtual Reality goggles on the other.

Connecting a number of drones to the framework would allow an operator to have a nice overview of all the different drone camera images and would allow an operator to switch between the active drones on the fly, and take over the control with his head.

 

dji

 

As currently all drones and VR headsets on the market have different control methods, research on this topic is paramount. The main objective is the design and creation of a framework prototype focusing on interoperability, responsiveness and modifiability. The framework will need to provide abstractions for dealing with different types / brands of drones on the one hand, and different Virtual Reality goggles on the other. The second focus point will be on techniques to minimize the experienced lag between head movements and the visual representation in the headset, when controlling the different drones. The final focus point is on easy modification of existing plugins to deal with SDK updates.

Practical applications :

For example, if a public place or an event needs to be monitored, this is usually done by placing a few cameras spread all across the field. Then, the camera images are monitored by a security member. However, when having a few drones in the air and all the drone camera images in a VR headset, it would allow the operator to keep a nice overview of the whole field, but also to take a closer look when an incident happens.

drone vr

Master’s dissertation submitted in order to obtain the academic degree of Master of Science in Computer Science Engineering

by
 Stefaan VERMASSEN

Supervisors: Prof. dr. ir. Filip De Turck, Bruno Volckaert Counsellors: Ir. Thomas Dupont, Ir. Wannes Kerckhove

Faculty of Engineering and Architecture Ghent University

 

2 Comments on Controlling the DJI Phantom 3 Professional drone with an Oculus Rift VR

  1. We just received the yuneec q500 4k and its fun to fly with it. Does somebody know if its possible to connect it to samsung gear vr and stream the camera onto it?

    Like

  2. Will this work with the phantom 3 advanced?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: