Remote-me IOTA project briefing

A look inside the Remote-me IOTA project, for the digest of IOTA-ISO and interested parties.

This article is in response to the video kindly made by IOTA-ISO to introduce the Remote-me project to the IOTA-ISO community, and the AMA question and answer session scheduled for Monday 3rd December 2018 at 18:00 CET in the IOTA-ISO community discord channel.

Introduction

Remote-me is the nickname of an academic PhD project currently underway with the Open University.

The full title of the Remote-me PhD project is: “The Possibilities of Real World First Person View (FPV) Unmanned Aerial Vehicle (UAV) Interactions with the users of Virtual Environments”.

Remote-me is with a view to bringing a product, a UAV (“Drone”) to market. Specifically, one which can be remotely controlled in first person view via the internet, from anywhere.

History

The Remote-me project started in 2014 when the founder ceased his role with Cobham, after completion of work on the ASTRAEA project, as an industrial Research Engineer, with the setup of REMMI Ltd. unrelated to Cobham.

The founder’s role with Cobham had been as one of two Researchers. ASTRAEA was a civilian UAV project, working with the UK Civil Aviation Authority, to establish a means of virtual certification of UAVs. Cobham’s interest as the long standing world leader in-flight refueling technology, was to investigate refueling possibilities, thus the work focused on simulations involving the automatic in-flight refueling of UAVs. In this, Cobham worked in collaboration with multiple Universities and Industrial partners, with the founder working as a Cobham representative.

Upon the setup of REMMI Ltd, the full technical implication of data lag through the internet, on the possibility of a Remote-me scenario was not apparent. The project assumed a remote user could be connected to a UAV directly in first person view via the internet, and that a profitable business could result from manufacturing and selling UAVs built to enable this.

During the system development (Simulations), the problem of internet data lag, and the limitations this imposed on the ability to control the UAV in first person view became fully apparent.

After it became clear the business could not immediately offer a product, the founder began contracting his services as a Consultant Engineer for businesses unrelated to Remote-me, in order to financially support the project during further research on the product. REMMI Ltd. functioned as the contracting company, with all client funds paid to the company, and the founder working as an employee of the company.

The idea of controlling the UAV from a virtual world came from a simulation mode intended for operator training to learn how to pilot the remote UAV in a virtual world. It was noted in the training simulation that real time operation between the pilot and the simulated UAV was routinely possible.

A theoretical scheme of dynamically creating the real world scene in the virtual world from the video imagery of the real UAV was hypothesised. It then became apparent that the technology needed to do this dynamically (Structure-from-motion) did not yet exist.

An academic project was proposed to the Open University with a mission to research as needed to establish a practical means of controlling the UAV from a virtual world. The Open University accepted this proposal as an appropriate application for a part time PhD academic project. The Remote-me project embarked in this academic mode in February 2017, with the founder continuing Engineering Consulting to fund REMMI Ltd, which then funded the PhD research as the project industrial sponsor.

In effect, REMMI Ltd. had become a pure research company, with the founder working 60 hours per week; 40 hours for Engineering clients on unrelated business, and 20 hours per week in evenings and weekends on Remote-me PhD research work.

This was unsustainable, with the founder on some contracts having to work near client sites in alternative accommodation, resulting in project work slipping behind academic deadlines, less than ideal client work, and subsequent financial difficulties, for both REMMI Ltd, and the founder.

But, the academic research work has achieved the goal of discovering the Structure-from-motion solution needed for Remote-me.

In addition, the research work has also uncovered theoretical possibilities for a new distributed ledger AR/VR based ecosystem with world scale implications.

A new project to realise that ecosystem, VRENAR was created early in 2018. Research emphasis has since shifted to this, as the main research project.

The academic PhD project is intended to be continued to conclusion, including with this new information.

Present

REMMI Ltd. has served its purpose by taking the research as far as needed to uncover a solution for Remote-me.

As a conventional business based on making profit, REMMI Ltd. has no place in leveraging the financial benefits of a distributed ledger ecosystem such as VRENAR, as those are normally non-profits operating on an endogenous business model.

Further, REMMI Ltd. is not ideal as a Research Entity, as those are also normally non-profits; a standard requirement for attracting research funding from conventional research funding sources.

To take up the mantle of REMMI as a research entity, a new non-profit company more suited to the function has been founded, REMMI Research CIC. This is now the legal entity representing both the Remote-me and VRENAR projects.

Future

All profits resulting from Remote-me payable to REMMI Research CIC, will be reinvested to cover (a) previous outlays into Remote-me research, and (b) future outlays for Remote-me research.

Remote-me is being conducted as a financially separate project from VRENAR, though VRENAR has technical inputs from Remote-me.

Academic Project (“Remote-me”).

Problem

The power of VR is to be able to remote a person’s perceived point of view to a 3D location other than the real world around them. For a realistic experience, the user must see the new location as if they were actually there. The user must perceive a stationary environment relative to their head movements, so that they can move around the new environment, moving their head as necessary to look at whatever takes their interest, just as they would in the real world (Note that 3D 360 degree movies do not allow this).

So the viewpoint of the person in the new environment has to be able to exactly track the position of the person’s head, however the person moves.

Achieving this in a virtual world is routine, and this capability has existed since the development of virtual environments many years ago.

To achieve a similar floating 3D viewpoint in a physical real world remote location, we have to use some kind of physical platform, with a stereoscopic camera, which can move in response to the person’s head, i.e. a UAV.

But the lag of data through the internet is such that the UAV cannot move until a noticeable time after the movement of a user’s head. This makes the scenario of directly connecting the user to a real world remote UAV in first person view via the internet impossible (Imagine turning your head, and the world turning with it, until a second or so later, the world turns back to where you thought it was).

Solution

The Remote-me solution is to copy a visual replica of the static environment of the real world as seen by the UAV, into the virtual world. The user can then interact with the virtual environment in the same way as tried and tested over many years. The real world UAV movements then can track the user’s head movements with no consequence of time delay, as the user gets an impression of real time interaction with the virtual world.

The technology needed to do this has only recently come from other PhD projects, namely a form of Structure-from-motion (SfM).

The PhD goal is to show that the end-to-end scenario of a person remotely controlling a UAV in first person view via the internet, using an intermediate virtual world, is an acceptable practical solution.

It is intended to complete the PhD including also research details relevant to VRENAR. Completion of the PhD will thus be beneficial to both projects.

Commercial Project (“Remote-me IOTA”)

Value Proposition

A valuable by-product of the Remote-me scheme described above is the automatic instant creation of a virtual replica of the real world, as captured by a UAV.

This happens to be automation of an already much used business service; that of UAV survey service companies. Currently, those companies exist to provide 3D scanned models of built architectures and landscapes for customers wishing to have such. The service involves the use of specialists to fly the UAV, taking many static pictures from all angles around the subject, then processing the captured pictures after flight, using manual photogrammetry techniques and tools to produce the 3D survey model, which is then given to the customer.

A Remote-me UAV in theory enables a non-expert user to carry out all of the tasks of that service, by remote control, in a fraction of the time, by accident.

Our proposition here is to restrict the Remote-me technology to only those participating in the IOTA-ISO, and to offer to those funders exclusive ownership and use of a small number of UAVs using the technology.

Likely funders include those who might wish to protect or disrupt the UAV 3D surveying commercial market, those who might have a valuable site for Remote-me visitors to visit, and others.

We propose to produce 130 units incorporating the technology, in the form of a product-as-a-service, for distribution to funders. The expected price of each unit is around 10K Euros. This compares similarly to prices of other high-end, application specific commercial UAVs.

Each unit will incorporate a mechanism to meter and securely monetise throughput data. Thus a unit could be hired in return for revenue from throughput data, or access to a unit at a valuable site by a remote user could be offered by a site owner in return for revenue from throughput data, and other such Use Cases.

The revenue and data monetisation mechanisms will all operate using IOTA.

Current Status and Plan

The Remote-me project is making the above proposal with a view to funding the project via IOTA-ISO.

The project has a significant wealth of information around the Remote-me concept, which is captured in the Remote-me project UML/SysML model. This is the primary non-book artifact of the PhD project, where most of the work on Remote-me (And now also VRENAR) is carried out.

The model contains the Remote-me subsystems from which the system executable code will be generated. These subsystems include the code of the required Structure-from-motion functionality. This part of the model is in executable form, but needs some work to make fully functional as an end-to-end Remote-me simulation. Also, the IOTA data charging and monetisation will be incorporated in this stage (WP1). Six months work are allocated to work package WP1.

Once done, the simulation needs to be tested with the SfM sensor, which has been purchased for that purpose, to the virtual world (WP2).

Then the control loop for controlling the live UAV will be created including the characteristics resulting from the above tests (WP3).

The whole UAV with control loop will then be simulated, to fine tune the control loop with the UAV handling characteristics (WP4).

One months work is allocated to each of the work packages WP2, WP3, and WP4.

The prototype UAV will be physically constructed, by manufacturing the chassis using our composite carbon fibre 3D printer, and assembling all of the UAV components in the chassis. Much of this can be done in parallel with the tasks WP1 to WP4. All of the mechanical design work and parts purchasing have been completed, except for the camera gimbal and battery, the final details of which need to come from the simulation of WP2, & WP4 respectively (WP5). One months work is allocated to WP5.

After de-risking by the simulation work of WP4, the executable code of the UAV will be generated from the model and integrated in the Physical UAV. The physical prototype will then be tested and executable code refined where necessary (WP6). Two months work are allocated to WP6.

130 units of the UAV prototyped in WP6 will be manufactured (WP7). Six months work are allocated to WP7. The material costs of WP7 include purchase or hire of four additional 3D Printers of similar specification to that already owned by the project, at an expected cost of 5K Eur per printer.

footnote: A video transcript, kindly produced by IOTA-ISO of the subsequent AMA (“Ask me anything”) session is here: https://www.youtube.com/watch?v=5UQn6J142uU