How 3D visualisation could unlock airspace challenges

Credit: Aerospace & Rail Research group

The ATS was developed by the Aerospace & Rail Research group, as part of the Virtual Airspace and Tower Project, and can simulate realistic actual and future worldwide gate-to-gate air traffic. For the high-fidelity prototype, the game development platform Unity3D was used. "The prototype provides a graphical user interface and an auditory display for voice control and warnings," says Rottermanner.

Features include plane visualisation, planned routes, separation helpers, spatial audio warnings for upcoming conflicts and extensive navigation possibilities in a three-dimensional space. It is possible to display air traffic both in orthographic (two-dimensional) and perspective (three-dimensional) ways.

The use of two-dimensional visualisation has been happening for many years and is a standard for air traffic management. However, with three-dimensional visualisation, ATCs can see in a virtual space, allowing them to have enhanced situational awareness. The system also labels air traffic, making it easier to identify and track.

It is clear something was needed to help better manage airspace. Described as "dysfunctional" by one industry representative, Europe's airspace has long been the busiest in the world. With it comes significant levels of stress when managing the skies. ATCs must deal with complex systems and processes, all supported by their extensive knowledge and expertise. They are in control of any number of flights, passengers and of course lives when carrying out their role, which can bring immense pressure. 

"Involvement of the target group was an important aspect for us, we wanted to know how ATCs fulfil their daily tasks and what are the challenges in everyday work," Rottermanner explains. The prototype is a result of interviews, focus group with ATCs, on-site visits of approach and terminal control centres, and the development and evaluation of three low-fidelity prototypes.

As part of their work, the research team conducted a study to assess how the different visualisation types — two dimensional versus three dimensional – contribute to performance and user experience during air traffic control. It looked at the effect of both visualisation types and user profiles on performance, situation awareness, workload and user experience, in a laboratory test environment. It involved 11 ATCs working as approach controllers operating at three different airports, ranging from approximately 15,000 to 400,000 aircraft movements in 2018.

In the tests, ATCs had to control the aeroplanes using their voice. There were two tasks - although each participant would only carry out one - lasting half an hour each; one was two dimensional, the other three. "After each task, participants had to fill out the SASHA_Q for assessing situation awareness and the NASA Task Load Index for assessing the workload questionnaires. At the end of the test, they had to fill out the User Experience Questionnaire to assess the whole user experience with the prototype," says Volker Settgast, a senior researcher at Fraunhofer Austria Research GmbH.

Additional questions assessed the implemented features and interactions, as well as a semi-structured interview to obtain additional feedback on the test and the prototype. "The results indicate that a three-dimensional view is an alternative way to visualise four-dimensional trajectories and should be further explored in different tasks and in other applications," Settgast adds. 

The team say they found that a three-dimensional representation could also have potential the in teaching and training of future ATCs. "The prototype test with students showed advantages due to the lower abstraction compared with the two-dimensional representation," says Settgast.

Credit: Aerospace & Rail Research group

The project was carried out in the hope of supporting the ongoing Single European Sky ATM Research Programme (SESAR). Launched by the European Commission in 2004, the programme aims to harmonise and synchronise the airspace above Europe, allowing for more flights at any one time and a reduction in emissions. Of course, it is much more complex and all underpinned by innovations in technology.

In time, SESAR advocates hope that four-dimensional trajectories will be incorporated into the management of Europe’s skies, meaning flights will be able to travel from one destination to another without needing to follow flight paths, as has previously been the case. However, the move will further increase the pressure on ATC providers and controllers. 

“Conflicting points will not be at fixed locations anymore, which contrasts with the current situation, where conflicts mostly appear at specific points where airways are crossing,” says Settgast. “ATCs are then required to envisage complex combinations of parabolic trajectories. Route and altitude changes can have an impact on the whole and other trajectories.”

Although the prototype has been developed, the team don’t see it as becoming widely available for use any time soon. In fact, they are keen to stress it is a starting point for discussion and should lead to more research in this field. “Based on our experiences within this project, it is very difficult to say whether this kind of three-dimensional representations will ever make it into real use,” concludes Settgast.

There is a lot more to do say the team, suggesting that a standardised testing framework for air traffic management software developers to evaluate new ideas is needed. “Another challenge in air traffic management is the availability of controllers for testing new features. We want to create testing methods that can also be performed by non-experts and then be transferred to give certain trends for tests results with experts,” Rottermanner adds.

It seems this technology is currently taxiing along the runway, not yet pressing its passengers into their seats. But it is one that could facilitate take off in other areas, if not this; a hope the team shares. Rottermanner concludes: “Our results indicate that a three-dimensional view is an alternative way to visualise four-dimensional trajectories and should be further explored in different tasks and in other applications.”

Credit: Aerospace & Rail Research group