Senses working overtime

Flying aircraft has never been easy. The Wright Brothers spent several years after their inaugural 1903 flight perfecting the three-axis control that would enable a person to fly a fixed-wing aircraft with any precision.Since then, aircraft have become immeasurably more complex. Modern fighter pilots use a wide range of sensors to fly their craft with split-second reactions, while the new fleets of unmanned aerial vehicles (UAVs) are guided remotely by ground-based pilots who rely entirely on machine sensors.Flying successfully means pilots have to make sense of a jumble of images and other data that come at them simultaneously. Making sure they can do so, and with constantly improving performance, is the job of the Air Force Research Laboratory’s Human Effectiveness Directorate.One of the three mission units of the AFRL’s new 711th Human Performance Wing at the Wright Patterson Air Force Base in Ohio, the directorate is focused on developing the technology that allows humans to operate the systems that make up the modern combat-ready Air Force.“The challenge with flying airplanes is how safely you can incorporate people into them,” said Maris Vikmanis, chief of plans and programs for the 711th. “Since World War II, airplanes have become so complicated that the human-machine interface is now critically important.”The Human Performance Directorate is near the top of all the world’s laboratories in terms of the breadth of the activities it engages in, he said. It must deal with a complex array of technology, a wide variety of missions and a national security environment that allows no room for error. At the same time, directorate officials are also trying to transform deeply rooted development processes that place human considerations second.Traditionally, the machine and the technology are designed first and then the pilot has to deal with what’s left over, usually through training, Vikmanis said.“We’d like to be able to design things in from the beginning that fit [the human] better,” he said.The growing prevalence of UAVs on the modern battlefield is helping to spur this change, said Dan Goddard, chief of the directorate’s Warfighter Interface Division. The desire now to have a single pilot operate multiple UAVs simultaneously means a well-designed human-machine interface is essential, he said.That requirement has carried over into other Air Operations Center weapons systems.“There’s now so much reconnaissance data flowing down into the AOC that it’s creating information overload,” Goddard said. “You need a much better human-machine interface to be able to get actionable information out of this very quickly.”For example, computer displays are basic equipment for any data-related task, but some situations require more than the regular 2-D view that standard devices provide. Analyzing columns and rows of numbers is fine with a spreadsheet application, but doctors evaluating a knee injury would do much better if they could see a ligament tear from all angles using a 3-D display.The same is true with certain command-and-control tasks. There’s only so much you can get out of a 2-D display, said Paul Havig, senior engineering psychologist for the Battlespace Visualization Branch of the Directorate’s Warfighter Interface Division. A third dimension means being able to see more of the data that’s on the screen and gives a better way of interacting with it, he said.“Being able to interact with the data means you get a much better understanding of them and what they mean,” he said. “With map displays, for example, you get a much better situational awareness.”In some cases, true 3-D may not be required. Havig’s branch is experimenting with so-called 2 1/2-D  displays that give a compelling sense of depth for anyone looking at the screen because of the perspective. For example, an image of a person standing in front of a wall on such a display may actually appear as though the person is separated from the wall, even though it’s not a real 3-D display.The advantage of 2 1/2-D is that images can be rendered on regular, inexpensive flat-panel displays, Havig said. True 3-D displays are much more expensive.The high cost of 3-D displays is not the only problem. The few 3-D displays that exist now are drastically different from each other, making it difficult to know how they would perform with different types of data, images and situations.“I can easily spec out to my folks what’s needed in a high-definition display, and if they went to the local Best Buy, they could pick out a fair number of displays that would meet the specs,” Havig said. “But we only have two 3-D displays in our lab, and if we try to spec out the contrast, luminance and other things, they are such different beasts.”Other research issues involve such tasks as deciding the best interface for people to use in interacting with on-screen data. A regular mouse turns out not to be so good for this; it would be better if someone could actually reach into the data to interact with it, which means devising more tangible interfaces.“But we don’t even know when the first 3-D displays will be used in an operational setting,” Havig said. “We’re still trying to work out when they would be useful and for what tasks.”Sound perception can play an equally important role in combat scenarios. On the battlefield, people often pick up aural cues about what’s happening before they see it. Developing technology that can take advantage of that is the goal of the Battlespace Acoustics Branch of the Warfighter Interface Division.“Sound sources are all around and people are good at telling what they are and where they come from,” said Doug Brungart, technology adviser at the Battlespace Acoustics Branch. “They turn their heads to where the sounds come from, and we are trying to use the same concepts for use by fighter pilots.”Pilots must track numerous sensors that have visual displays as they fly the aircraft and operate the weapons systems. Sound cues can be used to alert them to the most important area they need to pay attention to at a given time.That takes advantage of what Brungart called the cocktail party effect. While a crowded room full of voices and other sounds can seem like a cacophony, he said people are actually pretty good at perceiving individual sources.“Even if they are not aware of it at the time, people know when sounds change or are switched off in that kind of environment,” he said. “Most people are able to distinguish around six different sounds at any one time.”Human factor considerations are also important to less frontline military operations, such as coordinating global fuel tanker support for warfighters. The Air Force flies around 100,000 such sorties each year.It’s not an easy task. Controllers must juggle available ramp space at various airfields, heed the diplomatic clearances needed for flying over foreign airspace, and accommodate changes caused by weather and other factors.Current systems used for this are based on applications such as Microsoft Office, which don’t allow much insight into situational aspects of the operation, such as knowing the changing capacities at various airports for handling tanker missions.The Work-centered Interface Distributed Environment (WIDE) program in the Cognitive Systems Branch of the Warfighter Interface Division is looking at designing more intuitive systems that will give the controllers who h dle these tasks a better grasp on situations.“Guys can keep track of the capacities at particular airports and try to fix the problems there,” said Jeff Wampler, program manager for the WIDE program. “But missions can go through three or four different airports, crews need rest and so on. Problems have ripple type of effects for future missions, so people are in constant crisis mode trying to deal with this.”WIDE’s goal is to bring a more visual sense to the task of coordinating these flights. The first step is to understand the cognitive aspects involved by observing how people perform these operations.“The current dashboards that these people use are littered with red and yellow alerts, for example,” Wampler said. “We want to know how to bring the temporal contrasts of these out graphically, so operators can see the problems at a glance.”As helpful as many of these solutions sound, it’s not a simple matter of building them and expecting users to embrace them. It often seems to take as much effort to persuade people to use them as it does to develop them in the first place.There’s been a lack of awareness about the importance of the human-machine interface in the early designs of weapons systems, Goddard said. The directorate is constantly having to market itself and its work, to show what the challenges are and how its technologies and solutions can ultimately save money.The historical challenge with human interface technology has always been in putting a number on it, Vikmanis said. “We might intuitively know that a better interface leads to making better decisions,” he said. “But measuring effectiveness is a very hard thing to do.” 

Air Force research missions

The Air Force Research Laboratory’s Human Effectiveness Directorate has four divisions, each broken down into various branches of study.

Warfighter Readiness Research Division

  • Immersive environments.
  • Logistics readiness.
  • Continuous learning.
  • Cognitive models and agents.



Warfighter Interface Division


  • Battlespace acoustics.
  • System control interfaces.
  • Collaborative interfaces.
  • Cognitive systems.
  • Battlespace visualization.



Directed Energy Bioeffects Division


  • Biobehavioral systems.
  • Optical radiation.
  • Radiofrequency radiation.



Biosciences and Protection Division


  • Biomechanics.
  • Applied biotechnology.
  • Counterproliferation.
  • Biobehavioral performance.

— Brian Robinson









































Power of listening










Simplifying the picture




















X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.