PCT Home
Perceptual Control and Human Data Fusion
 

1. Introduction

2. Modes of Perception

3. Perceptual Control Introduction

4. Perceptual Control and imagination

5. Hierarchic Perceptual Control

6. Multiple data sources

7. Learning and Conflict

8. The Bomb in the Hierarchy

9. Degress of Freedom in the individual

10. Degrees of Freedom in the organization

11 Modes of Perception (Reprise)

12. Side Effects and Military intelligence

13. Communication

Information Overload in humans

  • Degrees of Freedom: At any level of the hierarchy, no more ECSs can be in active control than the number of effector degrees of freedom.
  • Sensor/Effector ratio: Humans have perhaps 104 times as many degrees of freedom for sensors as for effectors.
  • Relevant Control: Which perceptions are controlled varies over time. All perceptions are potentially controllable, but at any moment almost all are uncontrolled. There must be a way for the system to determine which perceptions to control at any moment, to avoid catastrophic failure.

Workload is an important issue. The rationale for the Data Fusion process is that the commander cannot attend to every message that comes in from every sensor system. The human has the same problem with the messages that come in from eyes, ears, skin, nose, and so forth. Not all perceptions can be controlled at the same time.

Certain facts are inescapable. One is that in an interwoven control hierarchy, no more independent perceptions can be under control at any one moment than there are independent possibilities for effectors to act on them. More specifically, around the circle from perception to action to the environment and back to perception, there must exist a bottleneck at which the number of independent degrees of freedom for variation is least. In humans, that bottleneck is not in the sensory system and is presumably not usually in the environment. It is likely to be in the restricted set of muscular operations that can be performed.

Degrees of freedom:

1. in the commander

The degrees of freedom available for control are limited by the smallest number at any level of abstraction. A rough count of human joints leads to an estimate of around 100 as an upper bound. At any level of abstraction no more perceptions than this can be under active control.

The human has on the order of 100 independent degrees of freedom for motion. Each optic nerve has around a million fibres that could, in principle, fire independently; each auditory nerve has around 30,000 fibres, and there are innumerable other sensory receptors. The data inflow from the senses would completely overwhelm the abilities of the human to act, were it not reduced by various techniques.

One technique for data reduction is passively statistical data fusion. The world displays many stable correlations: if one point in the visual field is bright, a very nearby point is likely also to be bright, and if it is not, then a neighbouring pair of points is likely to show the same brightness shift. If a point is bright at time t, it is likely to be bright a few milliseconds later. Sensory systems tend to respond strongly to change, either in space or in time, rather than reporting continuously the state of each individual point in sensory space at every moment.

A control net with more sensor than effector degrees of freedom

At each level, no more ECSs can be in effective control of their
percepts than there are effector degrees of freedom.

This figure illustrates the problem of degrees of freedom. It shows a hierarchy that has 14 sensor degrees of freedom but only 5 effector degrees of freedom. At each level of the hierarchy, no more than 5 independent perceptions can be simultaneously controlled, but which 5 are controlled from moment to moment is changeable. There is no need for passive data fusion. Indeed, passive data fusion would be deleterious in this situation, because it would preclude any change in which perceptions were under control. The available perceptions would be predetermined by the data fusion process.

There has to be some way in which the organisms can determine which perceptions are currently important to control. In a concrete sense, there is no need to control for distance from attacking tigers if there is no striped yellow object in sight and one is picking berries (thereby controlling many perceptions). But if a striped yellow animal appears, then it would be useful to stop picking berries and to control that distance perception by moving away. Humans, like all animals, have evolved to be able to shift control among a myriad of different perceptions.

Perceptions that are not being actively controlled may be “controlled” through imagination loops in the hierarchy. In other words, situation awareness can be maintained by tracking events in the world.

An alerting system may help to direct which degrees of freedom are actively controlled at any particular moment.

Although there are only a small number of controllable perceptions at any moment, the actual set that are controlled must be able to change in response to possible threats of opportunities. There are many possible ways in which such a shift might occur, including, as shown in this figure, the provision of a specialized "alerting system" dedicated to monitoring the world for the existence of patterns that might demand active perceptual control. The perceptions controlled might well be quite different from the perceptions that lead to the alert. A certain noise might be a pattern that alerts a car driver to park the car. The noise might have gone away, and therefore need not be controlled, but the threat of imminent police action must nevertheless be avoided.

Patterns leading to an alert require rapid response rather than accuracy. There is likely not to be time to ponder whether this object rapidly approaching through the air is a Bengal Tiger or some other kind of tiger. Such considerations of precision can wait until after the perception of “something jumping at me” has been brought to a zero level (the tiger has been avoided). Alerting systems must work massively in parallel, rapidly, and without too much concern for whether the situation exactly matches the alerting template pattern. They detect similarities with the template pattern, but do not distinguish subtle diagnostic differences that would demonstrate that the situation failed to conform to the pattern. They do not use “negative information,” the absence of anticipated data.