@Article{ChandrasekaranNarayananIwasaki1993,
author = {B. Chandrasekaran and N. Hari Nayanan and Yumi Iwasaki}
title = {Reasoning with Diagrammatic Representations},
journal = {AI Magazine},
year = {1993},
volume = {14},
number= {3},
pages = {49--56}
}
Symposium held by American Association for Artificial Intelligence (AAAI) March 25-27, 1992. [49]
Focused on issues relating to representations of diagrams and mental images and their functions in problem solving.
Multidisciplinary audience including philosophers, cognitive psychologists, design theorists, logicians, and AI researchers.
Diagrammatic Reasoning: A New Emphasis in AI--> Interest in using various types of representations (including external world as representation) and integrating perceptual and motor components with explicitly cognitive elements.
Organization--> Symposium organized by B. Chandrasekaran, Yumi Iwasaki, N. Hari Narayanan, and Herbert Simon. [50]
Consisted of 5 moderated sessions:
Categories of Diagrammatic Representations by Location:
There was agreement as to the phenomenal reality of mental images, but the extent to which they can be called pictures was debated.
Tradition in philosophy and psychology suggests images are epiphenomenal: Have no causal role in reasoning or problem solving.
The quality of being picture like is discussed from three perspectives: the actual experience, the neural pattern that gives rise to the experience, and the neural pattern that stores the mental image in memory. [50-51]
Memory: One can discuss what is the best language to use to describe it, but it is pointless to ask if the memory itself (a neural pattern) is a picture or set of propositions because it is neither: it is a neural pattern.
The same points can be applied to the neural pattern that gives rise to the phenomenal experience of an image.
No agreement among AI logicians as to the best language for describing mental images. Some argued that predicate logic is sufficient and that images are not privileged as a language to describe this content. Others (J. Baswise, Indiana University at Bloomington, and J. Etchemendy, Stanford) argued that pictures do have unique qualities as representational categories.
Some theories suggest activation patterns associated with seeing have a systematic spatial array character. If the phenomenal experience of an image shares commonalities with the seeing patterns it can be said to have picture like properties.
Memory for images is constructivist, but it is not clear how far the theory goes or whether there are “visual primitives in representation”.
Problem Solving Using Vision: involves shifting visual attention to extract information pertinent to current subgoals. [52]
Drawing: “external representation (...) of relevant aspects of a visual domain”
Drawings for Problem Solving: Drawings are simplified, but preserve information needed for a problem. Often abstract – intended to represent problem-relevant information in a way that is easily extracted visually.
Imaging: Making mental images of an aspect of the real or a fictional world. Some issues similar to drawing. Questions about generation and processing.
Using Mental Images for Problem Solving: Some issues similar to drawing in problem solving. Meaning of scanning and the role of perception is unclear.
Mental imaging shares the visual buffer with perception.
The visual buffer is the last stage of visual processing in which the content is already perceptually interpreted as objects in space. Only the content of this buffer is available for perception or construction (of mental images).
Some motor and perceptual processes are shared for seeing the world, a drawing, or using a mental image. Scanning may also be shared, but it is possible that scanning a mental image is only metaphorical. The image may be reconstructed partially to mimic the results of scanning.
In constructionist view images are reconstructed in the visual buffer. The reconstructions are based on objects that are stereotyped, resulting in imperfect images, and thus sharing the errors and biases of drawing.
“A persistent issue about images is the degree to which additional direct work is done on the images by visual modality-specific operations to yield further information.” This depends on the kind of problem. Sometimes it is the image that is used for problem solving (involving visual operations) and sometimes other processing (such as retrieval form factual knowledge) are used and imparted to the image. [52-53]
Proposals for representations of images and diagrams based on perceptual primitives and on a distinction between spatial and visual representations. [53]
Diagrams and images are commonly used in reasoning and problem solving in a vast array of domains ranging from architecture, through economic reasoning, to mathematical proofs.
Diagrams can be used as both mental images and external images. The processes used on them are similar in nature.
“Diagrams preserve or directly represent locality information. A number of visual predicates are efficiently computed by the visual architecture from this information.” In spatial problems with diagrams an iterative strategy is used in which additional constructs and symbolic annotations are made such that new information emerges on each iteration. This process uses a tight cooperation between modalities such that each extracts the information it is most suited to extract and sets up additional information that is most suited to be worked on by the next modality. [54]
In non-visual problems diagrams can be used if a mapping of the properties of the problem to visual properties easily recognized by the visual system exists. (Examples: relations such as larger/smaller and directions are easily recognizable, while the difference in area of complex shapes is not)
We employ a large number of visual analogs may of which are taught to us culturally (such as the mapping between lines and time).
Diagrams can also help organize cognitive activity such as selecting methods for solving a problem. (Novak, Koedinger, Simon and so-workers).
The strength of diagrams lies in the fast and direct inferences (using the natural abilities of the visual system.)
The weakness of diagrams is their inability to represent universal quantification without special techniques.
Disjunctive reasoning (especially with many disjunctions) is hard in purely spatial mode. [54-55]
These problems are why most complex problems involve both visual and non-visual reasoning. [55]
The use of sketches in visual tasks (Goel and Faltings) allows for generalization. This allows for representations of ambiguous properties of constructs/problems. It also aids creativity as the generality is a variable that allows a sketch to represent a while family of precise pictures.
Much of out qualitative knowledge of physics is stored in the form of abstract perceptual chunks. (Narayanan and Chandraskaran) This contrasts with the symbolic/axiomatic approach. Central questions were: “Representation of object-configuration diagrams and predictive knowledge indexed by shapes, as well as visual events”
“Using visual representation for analogical simulation (..) was a general theme.”
Inference rules can be organized partly as diagrams with symbolic annotations.
There was agreement about the need to add diagrammatic representations into AI models and integrate them with the use of other representations.[55-56]
An electronic mailing list was set up to facilitate further discussion.[55]
An informal survey revealed general satisfaction, surprise at the diversity of work presented, disappointment at lack of specific applications and connectionist models, and a demand for more discussion time.[55-56]
Differences in basic assumptions between researchers in various fields were brought to attention. [55]
No consensus emerged as to the nature and role of diagrammatic reasoning. [56]