MODULARITY
General readings:
Fodor. The Mind Doesn't Work That Way. MIT Press, 2000, Chap. 43.
------. The Modularity of Mind. MIT Press, 1983.
and one of the following sets of readings:
1. A Computational Prospective
Jacobs, R. A. "Computational Studies of the Development of Functionally Specialized Neural Modules." Trends Cogn Sci 3 (1999): 31-38.
PubMed abstract: Three hypotheses about the activity-dependent development of functionally specialized neural modules are discussed in this review. These hypotheses state that: (1) a combination of structure function correspondences plus the use of competition between neural modules leads to functional specializations; (2) parcellation is due to a combination of neural selectionism, the idea that learning results from a stabilization of some neural connections and the elimination of others, and a locality constraint, which states that connections between nearby neurons are more easily stabilized than those between distant neurons; and (3) a temporal and spatial modulation of plasticity can induce higher functional development in later-developing parts of the nervous system relative to earlier-developing parts. All three hypotheses have been implemented and evaluated in computational models. Limitations of current neuroscientific methodologies mean that computer simulation provides one of the only tools available for evaluating and refining our large-scale theories of the development of functionally specialized neural modules.
Gharamani, Z., and D. M. Wolpert. "Modular Decomposition in Visuomotor Learning." Nature 386: 392-395.
PubMed abstract: The principle of 'divide-and-conquer' the decomposition of a complex task into simpler subtasks each learned by a separate module, has been proposed as a computational strategy during learning. We explore the possibility that the human motor system uses such a modular decomposition strategy to learn the visuomotor map, the relationship between visual inputs and motor outputs. Using a virtual reality system, subjects were exposed to opposite prism-like visuomotor remappings-discrepancies between actual and visually perceived hand locations- for movements starting from two distinct locations. Despite this conflicting pairing between visual and motor space, subjects learned the two starting-point-dependent visuomotor mappings and the generalization of this learning to intermediate starting locations demonstrated an interpolation of the two learned maps. This interpolation was a weighted average of the two learned visuomotor mappings, with the weighting sigmoidally dependent on starting location, a prediction made by a computational model of modular learning known as the "mixture of experts". These results provide evidence that the brain may employ a modular decomposition strategy during learning.
2. An Evolutionary Perspective
Gallistel, C. R. "The Replacement of General-purpose Learning Models with Adaptively Specialized Learning Models." In The New Cognitive Neurosciences. Edited by M. Gazzaniga. MIT Press, 2000.
Duchaine B., L. Cosmides, and J. Tooby. "Evolutionary Psychology and the Brain." Curr Opin Neurobiol 11, 2 (2001): 225-30. Review.
PubMed abstract: The human brain is a set of computational machines, each of which was designed by natural selection to solve adaptive problems faced by our hunter-gatherer ancestors. These machines are adaptive specializations: systems equipped with design features that are organized such that they solve an ancestral problem reliably, economically and efficiently. The search for functionally specialized computational adaptations has now begun in earnest. A host of specialized systems have recently been found, including ones designed for sexual motivation, social inference, judgment under uncertainty and conditioning, as well as content-rich systems for visual recognition and knowledge acquisition.
3. Challenges from Psycholinguistics
MacDonald, M. C., N. H. Pearlmutter, and M. S. Seidenberg. "The Lexical Nature of Syntactic Ambiguity Resolution." Psychological Review 101 (1994): 676-703.
PubMed abstract: Ambiguity resolution is a central problem in language comprehension. Lexical and syntactic ambiguities are standardly assumed to involve different types of knowledge representations and be resolved by different mechanisms. An alternative account is provided in which both types of ambiguity derive from aspects of lexical representation and are resolved by the same processing mechanisms. Reinterpreting syntactic ambiguity resolution as a form of lexical ambiguity resolution obviates the need for special parsing principles to account for syntactic interpretation preferences, reconciles a number of apparently conflicting results concerning the roles of lexical and contextual information in sentence processing, explains differences among ambiguities in terms of ease of resolution, and provides a more unified account of language comprehension than was previously available.
Tanenhaus, M. K., M. J. Spivey-Knowlton, K. M. Eberhard, and J. C. Sedivy. "Integration of Visual and Linguistic Information in Spoken Language Comprehension." Science 268 (1995): 1632-1634.
PubMed abstract: Psycholinguists have commonly assumed that as a spoken linguistic message unfolds over time, it is initially structured by a syntactic processing module that is encapsulated from information provided by other perceptual and cognitive systems. To test the effects of relevant visual context on the rapid mental processes that accompany spoken language comprehension, eye movements were recorded with a head-mounted eye-tracking system while subjects followed instructions to manipulate real objects. Visual context influenced spoken word recognition and mediated syntactic processing, even during the earliest moments of language processing.
Trueswell, J. C., I. Sekerina, N. M. Hill, and M. L. Logrip. "The Kindergarten-path Effect: Studying On-line Sentence Processing in Young Children." Cognition 73 (1999): 89-134.
PubMed abstract: A great deal of psycholinguistic research has focused on the question of how adults interpret language in real time. This work has revealed a complex and interactive language processing system capable of rapidly coordinating linguistic properties of the message with information from the context or situation (e.g. Altmann & Steedman, 1988; Britt, 1994; Tanenhaus, Spivey-Knowlton, Eberhard & Sedivy, 1995; Trueswell & Tanenhaus, 1991). In the study of language acquisition, however, surprisingly little is known about how children process language in real time and whether they coordinate multiple sources of information during interpretation. The lack of child research is due in part to the fact that most existing techniques for studying language processing have relied upon the skill of reading, an ability that young children do not have or are only beginning to acquire. We present here results from a new method for studying children's moment-by-moment language processing abilities, in which a head-mounted eye-tracking system was used to monitor eye movements as participants responded to spoken instructions. The results revealed systematic differences in how children and adults process spoken language: Five Year Olds did not take into account relevant discourse/pragmatic principles when resolving temporary syntactic ambiguities, and showed little or no ability to revise initial parsing commitments. Adults showed sensitivity to these discourse constraints at the earliest possible stages of processing, and were capable of revising incorrect parsing commitments. Implications for current models of sentence processing are discussed.
4. Challenges from the Associationists
McClelland, J. L. "Integration of Information: Reflections on the Theme of Attention and Performance XVI." In Attention and Performance XVI. Edited by T. Inui and J. L. McClelland. Cambridge, MA: MIT Press, 1996.
Plaut, D. "Double Dissociations Without Modularity: Evidence from Connectionist Psychology." J. Clinical & Exper. Neuropsychology 17 (1995): 291-321.
PubMed abstract: Many theorists assume that the cognitive system is composed of a collection of encapsulated processing components or modules, each dedicated to performing a particular cognitive function. On this view, selective impairments of cognitive tasks following brain damage, as evidenced by double dissociations, are naturally interpreted in terms of the loss of particular processing components. By contrast, the current investigation examines in detail a double dissociation between concrete and abstract work reading after damage to a connectionist network that pronounces words via meaning and yet has no separable components (Plaut & Shallice, 1993). The functional specialization in the network that gives rise to the double dissociation is not transparently related to the network's structure, as modular theories assume. Furthermore, a consideration of the distribution of effects across quantitatively equivalent individual lesions in the network raises specific concerns about the interpretation of single-case studies. The findings underscore the necessity of relating neuropsychological data to cognitive theories in the context of specific computational assumptions about how the cognitive system operates normally and after damage.
5. Additive Factors Approaches to Functional Dissociations
Pinel, P., D. Riviere, D. Le Bihan, and S. Dehaene. "Modulation of Parietal Activation by Semantic Distance in a Number Comparison Task." Neuroimage. 2001 Nov; 14 (5): 1013-26.
PubMed abstract: The time to compare two numbers shows additive effects of number notation and of semantic distance, suggesting that the comparison task can be decomposed into distinct stages of identification and semantic processing. Using event-related fMRI and high-density ERPs, we isolated cerebral areas where activation was influenced by input notation (verbal or Arabic notation). The bilateral extrastriate cortices and a left precentral region were more activated during verbal than during Arabic stimulation, while the right fusiform gyrus and a set of bilateral inferoparietal and frontal regions were more activated during Arabic than during verbal stimulation. We also identified areas that were influenced solely by the semantic content of the stimuli (numerical distance between numbers to be compared) independent of the input notation. Activation tightly correlated with numerical distance was observed mainly in a group of parietal areas distributed bilaterally along the intraparietal sulci and in the precuneus, as well as in the left middle temporal gyrus and posterior cingulate. Our results support the assumption of a central semantic representation of numerical quantity that relies on a common parietal network shared among notations.
Sternberg, S. "Separate Modifiability, Mental Modules, and the Use of Pure and Composite Measures to Reveal Them." Acta Psychol 106 (2001): 147-246.
PubMed abstract: How can we divide a complex mental process into meaningful parts? In this paper, I explore an approach in which processes are divided into parts that are modular in the sense of being separately modifiable. Evidence for separate modifiability is provided by an instance of selective influence: two factors F and G (usually experimental manipulations) such that part A is influenced by F but invariant with respect to G, while part B is influenced by G but invariant with respect to F. Such evidence also indicates that the modules are functionally distinct. If we have pure measures MA and MB, each of which reflects only one of the parts, we need to show that MA is influenced by F but not G, while MB is influenced by G but not F. If we have only a composite measure MAB of the entire process, we usually also need to confirm a combination rule for how the parts contribute to MAB. I present a taxonomy of separate-modifiability methods, discuss their inferential logic, and describe several examples in each category. The three categories involve measures that are derived pure (based on different transformations of the same data; example: separation of sensory and decision processes by signal detection theory), direct pure (based on different data; example: selective effects of adaptation on spatial-frequency thresholds), and composite (examples: the multiplicative-factor method for the analysis of response rate; the additive-factor method for the analysis of reaction time). Six of the examples concern behavioral measures and functional processes, while four concern brain measures and neural processes. They have been chosen for their interest and importance; their diversity of measures, species, and combination rules; their illustration of different ways of thinking about data; the questions they suggest about possibilities and limitations of the separate-modifiability approach; and the case they make for the fruitfulness of searching for mental modules
INNATENESS AND PLASTICITY IN VISION
General readings:
Crair, M. "Neuronal Activity During Development: Permissive or Instructive?" Current Opinion in Neurobiology 9, 1 (1999): 88-93.
PubMed abstract: Experimental studies over the past year have shown that neural activity has a range of effects on the development of neural pathways. Although activity appears unimportant for establishing many aspects of the gross morphology and topology of the brain, there are many cases where the presence of neural activity is essential for the formation of a mature system of neural connections; in some instances, the pattern of neural activity actually orchestrates the final arrangement of neural connections.
Sur, M., and C. A. Leamey. "Development and Plasticity of Cortical Areas and Networks." Nat. Rev. Neuroscience 4 (2001): 251-262.
PubMed abstract: The development of cortical layers, areas and networks is mediated by a combination of factors that are present in the cortex and are influenced by thalamic input. Electrical activity of thalamocortical afferents has a progressive role in shaping cortex. For early thalamic innervation and patterning, the presence of activity might be sufficient; for features that develop later, such as intracortical networks that mediate emergent responses of cortex, the spatiotemporal pattern of activity often has an instructive role. Experiments that route projections from the retina to the auditory pathway alter the pattern of activity in auditory thalamocortical afferents at a very early stage and reveal the progressive influence of activity on cortical development. Thus, cortical features such as layers and thalamocortical innervation are unaffected, whereas features that develop later, such as intracortical connections, are affected significantly. Surprisingly, the behavioural role of 'rewired' cortex is also influenced profoundly, indicating the importance of patterned activity for this key aspect of cortical function.
Maurer, D., and T. Lewis. Visual Acuity: The Role of Visual Input in Inducing Postnatal Change. (In press).
and one of the following sets of readings:
1. Arealization and its Development
Felleman, D. J., and D. C. Van Essen. "Distributed Hierarchical Processing in the Primate Cerebral Cortex." Cereb Cortex 1 (1991): 1-47. Review.
PubMed abstract: In recent years, many new cortical areas have been identified in the macaque monkey. The number of identified connections between areas has increased even more dramatically. We report here on (1) a summary of the layout of cortical areas associated with vision and with other modalities, (2) a computerized database for storing and representing large amounts of information on connectivity patterns, and (3) the application of these data to the analysis of hierarchical organization of the cerebral cortex. Our analysis concentrates on the visual system, which includes 25 neocortical areas that are predominantly or exclusively visual in function, plus an additional 7 areas that we regard as visual-association areas on the basis of their extensive visual inputs. A total of 305 connections among these 32 visual and visual-association areas have been reported. This represents 31% of the possible number of pathways if each area were connected with all others. The actual degree of connectivity is likely to be closer to 40%. The great majority of pathways involve reciprocal connections between areas. There are also extensive connections with cortical areas outside the visual system proper, including the somatosensory cortex, as well as neocortical, transitional, and archicortical regions in the temporal and frontal lobes. In the somatosensory/motor system, there are 62 identified pathways linking 13 cortical areas, suggesting an overall connectivity of about 40%. Based on the laminar patterns of connections between areas, we propose a hierarchy of visual areas and of somatosensory/motor areas that is more comprehensive than those suggested in other recent studies. The current version of the visual hierarchy includes 10 levels of cortical processing. Altogether, it contains 14 levels if one includes the retina and lateral geniculate nucleus at the bottom as well as the entorhinal cortex and hippocampus at the top. Within this hierarchy, there are multiple, intertwined processing streams, which, at a low level, are related to the compartmental organization of areas V1 and V2 and, at a high level, are related to the distinction between processing centers in the temporal and parietal lobes. However, there are some pathways and relationships (about 10% of the total) whose descriptions do not fit cleanly into this hierarchical scheme for one reason or another. In most instances, though, it is unclear whether these represent genuine exceptions to a strict hierarchy rather than inaccuracies or uncertainities in the reported assignment.
Schiller, Peter. "On the Specificity of Neurons and Visual Areas." Behav Brain Res 76 (1996): 21-35.
PubMed abstract: The dominant view during the past 40 years has been that the visual system analyzes the visual scene by breaking it down into basic attributes such as color, form, motion, depth and texture. Individual dedicated neurons and specific visual areas were believed to be devoted to the analysis of each of these attributes. Current research has challenged these views by emphasizing that neurons, especially in the cortex, have multifunctional properties and therefore serve as general-purpose analyzers rather than feature detectors. Consequently, it appears that most extrastriate visual areas, rather than each being devoted to the analysis of a specific basic visual attribute, perform several different tasks and thereby engage in more advanced and complex analyses than had been realized.
2. Development of Ocular Dominance
Crair, M. C., D. C. Gillespie, and M. P. Stryker. "The Role of Visual Experience in the Development of Columns in Cat Visual Cortex." Science 179 (1998): 566-570.
PubMed abstract: The role of experience in the development of the cerebral cortex has long been controversial. Patterned visual experience in the cat begins when the eyes open about a week after birth. Cortical maps for orientation and ocular dominance in the primary visual cortex of cats were found to be present by 2 weeks. Early pattern vision appeared unimportant because these cortical maps developed identically until nearly 3 weeks of age, whether or not the eyes were open. The naive maps were powerfully dominated by the contralateral eye, and experience was needed for responses to the other eye to become strong, a process unlikely to be strictly Hebbian. With continued visual deprivation, responses to both eyes deteriorated, with a time course parallel to the well-known critical period of cortical plasticity. The basic structure of cortical maps is therefore innate, but experience is essential for specific features of these maps, as well as for maintaining the responsiveness and selectivity of cortical neurons.
Crowley, and Katz. "Early Development of Ocular Dominance Columns." Science 190 (2000): 1271-1273.
PubMed abstract: The segregation of lateral geniculate nucleus (LGN) axons into ocular dominance columns is believed to involve a prolonged, activity-dependent sorting process. However, visualization of early postnatal ferret LGN axons by direct LGN tracer injections revealed segregated ocular dominance columns <7 days after innervation of layer 4. These early columns were unaffected by experimentally induced imbalances in retinal activity, implying that different mechanisms govern initial column formation and their modification during the subsequent critical period. Instead of activity-dependent plasticity, we propose that ocular dominance column formation relies on the targeting of distinct axonal populations to defined locales in cortical layer 4.
3. Rewiring Sensory Cortex
Krubitzer, L., and K. J. Huffman. "Arealization of the Neocortex in Mammals: Genetic and Epigenetic Contributions to the Phenotype." Brain Behav Evol 55, 6 (Jun. 2000): 322-35.
PubMed abstract: The neocortex is composed of areas that are functionally, anatomically and histochemically distinct. In comparison to most other mammals, humans have an expanded neocortex, with a pronounced increase in the number of cortical areas. This expansion underlies many complex behaviors associated with human capabilities including perception, cognition, language and volitional motor responses. In the following review we consider data from comparative studies as well as from developmental studies to gain insight into the mechanisms involved in arealization, and discuss how these mechanisms may have been modified in different lineages over time to produce the remarkable degree of organizational variability observed in the neocortex of mammals. Because any phenotype is a result of the complex interactions between genotypic influences and environmental factors, we also consider environmental, or epigenetic, contributions to the organization of the neocortex.
Sur, M., A. Angelucci, and J. Sharma. "Rewiring Cortex: The Role of Patterned Activity in Development and Plasticity of Neocortical Circuits." J. Neurobiology 41 (1999): 33-43.
PubMed abstract: Visually driven activity is not required for the establishment of ocular dominance columns, orientation columns, and long-range horizontal connections in visual cortex, although spontaneous activity appears to be necessary. The role of activity may be instructive or simply permissive; evidence for an instructive role requires inquiry into the role of the pattern of activity in shaping cortical circuits. The few experiments that have probed the role of patterned activity include the effects of artificial strabismus, artificial stimulation of the optic nerve, and rewiring visual projections from the retina to the auditory thalamus and cortex. These experiments demonstrate that patterned activity is vital for the maintenance of thalamocortical, local intracortical, and long-range horizontal connections in cortex.
4. Rewiring in Humans
Cohen, L. G., P. Celnik, A. Pascual-Leone, B. Corwell, L. Falz, J. Dambrosia, M. Honda, N. Sadato, C. Gerloff, M. D. Catala, and M. Hallett. "Functional Relevance of Cross-modal Plasticity in Blind Humans." Nature 389 (1997): 180-183.
PubMed abstract: Functional imaging studies of people who were blind from an early age have revealed that their primary visual cortex can be activated by Braille reading and other tactile discrimination tasks. Other studies have also shown that visual cortical areas can be activated by somatosensory input in blind subjects but not those with sight. The significance of this cross-modal plasticity is unclear, however, as it is not known whether the visual cortex can process somatosensory information in a functionally relevant way. To address this issue, we used transcranial magnetic stimulation to disrupt the function of different cortical areas in people who were blind from an early age as they identified Braille or embossed Roman letters. Transient stimulation of the occipital (visual) cortex induced errors in both tasks and distorted the tactile perceptions of blind subjects. In contrast, occipital stimulation had no effect on tactile performance in normal-sighted subjects, whereas similar stimulation is known to disrupt their visual performance. We conclude that blindness from an early age can cause the visual cortex to be recruited to a role in somatosensory processing. We propose that this cross-modal plasticity may account in part for the superior tactile perceptual abilities of blind subjects.
Roeder, B. A., W. Teder-Salejarvi, A. Sterr, R. Roesler, S. A. Hillyard, and H. J. Neville. "Improved Auditory Spatial Tuning in Blind Humans." Nature 400 (1999): 162-166.
PubMed abstract: Despite reports of improved auditory discrimination capabilities in blind humans and visually deprived animals, there is no general agreement as to the nature or pervasiveness of such compensatory sensory enhancements. Neuroimaging studies have pointed out differences in cerebral organization between blind and sighted humans, but the relationship between these altered cortical activation patterns and auditory sensory acuity remains unclear. Here we compare behavioural and electrophysiological indices of spatial tuning within central and peripheral auditory space in congenitally blind and normally sighted but blindfolded adults to test the hypothesis (raised by earlier studies of the effects of auditory deprivation on visual processing) that the effects of visual deprivation might be more pronounced for processing peripheral sounds. We find that blind participants displayed localization abilities that were superior to those of sighted controls, but only when attending to sounds in peripheral auditory space. Electrophysiological recordings obtained at the same time revealed sharper tuning of early spatial attention mechanisms in the blind subjects. Differences in the scalp distribution of brain electrical activity between the two groups suggest a compensatory reorganization of brain areas in the blind that may contribute to the improved spatial resolution for peripheral sound sources.
Sadato, N., A. Pascual-Leone, J. Grafman, V. Ibaniez, M. P. Deiber, G. Dold, and M. Hallett. "Activation of the Primary Visual Cortex by Braille Reading in Blind Subjects." Nature 380 (1996): 526-528.
PubMed abstract: Primary visual cortex receives visual input from the eyes through the lateral geniculate nuclei, but is not known to receive input from other sensory modalities. Its level of activity, both at rest and during auditory or tactile tasks, is higher in blind subjects than in normal controls, suggesting that it can subserve nonvisual functions; however, a direct effect of non-visual tasks on activation has not been demonstrated. To determine whether the visual cortex receives input from the somatosensory system we used positron emission tomography (PET) to measure activation during tactile discrimination tasks in normal subjects and in Braille readers blinded in early life. Blind subjects showed activation of primary and secondary visual cortical areas during tactile tasks, whereas normal controls showed deactivation. A simple tactile stimulus that did not require discrimination produced no activation of visual areas in either group. Thus in blind subjects, cortical areas normally reserved for vision may be activated by other sensory modalities.
Sterr, A., M. M. Muller, T. Elbert, B. Rockstroh, C. Pantev, and E. Taub. "Changed Perceptions in Braille Readers." Nature 391 (1998): 134-135.
OBJECTS
General readings:
Kahneman, D., A. Treisman, and B. J. Gibbs. "The Reviewing of Object Files: Object-specific Integration of Information." Cognitive Psychology 24 (1992): 175-219.
PubMed abstract: A series of experiments explored a form of object-specific priming. In all experiments a preview field containing two or more letters is followed by a target letter that is to be named. The displays are designed to produce a perceptual interpretation of the target as a new state of an object that previously contained one of the primes. The link is produced in different experiments by a shared location, by a shared relative position in a moving pattern, or by successive appearance in the same moving frame. An object-specific advantage is consistently observed: naming is facilitated by a preview of the target, if (and in some cases only if) the two appearances are linked to the same object. The amount and the object specificity of the preview benefit are not affected by extending the preview duration to 1 s, or by extending the temporal gap between fields to 590 ms. The results are interpreted in terms of a reviewing process, which is triggered by the appearance of the target and retrieves just one of the previewed items. In the absence of an object link, the reviewing item is selected at random. We develop the concept of an object file as a temporary episodic representation, within which successive states of an object are linked and integrated.
Scholl, B. J. "Objects and Attention: The State of the Art." Cognition 80 (2001): 1-46.
PubMed abstract: What are the units of attention? In addition to standard models holding that attention can select spatial regions and visual features, recent work suggests that in some cases attention can directly select discrete objects. This paper reviews the state of the art with regard to such 'object-based' attention, and explores how objects of attention relate to locations, reference frames, perceptual groups, surfaces, parts, and features. Also discussed are the dynamic aspects of objecthood, including the question of how attended objects are individuated in time, and the possibility of attending to simple dynamic motions and events. The final sections of this review generalize these issues beyond vision science, to other modalities and fields such as auditory objects of attention and the infant's 'object concept'.
Spelke, E. S., P. Vishton, and C. Von Hofsten. "Object Perception, Object-directed Action, and Physical Knowledge in Infancy." In The Cognitive Neurosciences. Edited by M. Gazzaniga. Cambridge, MA: MIT Press, 1995, pp. 165-179.
and one of the following sets of readings:
1. Attentive Tracking in Adults
Culham, J., P. Cavanagh, and N. Kanwisher. "Attention Response Functions: Characterizing Brain Areas Using fMRI Activation During Parametric Variations of Attentional Load." Neuron. 2001 Nov 20; 32 (4): 737-45.
PubMed abstract: We derived attention response functions for different cortical areas by plotting neural activity (measured by fMRI) as a function of attentional load in a visual tracking task. In many parietal and frontal cortical areas, activation increased with load over the entire range of loads tested, suggesting that these areas are directly involved in attentional processes. However, in other areas (FEF and parietal area 7), strong activation was observed even at the lowest attentional load (compared to a passive baseline using identical stimuli), but little or no additional activation was seen with increasing load. These latter areas appear to play a different role, perhaps supporting task-relevant functions that do not vary with load, such as the suppression of eye movements.
Scholl, B. J., and Z. W. Pylyshyn. "Tracking Multiple Objects Through Occlusion: Clues to Visual Objecthood." Cognitive Psychology 38 (1999): 259-290.
PubMed abstract: In three experiments, subjects attempted to track multiple items as they moved independently and unpredictably about a display. Performance was not impaired when the items were briefly (but completely) occluded at various times during their motion, suggesting that occlusion is taken into account when computing enduring perceptual objecthood. Unimpaired performance required the presence of accretion and deletion cues along fixed contours at the occluding boundaries. Performance was impaired when items were present on the visual field at the same times and to the same degrees as in the occlusion conditions, but disappeared and reappeared in ways which did not implicate the presence of occluding surfaces (e.g., by imploding and exploding into and out of existence instead of accreting and deleting along a fixed contour). Unimpaired performance did not require visible occluders (i.e., Michotte's tunnel effect) or globally consistent occluder positions. We discuss implications of these results for theories of objecthood in visual attention. Copyright 1999 Academic Press.
Yantis, S. "Multi-element Visual Tracking: Attention and Perceptual Organization." Cognitive Psychology 24 (1992): 295-340.
PubMed abstract: Two types of theories have been advanced to account for how attention is allocated in performing goal-directed visual tasks. According to location-based theories, visual attention is allocated to spatial locations in the image; according to object-based theories, attention is allocated to perceptual objects. Evidence for the latter view comes from experiments demonstrating the importance of perceptual grouping in selective-attention tasks. This article provides further evidence concerning the importance of perceptual organization in attending to objects. In seven experiments, observers tracked multiple randomly moving visual elements under a variety of conditions. Ten elements moved continuously about the display for several seconds; one to five of them were designated as targets before movement initiation. At the end of movement, one element was highlighted, and subjects indicated whether or not it was a target. The ease with which the elements in the target set could be perceptually grouped was systematically manipulated. In Experiments 1-3, factors that influenced the initial formation of a perceptual group were manipulated; this affected performance, but only early in practice. In Experiments 4-7, factors that influenced the maintenance of a perceptual group during motion were manipulated; this affected performance throughout practice. The results suggest that observers spontaneously grouped the target elements and directed attention toward this coherent but nonrigid virtual object. This supports object-based theories of attention and demonstrates that perceptual grouping, which is usually conceived of as a purely stimulus-driven process, can also be governed by goal-directed mechanisms.
2. Object Perception in Infants: an Encapsulated Mechanism?
Johnson, S. P., and R. N. Aslin. "Perception of Object Unity in Young Infants: The Roles of Motion, Depth and Orientation." Cognitive Development 11 (1996): 161-180.
Johnson, S. P., J. G. Bremner, A. Slater, and U. Mason. "The Role of Good Form in Young Infants' Perception of Partly Occluded Objects." Journal of Experimental Child Psychology 76 (2000): 1-25.
PubMed abstract: Young infants have been reported to perceive the unity of a center-occluded object when the visible ends of the object undergo common motion, but not on the basis of stationary information (e.g., P. J. Kellman & E. S. Spelke, 1983). We investigated the possibility that 4-month-old infants will attend to and utilize the global configuration (i.e., the "good form") of a partly occluded, moving object to perceive its unity and coherence behind the occluder. In the first experiment, infants viewed a partly occluded circle or cross that translated laterally. Infants who habituated in the minimum number of trials ("fast habituators") showed a reliable posthabituation preference for a broken object over a complete object, indicating perception of unity in the habituation display. Slow habituators exhibited no posthabituation preference. In the second experiment, infants were presented with small ring and cross displays, and the infants looked longer at the broken object. There were no reliable differences in performance between fast and slow habituators. A control group demonstrated no reliable posthabituation preference. In three additional conditions, infants viewed either a partly occluded half ring or a display in which two rod parts were either relatable and nonaligned or nonrelatable. The results indicated that curvature per se provided information in support of completion, in addition to global configuration and motion. Implications for theories of infants' visual development are discussed. Copyright 2000 Academic Press.
Jusczyk, P. W., S. P. Johnson, E. S. Spelke, and L. J. Kennedy. "Synchronous Change and Perception of Object Unity: Evidence from Adults and Infants." Cognition 71 (1999): 257-288.
PubMed abstract: Adults and infants display a robust ability to perceive the unity of a center-occluded object when the visible ends of the object undergo common motion (e.g. Kellman, P.J., Spelke, E.S., 1983. Perception of partly occluded objects in infancy. Cognitive Psychology 15, 483-524). Ecologically oriented accounts of this ability focus on the primary of motion in the perception of segregated objects, but Gestalt theory suggests a broader possibility: observers may perceive object unity by detecting patterns of synchronous change, of which common motion is a special case. We investigated this possibility with observations of adults and 4-month-old infants. Participants viewed a center-occluded object whose visible surfaces were either misaligned or aligned, stationary or moving, and unchanging or synchronously changing in color or brightness in various temporal patterns (e.g. flashing). Both alignment and common motion contributed to adults' perception of object unity, but synchronous color changes did not. For infants, motion was an important determinant of object unity, but other synchronous changes and edge alignment were not. When a stationary object with aligned edges underwent synchronous changes in color or brightness, infants showed high levels of attention to the object, but their perception of its unity appeared to be indeterminate. An inherent preference for fast over slow flash rates, and a novelty preference elicited by a change in rate, both indicated that infants detected the synchronous changes, although they failed to use them as information for object unity. These findings favor ecologically oriented accounts of object perception in which surface motion plays a privileged role.
3. Object Tracking in Infants and Adults: A Domain-specific Mechanism?
Chiang, W. C., and K. Wynn. "Infants' Tracking of Objects and Collections." Cognition 77 (2000): 169-195.
PubMed abstract: Recent research suggests that infants' understanding of the physical world is more complex and adult-like than previously believed. One of the most impressive discoveries has been infants' ability to reason about medium-sized, material objects. They are able to individuate objects in a scene, and to enumerate and reason about them. This article reports a series of experiments investigating 8-month-old infants' ability to reason about collections of objects. Experiment 1 shows a sharp contrast between infants' understanding of single objects versus collections. While infants detected the discontinuous ('Magical') disappearance of a single object, they did not detect the Magical Disappearance of a non-cohesive pile of objects. Experiments 2-4 found that infants' difficulty remained even when the distinct identity of each object in the collection was emphasized, but could be overcome if infants (a) first saw the individual objects clearly separated from each other prior to their being placed together in a pile, or (b) had prior experience with the objects making up the collection. Our findings suggest that infants' expectations about object behavior are highly specific regarding the entities they are applied to. They do not automatically apply to any and all portions of matter within the visual field. Both the behavior of an entity, and infants' prior experience play roles in determining whether infants will treat that entity as an object.
Scholl, B. J., Z. W. Pylyshyn, and J. Feldman. "What is a Visual Object? Evidence from Target Merging in Multiple Object Tracking." Cognition 80 (2001): 159-77.
PubMed abstract: The notion that visual attention can operate over visual objects in addition to spatial locations has recently received much empirical support, but there has been relatively little empirical consideration of what can count as an 'object' in the first place. We have investigated this question in the context of the multiple object tracking paradigm, in which subjects must track a number of independently and unpredictably moving identical items in a field of identical distractors. What types of feature clusters can be tracked in this manner? In other words, what counts as an 'object' in this task? We investigated this question with a technique we call target merging: we alter tracking displays so that distinct target and distractor locations appear perceptually to be parts of the same object by merging pairs of items (one target with one distractor) in various ways - for example, by connecting item locations with a simple line segment, by drawing the convex hull of the two items, and so forth. The data show that target merging makes the tracking task far more difficult to varying degrees depending on exactly how the items are merged. The effect is perceptually salient, involving in some conditions a total destruction of subjects' capacity to track multiple items. These studies provide strong evidence for the object-based nature of tracking, confirming that in some contexts attention must be allocated to objects rather than arbitrary collections of features. In addition, the results begin to reveal the types of spatially organized scene components that can be independently attended as a function of properties such as connectedness, part structure, and other types of perceptual grouping.
Van de Walle, G., J. Rubenstein, and E. S. Spelke. "Infant Sensitivity to Shadow Motions." Cognitive Development 13 (1998): 387-419.
4. Infants' Use or Non-use of Kind Information to Parse Objects
Carey, S., and F. Xu. "Infants' Knowledge of Objects: Beyond Object Files and Object Tracking." Cognition 80 (2001): 179-213.
PubMed abstract: Two independent research communities have produced large bodies of data concerning object representations: the community concerned with the infant's object concept and the community concerned with adult object-based attention. We marshal evidence in support of the hypothesis that both communities have been studying the same natural kind. The discovery that the object representations of young infants are the same as the object files of mid-level visual cognition has implications for both fields.
Needham, A., and R. Baillargeon. "Infants' Use of Featural and Experiential Information in Segregating and Individuating Objects: a Reply to Xu, Carey & Welch (1999)." Cognition 74 (2000): 255-284.
Xu, F., S. Carey, and J. Welch. "Infants' Ability to Use Object Kind Information for Object Individuation." Cognition 70 (1999): 137-166.
PubMed abstract: The present studies investigate infants reliance on object kind information in solving the problem of object individuation. Two experiments explored whether adults, 10- and 12-month-old infants could use their knowledge of ducks and cars to individuate an ambiguous array consisting of a toy duck perched on a toy car into two objects. A third experiment investigated whether 10-month-old infants could use their knowledge of cups and shoes to individuate an array consisting of a cup perched on a shoe into two objects. Ten-month-old infants failed to use object kind information alone to resolve the ambiguity with both pairs of objects. In contrast, infants this age succeeded in using spatiotemporal information to segment the array into two objects, i.e. they succeeded if shown that the duck moved independently relative to the car, or the cup relative to the shoe. Twelve-month-old infants, as well as adults, succeeded at object individuation on the basis of object kind information alone. These findings shed light on the developmental course of object individuation and provide converging evidence for the Object-first Hypothesis [Xu, F., Carey, S., 1996; Xu, F., 1997b]. Early on, infants may represent only one concept that provides criteria for individuation, namely physical object; kind concepts such as duck, car, cup, and shoe may be acquired later in the first year of life.
5. Adults' Use or Non-use of Kind Information to Parse Objects
Marr, D. Vision. Freeman, 1982, pp. 270-271.
Peterson, M. A., B. de Gelder, S. Z. Rapcsak, P. C. Gerhardstein, and A. Bachoud-Levi. "Object Memory Effects on Figure Assignment: Conscious Object Recognition is Not Necessary or Sufficient." Vision Research 40 (2000): 1549-1567.
PubMed abstract: In three experiments we investigated whether conscious object recognition is necessary or sufficient for effects of object memories on figure assignment. In experiment 1, we examined a brain-damaged participant, AD, whose conscious object recognition is severely impaired. AD's responses about figure assignment do reveal effects from memories of object structure, indicating that conscious object recognition is not necessary for these effects, and identifying the figure-ground test employed here as a new implicit test of access to memories of object structure. In experiments 2 and 3, we tested a second brain-damaged participant, WG, for whom conscious object recognition was relatively spared. Nevertheless, effects from memories of object structure on figure assignment were not evident in WG's responses about figure assignment in experiment 2, indicating that conscious object recognition is not sufficient for effects of object memories on figure assignment. WG's performance sheds light on AD's performance, and has implications for the theoretical understanding of object memory effects on figure assignmentRiddoch, J., and G. W. Humphreys. "Visual Object Processing in Optic Aphasia: A Case of Semantic Access Agnosia." Cognitive Neuropsychology 4 (1987): 131-185.
PEOPLE I: FACES
General readings:
Moscovitch, M., G. Winocur, and M. Behrmann. "What is Special About Face Recognition? Nineteen Experiments on a Person with Visual Object Agnosia and Dyslexia but Normal Face Recognition." Journal of Cognitive Neuroscience 9 (1997): 555-604.
Tanaka, J. W., and M. J. Farah. "Parts and Wholes in Face Recognition." Q J Exp Psychol A 46 (1993): 225-45.
PubMed abstract: Are faces recognized using more holistic representations than other types of stimuli? Taking holistic representation to mean representation without an internal part structure, we interpret the available evidence on this issue and then design new empirical tests. Based on previous research, we reasoned that if a portion of an object corresponds to an explicitly represented part in a hierarchical visual representation, then when that portion is presented in isolation it will be identified relatively more easily than if it did not have the status of an explicitly represented part. The hypothesis that face recognition is holistic therefore predicts that a part of a face will be disproportionately more easily recognized in the whole face than as an isolated part, relative to recognition of the parts and wholes of other kinds of stimuli. This prediction was borne out in three experiments: subjects were more accurate at identifying the parts of faces, presented in the whole object, than they were at identifying the same part presented in isolation, even though both parts and wholes were tested in a forced-choice format and the whole faces differed only by one part. In contrast, three other types of stimuli--scrambled faces, inverted faces, and houses--did not show this advantage for part identification in whole object recognition.
and one of the following sets of readings:
1. On the Nature of Face Representations
Rhodes, G., S. Carey, G. Byatt, and F. Proffitt. "Coding Spatial Variations in Faces and Simple Shapes: a Test of Two Models." Vision Res 38 (1998): 2307-2321.
Young, A. W., D. Hellawell, and D. C. Hay. "Configurational Information in Face Perception." Perception 16, 6 (1987): 747-59.
PubMed abstract: A new facial composites technique is demonstrated, in which photographs of the top and bottom halves of different familiar faces fuse to form unfamiliar faces when aligned with each other. The perception of a novel configuration in such composite stimuli is sufficiently convincing to interfere with identification of the constituent parts (experiment 1), but this effect disappears when stimuli are inverted (experiment 2). Difficulty in identifying the parts of upright composites is found even for stimuli made from parts of unfamiliar faces that have only ever been encountered as face fragments (experiment 3). An equivalent effect is found for composites made from internal and external facial features of well-known people (experiment 4). These findings demonstrate the importance of configurational information in face perception, and that configurations are only properly perceived in upright faces.
2. Evidence for Special Neural Hardware for Faces
Allison, T., A. Puce, D. D. Spencer, and G. McCarthy. "Electrophysiological Studies of Human Face Perception. I: Potentials Generated in Occipitotemporal Cortex by Face and Non-face Stimuli." Cereb Cortex 9 (1999): 415-30.
PubMed abstract: This and the following two papers describe event-related potentials (ERPs) evoked by visual stimuli in 98 patients in whom electrodes were placed directly upon the cortical surface to monitor medically intractable seizures. Patients viewed pictures of faces, scrambled faces, letter-strings, number-strings, and animate and inanimate objects. This paper describes ERPs generated in striate and peristriate cortex, evoked by faces, and evoked by sinusoidal gratings, objects and letter-strings. Short-latency ERPs generated in striate and peristriate cortex were sensitive to elementary stimulus features such as luminance. Three types of face-specific ERPs were found: (i) a surface-negative potential with a peak latency of approximately 200 ms (N200) recorded from ventral occipitotemporal cortex, (ii) a lateral surface N200 recorded primarily from the middle temporal gyrus, and (iii) a late positive potential (P350) recorded from posterior ventral occipitotemporal, posterior lateral temporal and anterior ventral temporal cortex. Face-specific N200s were preceded by P150 and followed by P290 and N700 ERPs. N200 reflects initial face-specific processing, while P290, N700 and P350 reflect later face processing at or near N200 sites and in anterior ventral temporal cortex. Face-specific N200 amplitude was not significantly different in males and females, in the normal and abnormal hemisphere, or in the right and left hemisphere. However, cortical patches generating ventral face-specific N200s were larger in the right hemisphere. Other cortical patches in the same region of extrastriate cortex generated grating-sensitive N180s and object-specific or letter-string-specific N200s, suggesting that the human ventral object recognition system is segregated into functionally discrete regions.
Kanwisher, N., P. Downing, R. Epstein, and Z. Kourtzi. "Functional Neuroimaging of Visual Recognition." In Handbook of Functional Neuroimaging. Edited by Cabeza and Kingstone. MIT Press, pp. 117-125.
Sergent, J., and J. Signoret. "Varieties of Functional Deficits in Prosopagnosia." Cerebral Cortex 2 (1992): 375-388.
PubMed abstract: Prosopagnosia is a neurologically based deficit characterized by the inability to recognize faces of known individuals in the absence of severe intellectual, perceptual, and memory impairments. The nature of the underlying disturbance was investigated in three patients in an attempt to identify the structural and functional levels at which the processing of faces breaks down, the relation between prosopagnosia and associated deficits, and the specificity of the prosopagnosic disturbance. The breakdown of face processing resulted from unilateral damage in different cerebral structures of the right hemisphere in the three patients, and it involved different functional levels of face processing, but all three patients displayed perceptual impairments of unequal severity. In one patient (R.M.), the deficit encompassed all perceptual operations on faces, including matching identical views of the same faces, but it did not extend to all categories of objects characterized by a close similarity among their instances; the second patient (P.M.) exhibited a less severe perceptual impairment but was unable to derive the configurational properties from a facial representation and to extract its physiognomic invariants; the third patient (P.C.) had not lost the capacity to differentiate faces on the basis of their configurations but could not associate a facial representation with its pertinent memories. Associated deficits were present in each patient but differed depending on the anatomofunctional locus of the breakdown, although all patients were impaired at recognizing noncanonical views of objects that they readily recognized when shown from a conventional viewpoint. However, performance dissociation within patients and double dissociation between patients suggest that these associated deficits are not necessary concomitants of prosopagnosia.
3. The Role of Experience
Diamond, R., and S. Carey. "Why Faces Are and Are Not Special: An Effect of Expertise." Journal of Experimental Psychology: General 115 (1986): 107-117.
PubMed abstract: Recognition memory for faces is hampered much more by inverted presentation than is memory for any other material so far examined. The present study demonstrates that faces are not unique with regard to this vulnerability to inversion. The experiments also attempt to isolate the source of the inversion effect. In one experiment, use of stimuli (landscapes) in which spatial relations among elements are potentially important distinguishing features is shown not to guarantee a large inversion effect. Two additional experiments show that for dog experts sufficiently knowledgeable to individuate dogs of the same breed, memory for photographs of dogs of that breed is as disrupted by inversion as is face recognition. A final experiment indicates that the effect of orientation on memory for faces does not depend on inability to identify single features of these stimuli upside down. These experiments are consistent with the view that experts represent items in memory in terms of distinguishing features of a different kind than do novices. Speculations as to the type of feature used and neuropsychological and developmental implications of this accomplishment are offered.
Kanwisher, N. "Domain Specificity in Face Perception." Nat Neurosci 3 (2000): 759-63.