The results from subjective questionnaires certainly need to be handled with care, especially as the very fact of raising the question can bias participants in trying to guess what the experimenter expects. A minimal conclusion is that the experience of using a sensory substitution device remains endowed with a characteristic feeling and that the experience can feel relevantly different from the experiences that one usually has in the modality stimulated with the device touch or audition , while making it quite often comparable but not similar to previous experiences in the visual modality see also Kupers et al.
Besides these subjective changes, more robust data on the use of sensory substitution devices come from the documentation of neurological changes. The most investigated change concerns the increased activation in the visual cortex after practice with visual-to-auditory sensory substitution devices De Volder et al. The neural plasticity in sensory substitution has been here seen as being of wider interest to the study of neural plasticity following sensory deprivation or impairment e.
Some connections between the subjective and neurological changes can be tempting to make, and start to emerge, for instance, with the hypothesis that long-term users of the vOICe could both be subject to neurological reorganization in the occipital cortex e. However, such claims should be taken with caution, given the presence of conflicting evidence obtained with visual-to-tactile devices.
In particular, Kupers et al.
Before training, no subjective tactile sensations were reported. After training, some of the blind participants reported tactile sensations that were referred to the tongue three out of eight early blind and one out of five late-blind participants. The author concluded that stimulation of the visual areas induces tactile but not visual sensations in trained blind users of the Tongue Display Unit.
In other words, the recruitment of V1 does not necessarily mean that the associated subjective experience is visual. Whatever the answers to this problem, these debates still revolves around comparing using sensory substitution device and perceiving in a canonical sensory modality. Changes beyond primary sensory areas are now being documented. Amedi et al. This does not occur when they are merely associating soundscapes to objects or recognizing objects by their typical sounds.
The LOtv, which is a locus of multisensory convergence, is usually activated both by the visual and the tactile exploration of objects. Although there are good reasons to see this as a perceptual reorganization, this study suggests that some more complex reorganizations than strictly sensory ones are taking place, having to do with later or supra-modal in this case, shape recognition; rather than necessarily with primary sensory processing.
Mini Review ARTICLE
Moreover, as reviewed in Bubic et al. Noticeably, it is not yet clear that activation in the occipital cortex reflects genuine bottom-up activation for tactile or auditory stimulation, consistent with what occurs in sensory perception, rather than top-down visual imagery mechanisms at least in late blind and sighted individuals, see Poirier et al.
The perceptual assumption dominates the interpretation of the results and orientates most of the research programs. Technical improvements and training are made with this perceptual assumption in mind, leaving the limited success of sensory substitution devices among blind individuals unexplained, or a matter of prophetic improvement.
Many scientists postulate that the limited results obtained with sensory substitution devices are only transient and that the gap with sensory perception can be bridged through further technological development or training. As we have shown, the perceptual assumption is largely biased and leads one to overlook some important features of the use and integration of sensory substitution device.
It has led to premature, and irresolvable debates regarding the analogy with synesthetic rewiring of the senses Proulx, ; Ward and Meijer, , or with visual perception e. A better model is needed, that can take all of the evidence into account in a more comprehensive and potentially fruitful way.
- Sensation and Perception.
- Back in Action;
- Calibration: A Technician’s Guide.
- Policy Issues Affecting Lesbian, Gay, Bisexual, and Transgender Families.
- IN ADDITION TO READING ONLINE, THIS TITLE IS AVAILABLE IN THESE FORMATS:.
The previous review opens an obvious challenge: how can we make sense of the positive evidence collected within the perceptual assumption, while acknowledging the limits and negative evidence that has just been listed? Notice here that a failure to find an alternative model would, indirectly, validate the perceptual assumption as being the best available model we have for thinking about sensory substitution. We believe, however, that an alternative can be found, which lifts the apparent contradiction between the canonical and less canonical perceptual aspects of sensory substitution.
This alternative is to be found in an analogy with other acquired skills and forms of automatic recognition — namely reading skills. By analogy, we mean more than the similarity mentioned here or there that converting sounds into visual or tactile signs acts as a precursor to the more recent devices converting images into sounds or tactile stimuli Bach-y-Rita and Kercel, , p.
The deeper resemblance between the acquisition of reading skills and the integration of sensory substitution devices comes from the fact that, in both cases, the progressive automatization of new identification skills consists in building a second route, which presupposes the existence of a first sensory route and parallels it.
Thinking about sensory substitution devices by analogy with reading therefore goes away from the perceptual assumption, where the devices resemble a canonical sensory modality, but also from simple claims about the multisensory aspects of these devices. Reading is classically defined as learning to access words through vision instead of audition; or eventually, through touch instead of audition in the case of Braille. No new receptor is needed to access them. At the same time, strictly speaking, reading brings about something new, which is the possibility to access through vision some objects words, sentences, and from there meanings that were only available through audition before.
This does not mean that reading adds access to auditory objects — as the shapes on the page do not have properties detectable by audition.
Visual Perception | Simply Psychology
Reading does not constitute an independent visual road substituting for the auditory perception of spoken language. More importantly, it is not an autonomous or dedicated perceptual system, and it is well explained as the development of a second route , grafted onto the auditory speech route, as popularized by the dual-route models of reading e. Writing systems have been designed to preserve the phonemic structures that are relevant to access semantic information. The acquisition of reading itself relies on existing phonemic skills, not just on auditory perception, and consists in mapping what one hears onto what one sees through the mediation of what one knows the later means.
It is only as a result of mapping the known written signs to known spoken words and phonemes that readers can progressively entertain auditory representation on the basis of visual words, and this even for unknown or novel items. Reading, then, is far from merely being a straightforward sensory remapping. Even more specifically, the acquisition of reading is accompanied by a change in subjective experience: written words no longer appear as colored shapes but as meaningful graphemes and words, which get easily and naturally associated to sounds i.
No such documented change in experience, to our knowledge, has been observed as a consequence of the progressive establishment of visuo-tactile crossmodal transfer see Held et al. This said, both reading and other crossmodal transfers are internalized and become automatic, in such a way that they no longer require attention or effort from the trained reader. Finally, learning to read induces crucial neurological changes noticeably in bilateral dorsal occipital areas associated with higher-level visual processing, in superior temporal areas associated with phonological processing, and in the angular gyri as well as in posterior middle temporal regions associated with semantic processing, see Turkeltaub et al.
All these features, we argue, offer an analogy robust enough to think about the results obtained through sensory substitution devices within a reading framework. The relevance of the analogy with reading is more likely to be found with sensory substitution devices that do not use an analogical format e. Going one step further, we want to argue that, by analogy with dual-route models of reading, learning to use a sensory substitution device is no longer to be thought of as being merely a matter of perceptual learning or adaptation, but as the building of a parallel access to cognitive and spatial representations that get grafted onto some pre-existing perceptual-cognitive route e.
This analogy encompasses the existing evidence and allows further generalization or predictions which can promisingly be put to test.
Further analogies between reading and integration of visual-to-auditory sensory substitution devices. Full arrows indicate new elements brought about by training and new devices or artifacts, dotted arrows indicate elements that pre-existed. First and foremost, the dual-route model of integrating sensory substitution devices is more illuminating with respect to the existing results on training.
Instead of trying to decide between the cognitive or active aspects of the integration of sensory substitution devices, or to adjudicate the exact importance of an explicit teaching of rules Auvray et al.
- A Nun on the Bus: How All of Us Can Create Hope, Change, and Community.
- 3D Computer Vision: Efficient Methods and Applications.
- Border Crossing: A Spiritual Journey.
As in reading, a combination of practical active training and explicit teaching of codes is the most common method in the area of sensory substitution, and the one for which most of the results have been collected. One objection here might come from the fact that explicit teaching of the coding rules is not necessary for users to start showing improved performance with a sensory substitution device. This can nonetheless be accommodated within the analogy with reading.
Another possibility, again opened by the comparison with reading, is that the code can be at least to a certain extent intuitively figured out, as it happens with young children learning to read before any formal training. A more specific prediction, here, is that learning without explicit teaching will be more frequent or easy with visual-to-tactile devices, at least in sighted and late-blind individuals, as they rely on natural crossmodal equivalences between tactile and visual shapes which do not need to be independently taught Spence and Deroy, By contrast, visual-to-auditory devices benefit from less immediate transfers.
This, however, does not mean that their integration cannot be helped by pre-existing audio-visual correspondences, for instance, between pitch and size, pitch and brightness, or sounds and shapes. Interestingly, these crossmodal correspondences have an influence on a variety of tasks, including speeded detection tasks and word learning. In a forced-choice task, two and a half year-old children were asked by the experimenter to match two novel words to two visual objects. Many other studies have subsequently demonstrated similar effects across a variety of languages see Imai et al.
People also identify novel objects more rapidly when crossmodal label-object mappings follow these crossmodal correspondences than when they do not. Within the domain of visual-to-auditory devices, there is a likely correlation between the intuitive aspect of the code and the amount of explicit training needed to achieve a reasonable level of performance.
Cutting these to a single dimension, the prediction is that, in the absence of explicit teaching, the integration of devices relying on a single and robust crossmodal correspondence for instance, only between high-pitched sounds and brightness will be easier than integrating a device that does not rely on such intuitive correspondences.
Predictions for the amount of explicit training for devices using multiple dimensions like the vOICe are harder to make even if they use intuitive correspondences, as we lack good models of how crossmodal correspondences act in combination see Spence, for a review; see also Proulx, , for an analogous claim. The benefits of the analogy with reading goes further than previous claims and observations that using sensory substitution devices is, in a sense that remain quite often under-specified, a form of sensory cross-talk or rewiring e.
Take, for instance, the recognition of shape, known as a case of crossmodal transfer see Streri, , for a review : According to most models, shape is processed first in a modal-specific way, that is as a tactile or a visual shape, before these two can be encoded in a common, amodal format, or translated into one another see Streri, for a review. As a result, visual and tactile perception of shapes come to give access to what is considered as a single kind of information, or property, which is neither specific to vision nor to touch.
At first, it can seem appropriate to think about visual-to-auditory sensory substitution in such a way, that is as building an additional access e. This model, however, might overstate the similarities between the domain of sensory substitution and the domain of crossmodal transfer in a way that one is not obliged to accept, and which hide some specificities of sensory substitution. First, the fact that one can access information about shape through audition does not necessarily mean that there is such a thing as the perception of auditory shape, like there is a perception of visual shapes or tactile shapes.
It seems like a stretch to say that sensory substitution devices can change the proper objects of audition, turning the perception of sounds into the perception of shapes in space. The only way to constrain the inference is to learn an arbitrary translation from one to the other. This is then more similar to the case of reading where variations in the shapes of words or letters do not directly lead to differences in sound. What happens though is that variations in the shapes of words or letters can be translated into variations in sounds, and from there to meaning. The arbitrariness of the translation is sufficient to make the inference from seeing letters to auditory properties of words and meanings special explaining for instance why a change of code requires explicit teaching of new rules of inference.
Thinking about sensory substitution as a parallel to reading skills helps introducing a distinction between two sub-levels of sensory cross-talk or conversion, paralleling the inter-related levels of separate letters, and whole word recognition.