Do you see subtitles when someone is speaking? You aren’t alone

Subtitles are increasingly common in our video-dominated media landscape. Now imagine seeing captions whenever someone talks to you — or even an “lol” float by when you hear laughter. This is reality for people with ticker tape synesthesia (TTS). A new study, published in the journal Cortex, investigates this little understood form of synesthesia.

Subtitled speech

Synesthesia is a perceptual phenomenon that results in a crossing of sensory pathways. More common forms include grapheme-color synesthesia (in which synesthetes perceive letters or numbers as colored) and sound-to-color synesthesia (in which sounds are accompanied by experiences of colors). 

Synesthesia has been observed for centuries. John Locke first wrote about a blind man who “saw” scarlet when hearing a trumpet in 1690. In 1883, English polymath Francis Galton identified a rarer type when writing that some people mentally picture every spoken word as if read from a “long imaginary strip of paper.” 

Recently, researchers from the Paris Brain Institute invited 26 tickertape synesthetes to partake in a battery of audio tests. They were interested in TTS due to its uncommon attributes. As they wrote:

“TTS stands out of the list of known synesthesias because, in synesthesia parlance, both the inducer (speech) and the concurrent (orthographic image) are language representations. We therefore propose that the potential interest of TTS goes far beyond the anecdote, as it may provide a unique window in the mechanisms of written language processing and acquisition.”

Investigating tickertape synesthesia

The 26 tickertape synesthetes were presented with a series of auditory stimuli. They were asked to write down any subtitles they saw while listening to the sounds, which included non-speech stimuli (like animal sounds and yawning), words whose spelling don’t match the sound they produce (“yacht”), homophones (“leak” and “leek”), and pseudowords, meaningless words comprised of well-formed phonological strings (such as the French chadourne). 

The researchers found that every tickertape synesthete experienced subtitled speech while watching a speaker, and most saw subtitles when hearing new words. From there, the phenomenon affects synesthetes in a variety of manners — much like other forms of synesthesia. For example, sound-to-color synesthetes might all see colors when hearing musical notes, but the colors they see will depend on personal experiences. The researchers believe a similar phenomenon might be occurring with tickertape synesthetes.

The researchers did some more digging and uncovered possible insights into the cause and nature of TTS. For instance, there appears to be a genetic component, as nine of the participants reported a family history of TTS, while seven had family members who experienced other types of synesthesia. Perhaps unsurprisingly, 18 of the tickertape synesthetes also experienced other types of synesthesia, most notably time-space (12), number-space (9), and sound-to-color (7).

For 19 of the participants, TTS onset occurred after they first learned how to read, suggesting a potential link to reading comprehension. Most of the participants also did not report any disabilities due to their condition; in fact, a few hadn’t even heard of TTS until they applied to be in the study.The volunteers reported mixed results when asked if TTS was a net positive or net negative in their life. As the authors wrote, “The main subjective advantage was the support for spelling words correctly, and the main disadvantage was difficulty focusing their attention toward a single speaker in crowded places.”

A unique window into language

The development of language and orthography (written language conventions, such as spelling and hyphenation) has fascinated scholars for centuries. Synesthesia provides a unique window into how we process and communicate information and could help researchers better understand the developmental phases of language comprehension. 

Based on this study, the researchers believe that TTS makes writing easier and orthography better for those with the condition. They speculate that tickertape synesthetes perform better on verbal and memory tasks, and might even have a stronger expressive and receptive vocabulary. On the downside, tickertape synesthetes report difficulty focusing their attention, especially when engaged in a group conversation or while reading in public. Every superpower has its disadvantages.