Vithanage Erandi Kawshalya Madhushani Jade Times Staff
V.E.K. Madhushani is a Jadetimes news reporter covering Innovation.
The Overlap of Senses: A Human and AI Phenomenon
What does pink taste like? How does a Sauvignon Blanc sound? These questions might seem absurd, but the human brain naturally merges sensory experiences, often blurring the lines between sight, taste, and sound. This phenomenon, known as "cross-modal correspondences," is a subconscious process that shapes how we perceive and interact with the world.
Interestingly, artificial intelligence (AI) systems exhibit similar patterns, as shown by recent research. The AI’s sensory "merging" isn't a sign of independent creativity but rather a reflection of the data it was trained on—essentially mirroring human tendencies. Researchers hope to leverage this AI ability to explore untapped connections in human sensory perception.
How Humans "Taste" Colors and Shapes
The brain's integration of senses is an ongoing process, whether we're conscious of it or not. Colors, shapes, and sounds all influence how we experience taste.
For example:
- Colors and Taste: Pink and red hues are often associated with sweetness, while green suggests sourness, and brown or black evoke bitterness.
- Shapes and Taste: Round shapes are perceived as sweet, while spiky forms are linked to sourness or bitterness.
Research shows these sensory connections are consistent across cultures. A study led by Xiaoang Wang at Tsinghua University demonstrated similar associations in Chinese, Indian, and Malaysian participants. This suggests these tendencies may be rooted in shared human experiences, such as recognizing that fruits become sweeter as they ripen and turn red.
Other factors, like emotions and safety instincts, may also play a role. For instance, round shapes evoke comfort and sweetness because they feel non-threatening, while sharp edges align with bitterness, reflecting the brain’s association with harm or danger.
AI and Cross-Modal Perception
Inspired by the brain's sensory integration, researchers recently tested whether generative AI could replicate human-like associations. A team led by Carlos Velasco and Kosuke Motoki asked AI-powered tools like ChatGPT to respond to prompts commonly used in psychological studies.
For example:
- "To what extent do you associate round shapes with sweet, sour, salty, bitter, and umami tastes?"
- "Which color best represents sweetness?"
AI tools reflected patterns similar to human responses. ChatGPT-4, in particular, provided more reliable associations compared to earlier versions, likely due to its larger training dataset. For example, the AI linked pink to sweetness, green to sourness, white to saltiness, and black to bitterness—aligning with experimental evidence.
The Practical Implications of Sensory AI
This sensory overlap in AI opens up fascinating possibilities:
- Marketing and Design: AI could help companies design product packaging or advertisements that align with consumers’ subconscious sensory expectations. For instance, pink packaging could make a product seem sweeter, while rounded shapes might enhance perceptions of indulgence.
- Scientific Exploration: Generative AI could propose novel hypotheses about sensory associations that researchers can test on humans, broadening our understanding of perception.
Charles Spence, a pioneer in cross-modal research, believes that AI might reveal previously unknown sensory connections. However, he emphasizes that AI should serve as inspiration rather than a definitive solution, noting the nuances of human creativity.
AI’s Shortcomings and Surprising Insights
Despite its potential, AI has limitations. It often "hallucinates" (fabricates responses) and lacks the intuitive quirks of human perception. For example, when Google's Gemini AI was asked about the sweetest color, it cited research directly from scientific literature, suggesting it might have "read" the answer rather than generating insights independently.
Still, the uncanny ability of AI to reflect sensory associations highlights just how ingrained these patterns are in human cognition.
Practical Examples: Music and Mulled Wine
The blending of senses isn’t limited to visual and gustatory experiences—it extends to sound. Studies have shown that certain pieces of music can enhance the flavors of wine. High-tempo and high-pitch compositions like Debussy’s Jardin Sous la Pluie complement citrusy whites, while slower, deeper tones like Rachmaninoff’s Vocalise pair well with fuller-bodied reds.
Curious about festive pairings, I asked ChatGPT for music suggestions to accompany mulled wine and mince pies. The AI recommended Carol of the Bells for its “layered melodies,” likening them to the spices in mulled wine. Other options included jazz renditions of Have Yourself a Merry Little Christmas or the mellow sophistication of Christmas Time is Here by the Vince Guaraldi Trio.
While AI’s recommendations align with traditional associations, human input remains essential for adding the nuanced flair that AI lacks.
The Future of Sensory AI
AI’s ability to "taste" colors and shapes offers a window into the complexities of human perception. With further advancements, these tools could deepen our understanding of the brain, assist in product innovation, and enhance artistic creativity.
For now, however, AI serves as a reminder of how interconnected our senses are—and how much we still have to learn about the way we experience the world.
