A colorful language

Even if we are not synesthete nor have perfect pitch, it is possible to learn (for a while, for a long time actually) to be synesthete: obvious example: the alphabet. If I see an A, I have trouble not thinking about the sound "A". 

It is even possible to prove that the same way we prove syntesthetes have trouble sometimes because of the interference of symbols, for instance by writing BLUE in red - it takes more time to read it. Now, come the interesting idea: why do we use only black and white and SHAPES to create this sound-to-shape synesthesia? Because it is obvious that producing different colors was more difficult than creating different shapes; but now, we could easily control, say, an RGB light by, say, waving a magic stick like the sony controller in the air and exploring the RGB or VHS space. 

Then, the question is: could we create a solid color-to-sound synesthesia, and more particularly, a colour to LETTER or syllabe synesthesia? that would mean we could talk and write with colours. We could perhaps read matrices of colours as a machine read a bar code… And what about the combination of colours? (instead of a serial sequence, a superposition); could we find perhaps some combinations "impossible" because they would not "sound" or produce intelligible words? or use luminance and hue, etc to augment the expressivity of individual letters made of shapes as usual? So many possibilities! The first is to practice.

Vowels:
Blue = O
RED = A
GREEN  = E
YELLOW = I
ORANGE = U

Consonants:
For me these are rather metallic colours or grayish - it's perhaps a synesthesia related to the FEELING of the texture (for instance, a textured grey make me think about "shhh", the "K" is BLACK, the white is probably a space. Black and White are 0 and 1… for obvious cultural BINARY reasons.


Here's an example (find the meaning yourself, this is a Rosetta stone - code in processing):


The points is, a real synesthete, like someone with perfect pitch, may find it difficult to hear and ARBITRARY harmony and symphony of colours; they already have their own. Now, this experiment is about making this "language" as simple to learn and "extrapolate" as possible.
What is for instance "brown"? About 2/3 red 1/6 yellow and 1/6 blue… meaning a sound in between a, i and o, probably a vowel that exist in Korean…
That it is possible to train the eye to understand these "proportion" is proved by the fact that people can become very good at distinguishing colours (a sort of "perfect colour" ability). Then, the world around them would "sound" in words.

ALSO INTERESTING:
  •  This is a really nice example of a MINIMAL display as explained in my research. Non obtrusive, using modalities at their maximum without having to process much. 
  • The meaning of the mixing of colors, AND/OR the illumination. For instance, in the morning, the same book may be more "sad" or "happy" than when read in the sunset, or at night with artificial light. These subtleties, some may say, will render this color language unusable... I don't agree with this from a very basic point of view: the language itself has more than fifty shades of grey! (bad pun). What I mean is that the interpretation of the meaning of words is "analogic", the meaning is different and diffuse among people. We are not talking about a formal language, but a HUMAN language. This is exactly why the language of colors can be at the forefront of poetics (and, for me, poetry is no other thing than the work of the avant-garde "evolvers" of language. Hundred of thousands of years ago, human being saying "this smooth stone is heavy" may have been poets. Maybe at the time they would say: "stone, heavy... ahhh".  See what I mean? Poetry is the forefront of language evolution. But it is now stuck, graphically speaking, to black and white.

TO DO:
  • Train myself to learn an arbitrary language of colours (I can use in the future the twiddler to write on my “color” glasses without the need of a display with pixels).
  • Make a sms send single or little matrices of colors, human readable bar code (say, 4x4) on the glasses or faces of people.
  • Processing software to translate keystrokes into a matrix of colors and visceversa
  • Transform automatically a webpage into a color painting, a book
  • Use SPEECH RECOGNITION to modulate the color of an RGB led (could be in the mouth, as Daito's project... then deaf people could "see" the sounds).
  • Same could apply for people who have diminished vision, but retain color vision.
  • Postcard with color matrices project. 

Comments

Post a Comment

Popular posts from this blog

12.9.2008 - Being an afterthought

From time to time, we are all (philosophical?) zombies! In fact, we are most of the time consciousless: most of what we do, we just do without really being aware of. Why then it looks like to ourselves as if we "are" there all the time? well, perhaps for the same reason that, at any time, it looks like to us that we are "seeing a perfectly detailed and total" image of the world with our eyes: because the potentiality of eliciting consciousness is there at any time: we can "switch" at any time a routine of "self awareness" so as to describe (to ourselves and to others) our current experience - but this does not mean that this routine is always on - nor that it need to be for us to "function" efficiently. We are not continuously scanning our actions, or at least not all the time with the same level of self-attention. Internal or external stimuli can switch this process on and off (cf. attentional self-less concept proposed in my boxedEg
Read more

Real or Virtual Matter - or How I Learned to Stop Worrying and Love the Matrix

Image
On the way to the Space Elevator, an improbable view pop ups through the hyperloop oval window. It is the fleeting mirage of a floating ocean - or a gargantuan aquarium - stealing an entire city block. For a fraction of a second I can hear it, smell it, taste it... that is, I feel the seashore with all my senses. It was late afternoon, and a kaleidoscopic (virtually out-of-this-world) sunset had transfixed me, so it takes me a moment to realize that such experience was anything but real. These were not whales, dolphins and fish - just holographic projections rendered in a carefully crafted simulation (kids these days probably don’t mind the distinction; I guess I am too old not to feel cheated).  Dancing voxels of light: not a single flying fish. A short multimedia choreography took my senses by surprise, prompting that primitive part of my brain always eager to believe. These were fish - these were somehow fish - claims an inner voice rapidly gaining ground in my mind [1]. The
Read more

Reinforcement Learning and the Ergodic Principle across the Multiverses

How we improve a chatbot today? instead of creating a model of the mind, just use the universe to test the answers: the reaction of one person, a thousand, a million.. that’s how a reinforcement learning chatbot would learn today. Now, let’s apply the ergodic principle. Suppose that instead of learning over time , you learn over zillions of simultaneous trials. Now suppose on top of that, that you can try an answer not over a million, not over a googolplex of people, but on an infinity of individuals. All the individuals that compose the multiverse.  Suppose that the chatbot don’t even CARE about the “meaning” of the conversation, but instead of making the person smile, and be happy, and think the chatbot is intelligent - and not only right after its answer, but on all the multiverses that emerge from that interaction (perhaps also towards the past.. but I digress).  The point is: meaning may emerge by a blind measure of a single quantity , the purpose of the conversatio
Read more