Learning Pieces

Final Idea

After testing various fan-driven possibilities and having no progress in texture sensing of a desert, I started to spread other ideas around MIDI controller, in the thinking process of project 2, I also gave up a plan applying color sensors. So it's an opportunity to explore color instruments for this project, and what I have in my hands is an RGB sensor named TCS3200 from DFROBOT.

MIDI commands

First time I tried to implement the MIDI controller example from Tom on Uno because I thought Uno is more powerful to drive the sensor module. But I encountered an unknown error like below, then I moved everything to Nano 33, the error didn't jump out again.

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/b0867e5c-696a-4d30-ba50-0c6126f7ee07/uno_error.png

Then I met a huge obstacle that took up me so much time to debug: any Digital Audio Workstation couldn't play notes even they showed my Arduino connected. I checked every wire in mentioned MIDI controller circuit, as well as added returned numbers in the serial monitor, which proved nothing wrong on the physical end.

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/d71e53d6-9fe5-4d00-94a6-6fce3a3f8b1b/midi_test1.heic

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/5b371638-008d-46fc-8922-da005e953625/midi_test2.heic

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/e3e87344-0b7f-41f7-a83b-e77e5c578fb4/midi_disconnected.png

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/a4f9e6b7-b79d-4a66-b661-943fa06b840e/mini_connected.png

Since the hardware had no problem, so did the software, I moved to check the code. Finally, I found the following function actually didn't send MIDI messages out to DAW:

void midiCommand(byte cmd, byte data1, byte  data2) {
  Serial1.write(cmd);     // command byte (should be > 127)
  Serial1.write(data1);   // data byte 1 (should be < 128)
  Serial1.write(data2);   // data byte 2 (should be < 128)
}

The bug was removed after I added following two lines from Tom's one-key MIDI player example:

midiEventPacket_t midiMsg = {cmd >> 4, cmd, data1, data2};
  MidiUSB.sendMIDI(midiMsg);

https://player.vimeo.com/video/488921475

Interface

I made a two-sides color disk, including yellow, orange, red, black, green, blue, purple, white(my Airpods instead...) to test the sensor range. As the video shows, the initial ranges are very crowded, so I mapped them into 510~0(2*255[RGB values]) to get a more visible difference.

https://player.vimeo.com/video/488917323

Then I suddenly realized the most flexible moving tool to sense colors is the human hand. So I converted the position relation between the color disk and the sensor module. Below glove born. In consideration that I have to use one hand to take process videos and photos, I only made a one-hand glove with five colorful finger caps, one black pad in the palm of my hand, and one white pad on the back of my hand.

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/cd2c6cd8-271b-469d-80e3-b46f60a8f403/glove_back.heic

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/902e9103-78e3-474c-aa32-19809dba9239/glove_front.heic