Visualizing sound with processing: Part 1

1:37 PM on Saturday, March 23rd, 2019 (Claremont CA)

I recently got back into playing around with Processing, since I have more free time now and am able to devote long spans of time to trying to figure out things that interest me. One of the things that had caught my eye in recent weeks was a YouTuber, Devon Crawford(Link), who had used an Arduino to make a strip of LED lights that would react to sound. Since I didn’t want to wait around for lights to use with my Arduino to come in the mail, I decided I could resurrect my use of Processing and make an animation that would respond to changes in volume and frequency. This entry details my efforts to visualize the volume of a sound source

the process

To begin this project, I had to figure out 2 things: how to observe the volume of a surrounding sound source, and how to observe the frequency of it as well. I started with observing the volume of a sound source. To do this, I had to reflect upon my knowledge of sound as it relates to physics. For a sound wave, it is known that its amplitude determines its volume(aka a large amplitude implies a high volume). The amplitude of a sound wave is shown by Line A in Figure 1.1. Conversely, it is known that the frequency of a sound wave is related to its wavelength, which is shown in Figure 1.1 by Line B. Therefore, I knew that for making a sound visualizer that is responsive to volume, I would have to look at the amplitude of the sound I was processing.

To do this, I used the Amplitude class(Link) from the Sound library that Processing offers. This class creates an Amplitude object that can be assigned to an input, and can then return a float value between 0 and 1 that represents the root mean square of the amplitude of a given sound source. Now that I had a way to observe the relative volume of a sound source, I had a couple more questions to answer:

What animation would represent the sound?
- How would it change with response to the volume?
What would the sound source be/how would it be fed to the Amplitude object?

To answer the first question, I looked about on Processing’s website to see if I could begin by finding a simple animation that I could adapt for my purposes. There I found Daniel Shiffman’s tutorial for a recursive fractal tree(Link) that changed with response to the x location of the mouse. This was the perfect animation to use, since it was already made to adapt to an input, and was easy to implement. For the second question, I decided to have the angle at which the branches extend out from the tree change in response to the volume of the source. I also then implemented a scaling factor that would change the color of the tree with response to the volume as well.

To answer the second question, I used the Processing API to figure out how to create an audio input, as well as how to play a sound file. This means that my program isn’t actually directly reading the sound file, but rather, is playing it and then observing it through my computer’s speakers. For my example, I used 2 songs: When I’m Down by Whethan & Oliver Tree, and Bedford Falls by Ford(2 wonderful songs, by the way).

Below is the resulting code:

import processing.sound.*;
import controlP5.*;
import ddf.minim.*;

float theta; //for the angle at which the fractal tree branches
float hue = 0; //for the color of the stroke
Amplitude amp;
AudioIn in;
SoundFile file;

void setup() {
  size(1500, 1500);
  colorMode(HSB);
  //file = new SoundFile(this, "C:\\Users\\Computer\\Documents\\Arduino\\whenimdown.mp3");
  //file = new SoundFile(this, "C:\\Users\\Computer\\Downloads\\bedfordfalls.mp3");
  amp = new Amplitude(this); //creates a new Amplitude object
  in = new AudioIn(this, 0);
  in.start(); 
  //file.play(); 
  amp.input(in); 
}

void draw() {
  background(0);
  frameRate(30);
  stroke(hue, 255, 255);
  //Picks an angle from 0 to 90 degrees based on the 
  //volume(measured by the root mean square of the amplitude)
  float a = ((amp.analyze()*4000) / (float) width) * 90f;
  theta = radians(a);
  translate(width/2,height);
  line(0,0,0,-400);
  translate(0,-400);
  branch(400);
}

//recursive function to draw the tree
void branch(float h) {
  h *= 0.66;
  if (h > 2) {
    //relates the amplitude to color
    hue =amp.analyze()*4000/4;
    stroke(hue, 255, 255);
    pushMatrix();
    rotate(theta);
    line(0, 0, 0, -h); 
    translate(0, -h); 
    branch(h);      
    popMatrix();     
    
    pushMatrix();
    rotate(-theta);
    line(0, 0, 0, -h);
    translate(0, -h);
    branch(h);
    popMatrix();
  }
}