CS 161 Homework 6 - Guitar Player

Due Fri Nov 01 at 11:59pm

Overview

In this assignment, you will be able to apply your programming skills to the domain of digital audio. Specifically, you will be writing a program to simulate the plucking of a guitar string, creating an application that lets the user create synthesized music by hitting keys on the (computer) keyboard and see a visualization of that music. An example video can be seen below:

This assignment should be completed individually.

Objectives

To practice working with arrays
To practice with loops and conditionals--particularly using them with arrays
To learn about digital audio.
To once again practice with graphics and drawing

Necessary Files

You will need a copy of the Hwk6.zip file. The BlueJ project in this file contains a number of classes for you to use.

GuitarStringTester This is a simple tester that you can use to check that your GuitarString simulation algorithm is working correctly.
GuitarPlayerLite This is a sample main method demonstrating how the user will be able to interact with your GuitarString and "play" different notes. You will be creating a full GuitarPlayer class that is an extension of this.
Important note: Due to a bug in either Java for Windows or BlueJ for Windows, on some Windows machines the input frame doesn't automatically focus, and so doesn't register that you've typed on the keyboard. Minimizing and then restoring the frame seems to fix the problem.
IOFrame This is a provided Graphical User Interface (like a JFrame) that will let you easily respond to the user hitting buttons on the keyboard. You can see an example of how this class is used in the GuitarPlayerLite class. IOFrame has a couple of methods you might use:
- public IOFrame(String title, int width, int height)
  The constructor takes in 3 parameters: the title for the frame, the width of the frame's interior, and the height of the frame's interior
- public boolean hasMoreKeysTyped()
  This method returns whether the user has recently pressed a key (and allows the program to handle multiple keys being pressed at once)
- public char getNextKeyTyped()
  This method returns the char of which key the user typed. So if the user just hit the 'a' key, this method will return 'a'.
- public Graphics2D getGraphics2D()
  This method returns a Graphics2D object that represents the drawing in the frame. You can call this method to get access to that object, and then draw on it from any other method!
You are of course welcome to look over the code for this class. It uses one or two new concepts, but much of it should be familiar. Note that you will not need to edit this class for the assignment--if you find yourself wanting to change it, try to think of another solution (or ask me!)

The zip file also contains the README.txt that you will need to fill out and turn in along with your assignment.

Remember to extract the files from the zip folder before proceeding!

Digital Audio (A Rough Overview)

Digital audio is a complex topic; for this assignment we will be considering it from a very simple and specific point of view. I have placed a short (5-page) simple excerpt from an introductory text on Moodle; I recommend you read through that for an introduction.

The basic idea behind digital audio is the concept that sound waves can be represented mathematically as complex trigometic functions that indicate the amount of "energy" at any particular time. Different properteis of this trig function correspond to differnet properties of the sound. For example, the frequency of the function (how close each "hill" is to the previous) is related to the pitch of the audio (how 'high' the note sounds), while the amplitude of the function (how "high" each hill is) relates to the volume of the sound.

Because sound is continuous but computers work with integers (discrete values), we end up sampling the audio lots and lots of times in order to get a pretty good guess of what the curve looks like. CD quality audio samples the curve 44,100 times per second (measured in what is called hertz (Hz)--1Hz means 1 time per second, 10Hz means 10 times per second, etc). So this means that we'll represent 1 second of audio on a CD with 44,100 numbers--each number representing the sound you can hear for a particular 1/44,100 of a second.

In order to play audio through Java, we represent a "sound" as a collection of numbers (samples) between -1 and 1. You can almost think of each sample as representing a sound that is 1/44,100th of a second long, Since we will have a lot of these samples, we will store them in an array of doubles (a double[]). We can tell Java to either play a single sample (and just tell it to do that over and over), or to play an entire array of samples.

Note that in order to play sound we are using a class called StdAudio developed at Princeton. This class is secretly included in the BlueJ project you downloaded, and doesn't even require you to import any packages! You can see the documentation for this class here. Note that this class contains static methods (and no constructor!), so you don't need to instantiate it. The particular method you'll be using is public static void play(double in), which plays a single provided sample. More detail about this class is included in the Moodle reading.

Strumming a Guitar

When a guitar string is plucked, the string vibrates and creates sound. The length of the string determines its fundamental frequency of vibration. In this assignment, you will model a guitar string by sampling its displacement (a real number between -1/2 and +1/2) at N equally spaced points (in time), where N equals the sampling rate (44,100) divided by the fundamental frequency (rounding the quotient up to the nearest integer).

Plucking the string The excitation of the string can contain energy at any frequency. We simulate the excitation with white noise by setting each of the N displacements to a random real number between -1/2 and +1/2.
The resulting vibrations After the string is plucked, the string vibrates. The pluck causes a displacement which spreads wave-like over time. We will be modeling this vibration using what is called the Karplus-Strong Algorithm. This algorithm played a seminal role in the emergence of physically modeled sound synthesis (where a physical description of a musical instrument is used to synthesize sound electronically).

The Karplus-Strong algorithm simulates this vibration by maintaining an array as a "buffer" (an array used to temporarily store data) of the N samples: the algorithm repeatedly deletes the first sample from the buffer and adds to the end of the buffer the average of the first two samples, scaled by an "energy decay factor" of 0.994).

This process (removing from the front and adding to the back) produces what is called a "ring buffer" (or a "circular array"). We will talk about this data structure in more detail in CS 261.

Why it works

The ring buffer feedback mechanism This "ring buffer" models a medium (like a string tied down at both ends) in which energy travels back and forth. The length of the buffer array determines the fundamental frequency of the resulting sound. Sonically, the feedback mechanism created by having the front of the array influence the back of the array reinforces only the fundamental frequency and its harmonics (frequencies at integer multiples of the fundamental). The energy decay factor (.994 in this case) models the slight dissipation in energy as the wave makes a roundtrip through the string, so that the String eventually stops.
The averaging operation The averaging operation serves as a gentle low-pass filter (which removes higher frequencies while allowing lower frequencies to pass, hence the name). Because it is in the path of the feedback, this has the effect of gradually attenuating the higher harmonics while keeping the lower ones, which corresponds closely with how a plucked guitar string sounds.
Um... what? More intuitively (though less precisely): When you pluck a guitar string, the middle of the string bounces up and down wildly. Over time, the tension on the string causes the string to move more regularly and more gently until it finally comes to rest. High frequency strings have greater tension, which causes them to vibrate faster, but also to come to rest more quickly. Low frequency strings are looser, and vibrate longer.

If you think about the values in the ring buffer as positions over time of a point in the middle of a string, filling the buffer with random values is equivalent to the string bouncing wildly (the pluck). Averaging neighboring samples brings them closer together, which means the changes between neighboring samples become smaller and more regular. The "decay factor" reduces the overall amount the point moves, so that it eventually comes to rest.

The final kicker is the length of our buffer. Low notes have lower frequencies, and hence longer arrays (44,100 / N is bigger if N is smaller). That means you will have to go through more random samples before getting to the first round of averaged samples, and so on. The result is it will take more steps for the values in the buffer to become regular and to die out, modeling the longer reverberation time of a low string.
If you say so... This really does work---I promise!

Bonus confusing fun fact! From a mathematical physics viewpoint, the Karplus-Strong algorithm approximately solves the 1D wave equation which describes the transverse motion of the string as a function of time.

Assignment Details

Implementing this program is less complicated than it may sound (though not simplistic). For this assignment, you will be creating two new classes: GuitarString, which is a "model" or "object" class that represents a single string of a Guitar; and GuitarPlayer, which is a "main" class that lets you interact with keyboard input to strum the strings.

Be sure and name the classes as specified in this write-up!

GuitarString

The first class you'll need to make is the GuitarString class. It will have the following API (method signatures):

public class GuitarString
------------------------------------------------------------------------------------------------------------------------
    GuitarString(double frequency)  // create a guitar string of the given frequency, using a sampling rate of 44,100
    GuitarString(double[] init)     // create a guitar string whose size and initial values are given by the array
    void pluck()                    // set the buffer to white noise
    double sample()                 // return the current sample
    void update()                   // apply the Karplus-Strong update and advance the simulation one time step

Further details are below:

Instance variables
Your GuitarString will need an array to store the samples. Since samples are between -1 and 1, this should be an array of doubles.
- You should also create some constants to represent common values--for example, SAMPLING_RATE has a value of 44,100, while CONCERT_A has a frequency of 440Hz.
Constructors
Your GuitarString class will need two (2) constructors:
- The first constructor takes in a desired frequency and uses that to create the ring buffer (the array). The size of the buffer should be the sampling rate divided by the frequency, rounding up (hint: look at the Math.ceil function). Note that you should also initialize each value of the array to be 0.
- The second constructor takes in a provided array buffer and uses that. This constructor will be used primarily for testing and grading.
pluck()
This method should go through your buffer and replace each element with a random value between -0.5 and 0.5.
- Remember that you can use the Random class to produce a random number, or the Math.random() function. Note that if you are using the Random class, you should instantiate that in your constructor as an instance variable, rather than creating a new object every time you pluck the string.
sample()
This method is easy: it should return the value of the item currently at the front of your buffer.
update()
This method applies the Karplus-Strong update. It should calculate the value of the new sample (as described above--average the first two samples and multiply by the decay factor), remove the first value, and then add the new value to the end of the array.
- Note that because the array is fixed-length, you will effectively need to move every sample up one step (so that the value who was 2nd is now 1st, the value who was 3rd is now second, etc).

Once you have filled in all your methods, be sure and test your code using the provided GuitarStringTester class!

You can also test your program using the provided GuitarPlayerLite class. You should be able to hear two different pitches corresponding to A and C each time you press the key.

If the class doesn't print out that you pressed the key, then your keyboard input isn't being registered. Try making sure the frame has focus (minimizing and restoring it may help).
Also confirm that your volume is turned on so you can hear the sounds!

GuitarPlayer

Once your GuitarString is working, you can write the rest of the program. Create a new GuitarPlayer class that is similar to the provided GuitarHeroLite, but supports a total of 37 notes on the chromatic scale from 110Hz to 880Hz. Your music keyboard should map to the computer keyboard after the following diagram:

Specifically, this keyboard layout is based on the String
```
String keyboard = "q2we4r5ty7u8i9op-[=zxdcfvgbnjmk,.;/' ";
```
where the ith character in the above String plays the ith note. So 'w' plays the 2nd note. The frequencies associated with each key should follow the chomatic scale, so that the character 'q' is 110Hz, 'i' is 220Hz, 'v' is 440Hz, and ' ' is 880Hz. Mathematically, the ith character should have a frequency of 440 × 2^{(i - 24) / 12}. Each key should pluck() an appropriate GuitarString object.
In order to write this class, I suggest you copy and modify the provided GuitarPlayerLite class. Take a moment and read through how that works. In short, it uses an infinite loop to repeatedly pluck() guitar strings whenever the appropriate key is pressed. Since we're interested in hearing the sound of several guitar strings at once, and the combined result of several sound waves is the superposition of the individual sound waves, we simply play the sum of all the string samples!
- Don't even think about defining 37 individual GuitarString variables! Instead, you should create an array of 37 GuitarString objects (putting the appropriate object in the appropriate spot in the array). Using the loop and the above mathematical equation should make this easy.
- Similarly, don't you dare creating a 37-way if statement! Instead, use the indexOf(char key) method of the String class to easily "look up" the index of which key is pressed, which should correspond to an index in your array of GuitarStrings.
  - This is the same process that was recommended for your Pig Latin program, and that has been demonstrated in class!
- Also make sure your program doesn't crash (or throw an Exception/Error) if the key pressed is not one of your 37 notes.
- Finally, you'll need to make sure to play the sum of the samples of all your guitar strings.

Once you've completed your GuitarPlayer class, try playing this familiar melody:

nn//SS/ ..,,mmn //..,,m //..,,m nn//SS/ ..,,mmn   (S = spacebar, not the 's' key!)

You can play the beginning of Led Zeppelin's Stairway to Heaven with the following keys. Multiple notes in a column represent a dyad or a chord.

                                              w q q
        8       u       7       y             o p p
i p z v b z p b n z p n d [ i d z p i p z p i u i i

Visualization

Finally, you should add functionality to your GuitarPlayer class so that it displays a visual representation of the audio. You can get access to the IOFrame's Graphics2D object by calling the getGraphics2D() method. You can then draw on this frame during the while loop, making your drawing depending on the samples produced. Your basic drawing should be a wave form that looks like

but changes over time (as in the above video).

In order to do this, you should have your GuitarPlayer method also create a buffer/array of samples over time--in effect, collect a large list of samples (500 is a fine number) which then you can use to draw. By plotting each sample like on a graph (or drawing a line between consecutive samples), you can produce a nice wave form visualization
DO NOT try and draw the wave form each time through the loop (for each sample)--this will run too slowly and break your audio playback. Instead, try drawing the wave of the last n samples every n time steps. For example, every 500 times through the loop you can draw the last 500 samples. Note that since we're sampling close to 44100 times per second, even only drawing every 1000 samples will still produce animation at 45 frames per second (which looks smooth and nice).
- This means you will need to use a variable to keep track of how many times you've been through the loop!
Note that the Graphics2D class has methods drawOval() and drawLine() that will be easier and faster to draw with (rather than trying to rapidly construct new Ellipse2D.Double or Line2D.Double objects
You will need to "erase" your previous drawing before each new drawing--the easiest way to do this is to set your paintbush's color to Color.WHITE and then use the fillRect() method to white out the entire frame. Then you can se your paintbrush color back to Color.BLACK and draw your picture.
After each time you've drawn your visualization, you will need to refresh the drawing (basically tell the frame that it should update itself--similar to hitting "refresh" on a web browser). You can do this by calling the .repaint() method on the IOFrame object, so:
```
frame.repaint();
```
Do this after you've finished drawing your visualization for each round!
Feel free to experiment with different drawing techniques to produce something that looks nice. If you need help or ideas, please ask (and ask early!)

Helpful Advice: Iterative Development

As always, you should work to complete this assignment iteratively---you should build a small piece, test that it works, and then move on to the next bit. Below is a suggested order of steps, as well as good deadlines for getting each step working (if you haven't figure something out by the listed date, definitely ask for help!!):

GuitarString class: Mon 10/28
GuitarPlayer plays 37 String guitar: Wed 10/30
Visualization functionality: Fri 11/01

Extensions

Here are some possible next steps. It is possible to earn up to 10 points of extra credit on this assignment.

Add extra flashiness to your visualization! Maybe waves of color, 2D effects, etc. Try looking at the visualizations that go along with mp3 players.
Modify the Karplus-Strong algorithm to synthesize a different instrument. Consider changing the excitation of the string (from white-noise to something more structured) or changing the averaging formula (from the average of the first two samples to a more complicated rule) or anything else you might imagine. Alexander Strong suggests a few simple variants you can try:
- Stretched tuning: The frequency formula in the assignment uses "perfect tuning" the doesn't sound equally good in every key. Instead, most musicians use stretched tuning that equalizes the distortions across all keys. To get stretched tuning, using the formula f = 440 × 1.05956i - 24. Try experimenting a bit with the base of the exponent to see what sounds best.
- Extra notes: Add additional keys to your keyboard string to play additional notes (higher or lower). Higher notes especially will benefit from stretched tuning. You will need to update the 24 in your frequency formula to change the frequency of the lowest note.
- Better decay factors: Make the decay factor dependent on the string frequency. Lower notes should have a higher decay factor; higher notes should have a smaller decay. Try different formulas and see what sounds best.
- Harp strings: Flipping the sign of the new value before enqueing it in tick() will change the sound from guitar-like to harp-like. You may want to play with the decay factors and note frequencies to improve the realism.
- Drums: Randomly flipping (with probability 0.5) the sign of the new value before enqueing it in tick() will produce a drum sound. You will need lower frequencies for the drums than for the guitar and harp, and will want to use a decay factor of 1.0 (no decay). The note frequencies for the drums should also be spaced further apart.
- Mix and match: Assign some keyboard keys to drums, others to guitar, and still others to harp (or any other instruments you invent) so you can play an ensemble.
Note that you will need to create a new GuitarPlayer class for this (your basic GuitarPlayer should perform the core assignment so I can grade it!)

Submitting Your Assignment

Be sure and test your program thoroughly to make sure it works, and make sure it is documented with lots and lots of informative comments!
Remember to fill out the README.txt file!
Upload the entire project directory (including all Java files, as well as the README.txt) to the Hwk6 submission folder on the hedwig server. Make sure you upload your work to the correct folder!.
This assignment is due at 11:59pm on Friday, Nov 01.

Grading

This assignment will be graded on approximately the following criteria:

Your GuitarString class uses an array as a sample buffer [10%]
Your GuitarString class applies the Karplus-Strong update [15%]
You have supplied all method of the GuitarString class [10%]
GuitarPlayer class models 37 different guitar strings [10%]
GuitarPlayer class responds to 37 different keys (and only those!) [15%]
GuitarPlayer class plays the combined sampling of all the guitar strings [5%]
GuitarPlayer shows a wave-form visualization based on past samples [20%]
Your code is properly formatted, styled, and commented Each method should have a comment explaining what it does, including parameters and returned values! [10%]
You have completed the included README.txt file [5%]