| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • Stop wasting time looking for files and revisions. Connect your Gmail, DriveDropbox, and Slack accounts and in less than 2 minutes, Dokkio will automatically organize all your file attachments. Learn more and claim your free account.

View
 

Rupakvignesh_FinalProjectDocumentation

Page history last edited by Rupak Vignesh 7 years, 2 months ago

         The "Gesture controlled" Drum-kit. 

 

Concept : The idea behind the project is to let the user play a drum kit by the "gesture of hitting a drum". I made use of the mighty LeapMotion™ controller to get a view of the fingers. I came up with this idea, because to me "interaction" means fun. I felt that the usual way of letting the user interact through touch-screen,buttons,switches was quite ordinary. 

 



 The computer screen shows the instruments in a drum kit and all you have to do is to look at one instrument at a time and perform a "hit gesture".


 

Verplank Diagram.

 

Motivation 

Johnny Chung Lee's DIY article and Pranav Mistry's TED talk.

Functionality

 

When the user gives an input, the leap motion controller gets the input and plots it in an array where the X- and Y- axes are the width and the height of the monitor and the Z-axis is the proximity of the finger from the controller against the direction of "hitting the drum". I used the Leap motion library in processing (an application that supports Java). The Library come with an example of plotting your fingers' tip on your display's coordinates whenever you move your hand. Note that four or five blobs get plotted at one time,since the controller gets a view of your whole hand. Be sure to use only one finger when you hit a drum or else you are going to here interference of all the drums. 

Also to notify you which drum you hit, I made an indicator with six LEDs. The processing file talks to my arduino board serially through a program called Serialread which you can find under the processing examples. When the user hits a drum, the corresponding LED glows. 

I simply divided the application window into six regions and gave 'if' condition checks to know which region the user hits. For instance, When the user hits the first region, 

if (a < -40 && b>140 && d<60) {  

    myPort.write('A'); 

     kick.trigger(); 

 

   } //this is the sketch in processing 

*reading it in arduino*

if (Serial.available()) { // If data is available to read,

 val = Serial.read(); // read it and store it in val

 }

if (val == 'A') { // If A was received

 digitalWrite(ledPin1, HIGH);

digitalWrite(ledPin2, LOW);

digitalWrite(ledPin3, LOW);

digitalWrite(ledPin4, LOW);

digitalWrite(ledPin5, LOW);

digitalWrite(ledPin6, LOW);

 }

The Indicator


 

 


 Arduino Code (that you upload first before running the processing program)

 

 char val; // Data received from the serial port

 int ledPin1 = 12; 

 int ledPin2 = 10;

 int ledPin3 = 8;

 int ledPin4 =6;

 int ledPin5 = 4;

 int ledPin6 = 2;

 

 void setup() {

 pinMode(ledPin1, OUTPUT); // Set pin as OUTPUT

 pinMode(ledPin2, OUTPUT);

 pinMode(ledPin3, OUTPUT);

 pinMode(ledPin4,OUTPUT);

 pinMode(ledPin5,OUTPUT);

 pinMode(ledPin6,OUTPUT);

 Serial.begin(9600); // Start serial communication at 9600 bps

 }

 

 void loop() {

 if (Serial.available()) { // If data is available to read,

 val = Serial.read(); // read it and store it in val

 }

 if (val == 'A') { // If H was received

 digitalWrite(ledPin1, HIGH);

digitalWrite(ledPin2, LOW);

digitalWrite(ledPin3, LOW);

digitalWrite(ledPin4, LOW);

digitalWrite(ledPin5, LOW);

digitalWrite(ledPin6, LOW);

 }

 else if(val == 'B') {

 digitalWrite(ledPin2, HIGH);

digitalWrite(ledPin3, LOW);

digitalWrite(ledPin1, LOW);

digitalWrite(ledPin4, LOW);

digitalWrite(ledPin5, LOW);

digitalWrite(ledPin6, LOW);

 }

 else if(val == 'C') {

 digitalWrite(ledPin3, HIGH); 

 digitalWrite(ledPin1, LOW);

 digitalWrite(ledPin2, LOW);

 digitalWrite(ledPin4, LOW);

 digitalWrite(ledPin5, LOW);

 digitalWrite(ledPin6, LOW);

 }

 else if(val =='D'){

  digitalWrite(ledPin4, HIGH); 

  digitalWrite(ledPin1, LOW);

  digitalWrite(ledPin2, LOW);

  digitalWrite(ledPin3, LOW);

  digitalWrite(ledPin5, LOW);

  digitalWrite(ledPin6, LOW);

 }

 else if(val == 'E'){

   digitalWrite(ledPin5, HIGH);

   digitalWrite(ledPin1, LOW);

   digitalWrite(ledPin2, LOW);

   digitalWrite(ledPin3, LOW);

   digitalWrite(ledPin4, LOW);

   digitalWrite(ledPin6, LOW);

 }

 else if(val == 'F'){

   digitalWrite(ledPin6, HIGH);

   digitalWrite(ledPin1, LOW);

   digitalWrite(ledPin2, LOW);

   digitalWrite(ledPin3, LOW);

   digitalWrite(ledPin4, LOW);

   digitalWrite(ledPin5, LOW);

 } 

}

 

Processing code 

 

/**

 * Simple Write. 

 * 

 * Check if the mouse is over a rectangle and writes the status to the serial port. 

 * This example works with the Wiring / Arduino program that follows below.

 */

import com.leapmotion.leap.Finger;

import com.leapmotion.leap.Frame;

import com.leapmotion.leap.Hand;

import com.leapmotion.leap.Controller; 

import com.leapmotion.leap.Vector;

import java.util.Map;

import java.util.concurrent.ConcurrentMap;

import java.util.concurrent.ConcurrentHashMap;

import com.leapmotion.leap.processing.LeapMotion;

import processing.serial.*;

import ddf.minim.Minim;

import ddf.minim.AudioSample;

import ddf.minim.AudioPlayer;

 

Minim minim;

AudioSample plate;

AudioSample rattle;

AudioSample sheet;

AudioSample snares;

AudioSample kick;

AudioSample bass;

 

PImage img;

PImage img2;

PImage img3;

PImage img4;

PImage img5;

PImage img6;

 

Serial myPort;  // Create object from Serial class

int val;        // Data received from the serial port

 

LeapMotion leapMotion;

ConcurrentMap<Integer, Integer> fingerColors;

ConcurrentMap<Integer, Vector> fingerPositions;

 

void setup() 

{

  size(16*50, 9*50);

  background(20);

  frameRate(120);

  ellipseMode(CENTER);

 

  leapMotion = new LeapMotion(this);

  fingerColors = new ConcurrentHashMap<Integer, Integer>();  

  fingerPositions = new ConcurrentHashMap<Integer, Vector>();

  // I know that the first port in the serial list on my mac

  // is always my  FTDI adaptor, so I open Serial.list()[0].

  // On Windows machines, this generally opens COM1.

  // Open whatever port is the one you're using.

  String portName = Serial.list()[0];

  myPort = new Serial(this, portName, 9600);

 

  minim = new Minim(this);

  plate = minim.loadSample("metalplate.wav", 512);

  rattle = minim.loadSample("metalrattle.wav", 512);

  sheet = minim.loadSample("metalsheet.wav", 512);

  snares = minim.loadSample("metalsnares.wav", 512); 

  kick = minim.loadSample("kick.mp3",512);

  bass = minim.loadSample("bass.wav",512);

 

  img=loadImage("snaredrum.jpeg");

  img2 = loadImage("drumkick.jpeg");

  img3= loadImage("rattle.jpeg");

  img4 = loadImage("sheets.jpeg");

 img5 = loadImage("plates.jpeg");

 img6 =loadImage("lowkick.jpeg");

}

void draw() {

 

  fill(255);

  rect(0, 0, width, height);

  line(width/3,0,width/3,height - 100);

  line(width*2/3,0,width*2/3,height-100);

  line(0,(height-100)/2,width,(height-100)/2);

  line(0,(height-100),width,(height-100));

 

  img.resize(0,150);

    image(img, width*2/3 +5,5); 

  img2.resize(0,150);

image(img2, 5,5);

img3.resize(0,150);

image(img3,width/3 +60,(height-100)/2 +5);

img4.resize(0,150);

image(img4,5,(height-100)/2 +5);

img5.resize(0,150);

image(img5,width/3 +5, 5);

img6.resize(0,150);

image(img6,2*width/3 + 5, (height-100)/2 +5);

 

 

}

 

void onFrame(final Controller controller)

{

  Frame frame = controller.frame();

  for (Finger finger : frame.fingers())

  {

    int fingerId = finger.id();

    fingerPositions.put(fingerId, finger.tipPosition());

    Vector x = finger.tipPosition();

    float a = x.get(0);

    float b= x.get(1);

    float d = x.get(2);

    System.out.println();

 

    if (a < -40 && b>140 && d<60) {  

    myPort.write('A'); 

     kick.trigger(); 

 

    } 

    else if(a > -40 && a<40 && b >140 && d<50){

    myPort.write('B');

    plate.trigger();

    }

 

    else if(a>  40 && b>140 && d<50) {                                        

    myPort.write('C'); 

    snares.trigger();

    }

  else if(a< -40 && b<125 && d<50){

    myPort.write('D');

    sheet.trigger();

  }

  else if(a> -40 && a<40 && b<125 && d<50){

    myPort.write('E');

    rattle.trigger();

  }

  else if(a>40  && b<125 && d<50){

    myPort.write('F');

    bass.trigger();

  }

  else{

    myPort.write(' ');

  }

  }

}

 

static float LEAP_WIDTH = 120.0; // in mm

static float LEAP_HEIGHT = 220.0; // in mm

 

float leapToScreenX(float x)

{

  float c = width / 2.0;

  if (x > 0.0)

  {

    return lerp(c, width, x/LEAP_WIDTH);

  }

  else

  {

    return lerp(c, 0.0, -x/LEAP_WIDTH);

  }

}

 

float leapToScreenY(float y)

{

  return lerp(height, 0.0, y/LEAP_HEIGHT);

}

void stop()

{

  plate.close();

  rattle.close();

  sheet.close();

  snares.close();

  kick.close();

  minim.stop();

  bass.close();

  super.stop();

}

 

Video link :
http://www.youtube.com/watch?v=RAgFGGXA4uA&list=HL1376895098 

 

 

 


 

Comments (1)

kdade said

at 12:34 pm on Aug 20, 2013

Dear Rupak,

We like your idea of using gestures to control drum sounds, and would love to hear even more about a specific point of view. Perhaps the idea is that drummers want to practice playing their beats on-the-go, which isn't possible with a traditional drumset.

We also like your concept sketch, and it clearly shows the intended interaction you are designing for. We wish that in your Verplank diagram, you had been a little clearer about what the images represent - we can't quite follow the relationship between the metaphor and the project, although your drawings are great and really clear!

We really like your final enclosure, and we would love to see documentation of any prototypes (paper or otherwise) you might have used in designing the final box. We always love to read about the different design steps and obstacles you faced during your project.

This is a great project idea, and you executed it really well - you should be very proud of your working hands-free drum machine!

Best,
Kevin

You don't have permission to comment on this page.