Interactive Music Installation Final Reflections: Update #6

Some final reflections..

Last night in the Con cafe, our class had a great night presenting all of our music technology related endeavours. It was a really cool evening to be a part of (thanks to those who came!). The night was filmed, so I’m hoping I’ll be able to upload a video of my demonstration for you to see in the near future.

12241590_10153737997191369_7480534500287822291_n

In light of last night, I’ve been thinking about my installation and how the music is actually created/played with it. What I enjoy about it is that anyone, musical or not can have a go and instantly be able to create something that sounds cool. Another thing is that the buttons and controllers that I’ve used are large enough to be used by people who might struggle with fine motor skills, have a disability, small children, or people who are in rehab.

Here are a few articles on Nintendo Wii’s in rehab (I used nintendo controllers for my installation):

http://www.physicaltherapytoolbox.com/expanded/wii.html

http://www.physicaltherapytoolbox.com/pdfs/wii07.pdf

http://www.centerforphysicaltherapy.com/wii.html

http://www.apta.org/PTCareers/Profiles/HesGotGame/

What if you could combine Music Therapy with this kind of rehab? Using the controllers to create the music? I’m unsure as to how compatible both are, but I think it could be very exciting and interesting area to experiment with and develop.

If you’re reading this and haven’t a clue what I am on about, I recently finished a project that you can catch up on in my previous blog posts.

Peace – Jonno

Interactive Music Installation Tech in Music Ed Task 2: Update #5

Really excited to share that I am finished working on my installation and have also completed the visuals in Mainstage. Here’s a look!

Screen Shot 2015-11-13 at 8.38.07 AM

Using the layout menu in Mainstage, I was able to make the page you see above. As I play the different sounds from the Guitar Hero Guitar and Wii Balance Board, the on screen controls will also move – pretty cool! The squares on the top represent the coloured buttons on the Guitar Hero Guitar and are also coloured to match. On the right shows the volume levels of both the dishwasher and thunder strike sounds. Below is a pitch shift gauge and the backing track playback (blue). The squares resembling the Wii Balance Board light up when you stand in the corresponding places on the actual Wii Board and the X and Y Pad on the left show the Left and Right delays as I control them with the Joystick.

I had a practice run yesterday and all was running well so that’s good. Hoping I don’t run out of batteries on the night but I’ve bought some extras just in case.

Accompanying my installation, I’ve made some instructions so people aren’t completely lost when they have a go.

Screen Shot 2015-11-13 at 9.04.26 AM

Hope to see you there!

Peace – Jonno

Interactive Music Installation Tech in Music Ed Task 2: Update #4

This afternoon I made the decision to only focus on using the Guitar Hero Guitar and the Balance Board as Midi controllers for my installation. There were plenty of buttons and features to utilise on the guitar that I hadn’t yet mapped and I also wanted to simplify the interaction experience for any participants.

Screen Shot 2015-11-11 at 9.12.06 PM

The picture above shows a selection of the instruments (with effects) from Genesis that I have mapped to control from the Guitar Hero Guitar and Wii Balance Board. As you can see, I have added a Tremelo effect and a Pitch Shifter in the Vox Channel and I have also added a limiter to the ‘Gurgles’ to prevent them from clipping. Both the Tremolo and Pitch Shift effects are bypass-able by pressing their corresponding buttons on the Wii Remote inside the Guitar. Button ‘1’ toggles the pitch shift effect, and that can then be controlled by the Whammy Bar and Button ‘2’ toggles the Tremolo effect on and off. This is to create some extra textures and experiment with.

The sound that I am most pleased with is the Thunder. In the recording of Genesis, the thunder rolls in and out with many varying strikes. I still wanted quite a selection of strikes at my disposal for the installation and so I learnt via Music Tech Help Guy (by the way his videos are really great) on Youtube how to alternate different audio samples within the Logic EXS 24 Sampler Instrument. Now I have 5 different lightening strikes mapped to each of the two black buttons (+ and -) on the Guitar Hero guitar. I unchecked the ‘Pitch’ selection so no matter what note I map them to, they will still sound the same. From there I moved them up high on the keyboard so I wouldn’t trigger them with any other buttons, intending to play other sounds.

Screen Shot 2015-11-11 at 3.01.23 PM

So this next picture (below) is of the Sample Delay effect for the instrument that I’ve named ‘Dad’. It’s called that because I sampled my Dad’s impressive tongue clicking skills and they sound really cool with a delay effect on them. Mapping the Right and Left delays to the X and Y axis on the Joy Stick found on the Guitar Hero Guitar (through OSCulator) I am able to control this delay on the go. It’s really cool (I also secretly love just watching the little sliders move as well).

Screen Shot 2015-11-11 at 10.15.03 PM

I then linked this up neatly with an X and Y Pad on the Workspace in Mainstage to make it look pretty. There isn’t any function here really, but the visual cue helps understand what effect it’s having as you change it in real time. The X and Y pad is below on the left and the little dot moves around as the Joystick does.

Screen Shot 2015-11-11 at 9.11.20 PM

The big blue strip along the bottom shows the Playback audio (but at the start is fairly quiet so that’s why it looks bland). I plan to edit and add more features to the Workspace soon.

And here’s a picture showing all the mapping that I have now completed.

Screen Shot 2015-11-11 at 9.11.49 PM

Since I last posted, I’ve decided to sign up and do a demonstration performance of my installation, performing Genesis live – so that should be pretty fun!

And here’s a video of how it was working before I added in all these extra instruments! Hope you enjoy!

As always, if you have any questions or comments please go for it.

Blessings – Jonno

Technology in Music Education Task 2: Update #3

Okay! So much to report here all of a sudden. Yesterday I had both a discouraging and victorious day regarding this assignment.

I spent a large portion of the morning troubleshooting on my computer only to find that somehow I managed to delete Logic Pro (how does that even happen..). So a lot of time vanished as I was trying to fix this, which only really left me with 1 hour to try things out before going to work in the afternoon. At which point I discovered another weird problem. Anywho, point of the story was that not much happened on the computer, but my brain has been hard at work!

But! After work in the evening, I made a lot of progress in a relatively short amount of time with very little/possibly no troubleshooting at all. In fact I was actually able to set up most of my installation. So as you will have read, I will be using Genesis (a recent composition of mine) as the experimental platform to demonstrate the skills that I have taught myself, and the creative potential of this interactive installation. I’ve decided to use Mainstage as the host for the installation. If you are not familiar with Mainstage, basically it is a live version of Logic, designed to be used for performing.

Here’s a photo of all the controllers (including the Wii Balance Board and my iPad) that I have access to and considering using. I have a feeling I won’t be able to use all of them but I will let you know how many I can use without cluttering the interactivity.

IMG_6029

The first thing I achieved was muting a large number of tracks in the final mix of Genesis (in Logic) to make a backing track in which I can layer interactive instruments on. I then opened the Playback plugin in Mainstage and inserted the newly bounced audio file.

Playback

The next step was to open each instrument (that I had muted in the final mix) in new channel strips within Mainstage. The interactivity will come from mapping the controllers that I have to the various instruments and their effects.

In my composition Genesis there’s a couple of sounds that I made using The Mangle, which is a granular synth plugin. I sampled my mum washing up one night (thankfully I didn’t damage my mic!) and opened the audio sample within The Mangle. Taking little (editable) grain sized chunks of the gurgles and sloshes of washing up, The Mangle can play them back at any pitch, speed, reversed, panned etc. The cross hairs you can see is where the grain is playing back from in the audio file. I also randomised the section of where the grains are playing back from whilst also randomising the volume and reversability of the grain play back. This creates a pretty unpredictable sound, especially when the source sound is water.

Screen Shot 2015-11-11 at 9.50.10 AM

It’s pretty limitless. From there I mapped the the rate of the grain playback as well as the pitch of the playback to the Mod Wheel Midi CC – which is 1.

There are Midi CC standard messages which most softwares conform to – here’s a link to a handy site that says what each one typically does.

The next picture shows the Midi mapping from within The Mangle.

Screen Shot 2015-11-11 at 10.09.41 AM

This is all pretty cool if you have a keyboard that is already sending through Midi CC 1 from the Mod Wheel. But I am only halfway there as I want to control this with a Wii Remote. This is where OSCulator comes into play. When you open up OSCulator for the first time it looks like this. Pretty blank.

Screen Shot 2015-11-11 at 9.57.21 AM

Having automatically found the wii remote using the sync button, OSCulator has a number of ‘events’ that each button or motion control can be used for. If you’re using OSCulator for the first time, make sure you press every button on your remote so that the program knows about them. It will automatically know about the motion controls however, as they are always sending through data. In the picture below, you can see that I have two wii remote devices connected. One is in fact the Wii Balance Board and the other is a standard Wii Remote connected to a Guitar Hero Guitar attachment.

 The reason behind the Wii Balance Board is that I want participants of this installation to have a designated place to experience the music from. Looking at the picture below, you can see up the top that I have the ‘SUM’ message mapped to the Midi Note D1. This means whenever someone steps on the board, it will sense their weight and play a D1 on an instrument. I then went ahead and mapped the top left and right regions of the board to two other notes so when the person shifts their weight on the board different notes/pitches will play. However, these pitches aren’t really going to be heard as the sound it will be triggering is going to be water sloshes.

Screen Shot 2015-11-11 at 10.05.03 AM

I got really excited when I started playing with the actual Wii Remote events as I mapped the motion ‘Pitch’ event to the Midi CC 1 Value. What this means is that the Wii Remote effectively just became the Mod Wheel of a keyboard and instantly starting sending value messages to The Mangle grain pitch and rate playback. The graph below shows the values (up to 127) of the Pitch event being changed by the Wii Remote motion within OSCulator (angling the remote up and down).

Pitch

All this together creates a morphing, ambient sloshing and gurgling of water that can only be controlled by standing on the Wii Balance Board and angling the Wii Remote up and down – which means it can be played by anyone. Totally cool. At this point I bounded around the house in victory.

But I wasn’t finished yet! I also had plans for the Pad Sound that I had also created. Sampling my own voice, I was able to use The Mangle to create a very smooth Synth Pad sound by editing the Attack, Sustain and Release of the grain playback. As I used only 4 chords for the Pad Sound, I decided I wanted to map each chord to a different button the Guitar Hero Guitar.

Screen Shot 2015-11-11 at 9.55.12 AM

FullSizeRender

Using the Chord Trigger Midi FX in Mainstage, I was able to map a different key to play one of the four chords. It looks like this (below). Hit the learn button, choose the note you want to play the chord with  on the blue keyboard (top). Then choose the notes that you want in your chord on the orange keyboard below. Don’t forget to hit Learn button again before moving on to the next chord. 

Screen Shot 2015-11-11 at 9.55.02 AM

The reason why I chose notes so high on the blue keyboard was because I haven’t yet found a way to make OSCulator send sounds to a different Midi input. This meant that as I played the pad sound, the water gurgles were also being triggered by the Guitar Hero Notes. I went back to The Mangle and adjusted which keys are allowed to play the gurgles so that I didn’t get any over laps. This is my quick fix, but if you happen to know a solution please comment below as it will just get more messy to program as I add more sounds in.

So now I have both the Wii Remote, the Balance Board and the Guitar Hero Guitar all working together! The really cool part is that all of these are playable by a single person at once as the Wii Remote sits inside the Guitar Hero Guitar. So to adjust the ‘Pitch’ event in OSCulator all you have to do is pretend to be a rock star with the ‘Guitar’ and angle it up and down. Combine that with standing on the Board you can control the sloshes and gurgles whilst playing the Pad chords with the coloured buttons in your left hand.

I want as much of this installation as possible to be ‘computer free’ and so I came up with the idea of mapping a single button on the Wii Remote to begin playing the backing track. That way the entire performance can be controlled without touching the computer. So I mapped the Home button on the Wii Remote to the Playback plugin Play/Stop button in the assignments/mapping menu in Mainstage. Below is a picture. See how on the left there’s a column with ‘OSCulator Out’? Under channel 1, I have made the Midi CC number 21 . Following on to the right you can see it’s mapped to the Play/Stop Switchable button.

Screen Shot 2015-11-11 at 9.56.01 AM

In OSCulator, the event I chose was a Midi CC Toggle event and I also had to choose CC 21. However you have to be careful when trying to map it within Mainstage as it will automatically think that you’re trying to map motion control as they are always sending messages, remember? So you have to enter it in manually within Mainstage to only look for Midi CC 21 and not anything else. To do that click on the ‘Ctrl 21 (21) and edit it in the little menu that pops up.

I’m really excited to share this! The next things that I plan to do is add in some of the other effects that I have in Genesis such as the Thunder (door slams), Tongue Clicks, and Dishwasher Hits. These could prove to be difficult as I’ll have to choose notes that aren’t being used by any other instrument unless I find a way to send through another Midi input from OSCulator into Mainstage.

I also plan to make a poster or something similar to accompany the installation showing how it works so people aren’t so lost when they have a go.

I think that’s everything for now, I’ll let you know how the rest is going shortly!

Peace – Jonno

Technology in Music Education Task 2: Update #2

So a lot has happened between now and my last post regarding this, however not so much has actually happened on this particular project. To be fair I’ve been doing my other assignments, but now is the time that I can start applying more of my brain’s CPU usage to this and I am itching to get started. I have, however, spent some good time brainstorming things I could potentially do for the interactive installation that I am creating and so I’m looking forward to actually attempting to realise those soon.

I went into Uni early on Tuesday morning to meet up with someone who has plenty of advice and experience in this area. I shared with him my idea of connecting various game controllers/gear to my computer to connect with the music programs I have. Showing him OSCulator (which he hadn’t seen before), we spent the next hour or so ‘fiddling’ with the various parameters in both Logic and OSCulator, getting my hardware to properly communicate with them simultaneously.

— Cue amazing artwork of a diagram that I drew on the train to aid those who learn visually —Screen Shot 2015-10-29 at 4.09.54 PM

As a result I have a better understanding of MIDI CC messages and how I can map them to the various parameters and plugins within Logic.

Having just completed an electroacoustic/soundscape composition, I’ve decided to use that work in the interactive installation. I plan to remove two main instruments from the mix and give those who are interested to have a turn with the game controllers. They will have the opportunity to control and create a new version of the piece. Sound like fun? You can hear the piece here. I am still yet to do this, but so far I am thinking that the sounds you can play will be the Pad Sound, the ‘Water Sloshes’ (not sure on a good title for that part), and the Thunder (possibly some more). Along side the installation will be a small key for what each button does for each MIDI Controller so participants can musically decide their choices.

By the way if you are in Sydney and wish to come and see my installation and all the wonderful variety of projects that my classmates have undertaken, visit the Cafe in the Sydney Conservatorium of Music on Friday the 13th of November from 6pm to 9pm.

Please comment below if you plan on coming and come and say hi!

Technology in Music Education Task 2: Update #1

Hi Everyone,

For our next Technology in Music Education assignment, we are to pave our own way – a DIY kind of assignment. Basically I get to choose what I want to do for this entire assignment, as long as I document my progress here on my blog. The catch is that it has to be technology based (who would have thought).

If you aren’t aware, for most of my life I have been a bit of a Nintendo fan boy and so I’ve decided to try and combine my love for that and my love for music. I’m not planning on composing any video game music, but I’m going to be focussing on this little idea:

‘What if you could play music like you play a video game??’

Using a program called OSCulator, I’ll be able to use my Nintendo Controllers as MIDI controllers with any DAW such as Logic Pro X/Mainstage or Ableton. This will enable me to map the various buttons (and motion control!) to different musical notes, instruments or effects. This works because (believe it or not) the nintendo controllers just connect simply via Bluetooth which means not only could I connect those, but I’ll also experiment with connecting and mapping my iPhone to the various music parameters.

I’m pretty excited to try all of this out, I’ll keep you updated as I go.

Skye Boat Song iBook Complete

I have finished working on my ibook and I’m excited to share this resource with you all. Click on this link to find the ibook (which you can use on your iPad or Mac computer) and a brief outline of how this resource could be used as part of a lesson sequence.

Feel free to use all of it or part of it. The only condition is that I would ask you to comment below and let me know how it went and whether I could make any helpful updates to make it better!

Screen Shot 2015-10-04 at 10.37.42 pm