Digital Video, DIY style

I’ve already mentioned in previous posts that I’ve been working on a music visualiser application based on Digital Signal Processing (DSP) of sound, which could then be used to project images to screen during performances of my music.  It’s a significant detour from writing music itself, but would be particularly valuable for when I am performing instrumental pieces, to add interest to the listener experience. 

Unfortunately, I have a few challenges to overcome before my work so far can be used to animate live music. Namely, I can currently only use the visualiser on pre-recorded music, which obviously isn’t any good for live work, and it is only coping with small files at this stage. So, I need to learn how to get it to accept streaming audio, and figure out how to get a live sound signal into it.  If indeed that is possible.

I discovered another potential use for my work today, however, and that is to use the applications I’m writing to generate video art. It turns out that this is pretty easy to do by recording the app running on my screen with Quicktime then trimming it in video editing software.  The most difficult thing seems to be getting the sound and images to line up correctly where they are supposed to be synchronised, because recording the app running doesn’t capture the sound (unless that was a mistake on my part… I must check out if I missed any settings). I’ve probably not been completely accurate syncing up the attached example, but it’s close enough this time.

More Digital Art

This slideshow requires JavaScript.

I’ve kept going with the Creative Programming course that I talked about last time, and continued coding. After an amazingly good start, I reached a frustrating head-butting stage which I’m not sure I’m out of yet. I’ve now covered the whole syllabus, but need to go back and properly get to grips with waveform synthesis, to be able to do the last assignment and get the qualification. Plus, it might also be useful for performances if I can create an interesting and unique new digital instrument.

I’ve been working mostly on a music visualiser application based on Digital Signal Processing (DSP) of the sound, which I’ll talk about more in another post. In the meantime, whilst the visualiser is still in experimental mode, I thought I’d share a few more images I’ve created with my graphics-only app and an interactive version where sound was played and was changed in pitch and speed by what was happening on the screen.


A Worthwhile Detour


A couple of weeks ago, I went to a couple of events at Lincoln’s Sonophilia Festival (the UK one, not the one in Nebraska). One of these events was the Weird Garden experimental music club, and a chap called Dave C was demonstrating Lissajous curves by generating four tones using a Raspberry Pi and an Arduino board, then plotting these on an oscilloscope, projected so we could all see it. I thought that this would interest my Dad – he’s been known to experiment with a Raspberry Pi – and sent him some photos of the set-up.

Well, this started a conversation and a half. It basically headed in the direction of ‘you should learn some Digital Signal Processing’, with me misunderstanding what that might entail, thinking that it would involve circuit board design and a very steep learning curve. There was a lot of talk at cross-purposes, but Dad eventually explained I would need to learn some new programming skills, in a language called Visual DSP. The boards are already designed, so I wouldn’t need to worry about that, and you can do DSP on your computer, anyway, because computers already have the physical tools needed to do DSP.

I said I would look into it, so that I could possibly learn to present my music in a visual form when playing live. Any programming skills I pick up along the way are a bonus, anyway.  So, I enrolled on a several of Coursera courses, to check out the content, and got stuck into one of these, which I will follow through and hopefully complete.  This course isn’t specifically a DSP course, it is Creative Programming for Digital Media & Mobile Apps, but it seems to be pitched at about my level. It uses Java/Javascript (another language I haven’t used before) so any DSP I do when I get more advanced at programming might need to be via my computer, rather than an external board, unless there is a way round that… Actually, it looks like I can tell it to output the program in Python, the language that Raspberry Pi uses…. I have a lot to learn.

I’m still working my way through week 1 of the course, but the images above are all screenshots made from code that I put together* and then ran and interacted with to make art. I modified the code a little between each image captured.

Even if I get horribly stuck from here onward, I will have an app that I can use to make some unique album art!

*Disclaimer: my code also uses functions taken from a module coded by the course providers.