Adv Projects in Computer Music

Helping Annotate Music

2006-11-28T19:00:00.000-08:00

Dear friends,

For my PhD thesis, I'm working on a computer system that learns to associate music with appropriate and descriptive words . The machine listens to some music and reads the corresponding text description. After doing this for long enough, our clever little computer learns what words ought to be used to describe new music or, if you give it some words, it can play you songs that match your description.

While I have lots of music, I'm lacking the corresponding text descriptions. This is where you come in. I've built a website where you can listen to songs and fill out a form that records what you hear in the music. It involves clicking some boxes and buttons - no typing - and it's very straight-forward. Every click that you make is recorded and then I can use your descriptions to train my computer system.

So I ask you, please;

1) Go to http://cosmal.ucsd.edu/cal/friends/
2) Register with a username (e.g. DougTurnbull)
3) Answer a few questions about your musical background
4) Listen to some music!

Once you've registered, you can log on again at any time and pick up where you left off. If you can annotate even 2 or 3 songs, it would be a great help to me and you might discover some cool new tunes. If you have any more questions about what it all means, how it really works and what, exactly, I do... please get in touch.

Thanks,
Doug Turnbull & Luke Barrington

ps - Please forward this to anyone you think might be interested in helping out.

--
-- --- ----- ------- ----------- ------------- -----------------
Douglas Turnbull
Graduate Student
Computer Science
UC San Diego

New Musical Interface: Air Guitar

2005-11-30T10:09:00.000-08:00

I am guessing some of you have seen the website for the "Project: Air Guitar", but if not, here is some light reading:

http://www.newscientist.com/article.ns?id=dn8383
http://airguitar.tml.hut.fi/tech.html

The authors discuss gesture recognition and musical style. It seems like a pretty cool, though somewhat kitschy project to me. I bet it would have made waves at the Cal-IT2 opening.

Doug

--
-- --- ----- ------- ----------- ------------- -----------------
Douglas Turnbull
Graduate Student
Computer Science
UC San Diego

Re: branch prediction and multithreading

2005-11-18T09:56:00.000-08:00

here is the class website !

http://www-cse.ucsd.edu/classes/fa05/cse240a/lecture.html

> Here is a list from Dave on issues of computer architecture.
>
>
>> Multithreading
>>(papers found in following website)
>>http://www.cs.washington.edu/research/smt/
>>1) Simultaneous Multithreading: Maximizing On-Chip Parallelism
>>2) Compilation Issues for a Simultaneous Multithreading Processor
>>3) Simultaneous Multithreading: A Platform for Next-generation Processors
>>
>>Branch Predictors
>>
>>Trading Conflict and Capacity Aliasing in Conditional Branch Predictors.
>>www.lems.brown.edu/~iris/en291s9-04/ papers/seznec-branch-isca97.pdf
>>
>>(papers below found in following website)
>>(http://www-cse.ucsd.edu/classes/fa05/cse240a/proj.html)
>>
>>McFarling, Combining Branch Predictors, WRL TN-36. Good description of
>>local, correlating, gshare, and combining predictors.
>>
>>Kessler, The Alpha 21264 Microprocessor, IEEE Micro, 1999. Uses variant
>>of McFarling's combining (tournament) predictor.
>>
>>A. Eden, and T. Mudge. The YAGS branch predictor. 31th Ann. IEEE/ACM
>> Symp.
>>Microarchitecture (MICRO-31), Dec. 1998. Good description of a variety
>> of
>>anti-aliasing predictors (predictors that still work with large working
>>sets of branches). Don't take their word that theirs is best without
>>testing it out...
>>
>>C.-C. Lee, I.-C. Chen, and T. Mudge. The bi-mode branch predictor. 30th
>>Ann. IEEE/ACM Symp. Microarchitecture (MICRO-30), Dec. 1997. Although
>> the
>>previous had a good description of bi-mode, this is the original.
>>
>>A. Seznec, S. Felix, V. Krishnan, Y. Sazeides. "Design trade-offs for the
>>Alpha EV8 conditional branch predictor", in : Proceedings of the 29th
>>International Symposium on Computer Architecture, May, 2002. Take a deep
>>breath before you enter...
>>
>>
>>Thanks,
>>David Camargo
>

branch prediction and multithreading

2005-11-17T11:05:00.000-08:00

Here is a list from Dave on issues of computer architecture.

> Multithreading
>(papers found in following website)
>http://www.cs.washington.edu/research/smt/
>1) Simultaneous Multithreading: Maximizing On-Chip Parallelism
>2) Compilation Issues for a Simultaneous Multithreading Processor
>3) Simultaneous Multithreading: A Platform for Next-generation Processors
>
>Branch Predictors
>
>Trading Conflict and Capacity Aliasing in Conditional Branch Predictors.
>www.lems.brown.edu/~iris/en291s9-04/ papers/seznec-branch-isca97.pdf
>
>(papers below found in following website)
>(http://www-cse.ucsd.edu/classes/fa05/cse240a/proj.html)
>
>McFarling, Combining Branch Predictors, WRL TN-36. Good description of
>local, correlating, gshare, and combining predictors.
>
>Kessler, The Alpha 21264 Microprocessor, IEEE Micro, 1999. Uses variant
>of McFarling's combining (tournament) predictor.
>
>A. Eden, and T. Mudge. The YAGS branch predictor. 31th Ann. IEEE/ACM Symp.
>Microarchitecture (MICRO-31), Dec. 1998. Good description of a variety of
>anti-aliasing predictors (predictors that still work with large working
>sets of branches). Don't take their word that theirs is best without
>testing it out...
>
>C.-C. Lee, I.-C. Chen, and T. Mudge. The bi-mode branch predictor. 30th
>Ann. IEEE/ACM Symp. Microarchitecture (MICRO-30), Dec. 1997. Although the
>previous had a good description of bi-mode, this is the original.
>
>A. Seznec, S. Felix, V. Krishnan, Y. Sazeides. "Design trade-offs for the
>Alpha EV8 conditional branch predictor", in : Proceedings of the 29th
>International Symposium on Computer Architecture, May, 2002. Take a deep
>breath before you enter...
>
>
>Thanks,
>David Camargo

ICMC miami paper- network performance

2005-11-04T11:04:00.000-08:00

http://ccrma.stanford.edu/~grace/gurevich.pdf

Follow up: Moving with or without phase delay

2005-10-21T13:54:00.000-07:00

One of the conclusions of our little test today was that moving sounds with
phase delay seem to create flanging (it is not clear why this does not
happen with panning, or if it does, why we are less sensitive to it). An
interesting idea that Miller suggested was to see if phase helps in helping
the brain better localize non-moving sources at different locations in
space. This means running the experiment with actually recordings and
constant localization using the two methods.

There will be no meeting next Friday due to CALIT2 building opening. Please
come to the presentations!

Our general plans for future meetings are:
1. Follow up on the "phase for localization" idea.
2. Windowing / instanteneous freq. within one fft (Miller / Shlomo?)
3. Grace paper from ICMC04 and survey of Spatialization techniques.
4. David: awaiting your bibliography
5. Joe: ADIOS ? (algorithm)
6. Doug: talk about his recent work on annotation.

Moving with or without phase delay

2005-10-21T08:03:00.000-07:00

I think it is possible to tell the difference in sounds moving with and
without a phase delay. I'll play some examples today.
The code is at http://music.ucsd.edu/~sdubnov/Mu270d

2005-10-16T18:21:00.000-07:00

Hi all,

Please ignore my last post. I've decided to make a blog all for myself so I won't clog this one. Here it is.

Thanks, Grace

Spatialization techniques

2005-10-16T10:59:00.000-07:00

Here is what I have compiled so far about current spatialization technology, and I've tried to roughly categorize the papers. As I mentioned in the last meeting, I won't be able to come to this week's meeting, but I would appreciate some feedback on this list.

Thanks, Grace

Spatial Perception

Spatial perception is frequency dependent, cues mostly come from higher spectral components, which makes it easier to localize widely distributed spectra.

Frequency cues for localization seems to be based on experience. In nature, high frequency sounds are more likely to originate above our heads and low sounds from below. Inverting this mapping will create bad localization.

Locating stationary sounds is easier; adding Doppler shift eases localization.

Malham, CMJ 19(4)

D. G. MALHAM “Approaches to spatialization” Organised Sound 3(2): 167–77.
J. Chowning, "The simulation of moving sound sources", Journal of the Audio Engineering Society, vol. 19, no. 1, pp. 2-6, 1971.
F. R. Moore, "A general model for spatial processing of sounds", Computer Music Journal, vol. 7, no. 6, pp. 6-15, 1983.
M. Kleiner, B.-I. Dalenback, P. Svensson, "Auralization - An overview", Journal of the Audio Engineering Society, vol. 41, no. 11, pp. 861-875, 1993
Thiele, G. and G. Plenge “Localization of Lateral Phantom Sources: JAES 25(4): 196-200.

Hrtf-based binaural

Mimic the change in timbre, delay, and amplitude that happens directly at the ears.

Are most successful when personal hrtfs are used to custom tailor the experience for the listener, and when the listener’s head movement is tracked (a way to prevent front-back reversal error). Measurements of dummy heads such as KEMAR provide an approximate effect. Recordings made in binaural format can be transferred to stereo using interaural crosstalk cancellation, though the result will have an extremely small sweet spot. Binaural setups are suitable for Internet concerts, but not for traditional concert settings. HRTFs take a huge amount of processing power compared to other spatialization techniques.

Researchers: Larcher, Jot, Warusfel

Véronique Larcher et Jean-Marc Jot “Techniques d'interpolation de filtres audio-numériques: Application à la reproduction spatiale des sons sur écouteurs” Congrès Français d'Acoustique, Marseille, France, Avril 1997
Psychophysical calibration of auditory range control in binaural synthesis with independent adjustment of virtual source loudness William L. Martens PDF (JASA)
Practical system for recording spatially lifelike 5.1 surround sound and 3D fully periphonic reproduction Robert E. (Robin) Miller III PDF (JASA)
Individualized HRTFs using computer vision and computational acoustics, JASA Volume 108, Issue 5, p. 2597 PDF
Cooper, D. H., and Bauck, J. L. 1989. "Prospects for transaural recording". J. Audio Eng. Soc. 37(1/2).

Cinema-style loudspeaker arrays (5.1, 8-channel)

Phantom sound images are created. When there is at least an 18dB difference in power between loudspeakers, the sound-producing body will seem to be located near the louder speaker. The perceived width of the sound image varies depending on the spectral content of the sound. The optimum angle between speakers is 60 degrees, meaning that a minumum of six speakers is needed to cover the two dimensional space surrounding the listener; smaller numbers of speakers will create more unstable images. The speaker layouts cannot change between studio monitoring and concert hall settings.

VBAP (Vector-Based Amplitude Panning)
Jot, Jean-Marc et Olivier Warusfel: Spat~ : A Spatial Processor for Musicians and Sound Engineers, CIARM: International Conference on Acoustics and Musical Research, Mai 1995.
Jot, J.-M., and Warusfel, O. 1995. "A real-time spatial sound processor for music and virtual reality applications". Proc. 1995 ICMC
Dérogis, Philippe, René Caussé et Olivier Warusfel: On the Reproduction of Directivity Patterns Using Multi-Loudspeaker Sources, ISMA: International Symposium of Music Acoustics 1995

Ambisonics

Ambisonic stages can be either pantophonic (2-D) or periphonic (3-D or with-height). Ambisonic encoding and decoding are completely separate and mostly autonomous processes, meaning that decoding of an ambisonic signal can be performed with little information about the encoding process. Ambisonic microphones can provide 3-D encoded signals. After encoding, each dimension (x,y,z,w) of an ambisonic signal can be manipulated independently. A pantophonic setup requires 3 channels and 4 loudspeakers, and a with-height setup requires 4 channels and 8 loudspeakers.

Peter Felgett. ``Ambisonics. Part One: General System Description''. Studio Sound, 1:20--22,40, August 1975.
Michael A. Gerzon. ``Periphony: With-Height Sound Reproduction''. Journal of the Audio Engineering Socitey, 21(1):2--10, 1973.
Michael A. Gerzon. ``Ambisonics. Part two: Studio Techniques''. Studio Sound, pages 24--30, October 1975
Malham, David 'Higher order Ambisonic systems for the spatialisation of sound' Proceedings, ICMC99, Beijing, October 1999
Malham, D and Myatt, A “3-D Sound Spatialization using Ambisonic Techniques” CMJ 19(4) 1995
D.G. Malham “Homogeneous And Nonhomogeneous Surround Sound Systems”.
Ambisonics Encoding of Other Audio Formats for Multiple Listening Conditions Jérôme Daniel, Jean-Bernard Rault, and Jean-Dominique Polack, AES Convention 1998
David Malham at University of York

Holophonics and Wave-field synthesis

Multiple speakers are used to represent secondary sources of the wave front (see Huygens’ Principle). Because the actual sound wave is reproduced, this technique creates an acceptable image over a larger area, but spatial aliasing is a potential error.

Acoustic rendering with wave field synthesis, Marinus M. Boone
Wave field synthesis: A promising spatial audio rendering concept, Gunther Theile and Helmut Witteky, Acoust. Sci. & Tech. 25, 6 (2004)
Wave Field Synthesis And Analysis Using Array Technology, Diemer de Vries and Marinus M.Boone, Proc. 1999 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, New York, Oct. 17-20, 1999
Nicol, R and M. Emerit 1999

Miscellaneous

Hyper-dense transducer array (Malham)
Hybrid speaker-headphone approach?
Huopaniemi, J. 1999. Virtual acoustics and 3-D sound in multimedia signal processing

Organized Sound Issue 3(2)
Computer Music Journal Issue 19(4)
Jean-Marc Jot Synthesizing Three-Dimensional Sound Scenes in Audio or Multimedia Production and Interactive Human-Computer Interfaces 5th International Conference: Interface to Real & Virtual Worlds, Montpellier, France, Mai 1996
Belin, Pascal, Bennett Smith, L Thivard, Sophie Savel, Séverine Samson et Yves Samson: The functional anatomy of sound intensity change detection, Society for Neuroscience, 1997.
Belin, Pascal, Stephen McAdams, Bennett K Smith, Sophie Savel, Lionel Thivard et Séverine Samson: The functional anatomy of sound intensity discrimination, Journal of Neuroscience, 1998.
Jean-Pascal Jullien, Olivier Warusfel Technologies et perception auditive de l'espace. Cahiers de l'Ircam (5), mars 1994
Jot, Jean-Marc: Efficient models for reverberation and distance rendering in computer music and virtual audio reality, ICMC: International Computer Music Conference, Septembre 1997.

Conferences Calendar link

2005-10-12T10:25:00.000-07:00

Hi all,

I found this nice calendar with submission deadlines for audio/computer music/signal processing conferences.

http://www.iua.upf.es/mtg/workshops/

Grace

No 270D meeting Friday Oct 14

2005-10-11T12:33:00.000-07:00

Hi all,

Due to a rehearsal that I have this Friday, Oct 14, there will be no
meeting of 270D.
For the next meeting please try to post your bibilography surveys on the blog.

Cheers,
Shlomo

2005-10-07T17:27:00.000-07:00

The paper I mentioned is BOUALEM BOASHASH, "Estimating and Interpreting The Instantaneous Frequency of a Signal-Part 1: Fundamentals", PROCEEDINGS OF THE IEEE, VOL. 80, NO. 4, APRIL 1992 http://ieeexplore.ieee.org/iel1/5/3687/00135376.pdf?tp=&arnumber=135376&isnumber=3687

2005-09-28T14:06:00.000-07:00

This article in the newest issue of CMJ talks about networked music and performances, which the ICMC paper examines:

http://muse.jhu.edu/journals/computer_music_journal/v029/29.2weinberg.pdf

2005-09-28T10:26:00.000-07:00

Here's our ICMC 2004 paper:

http://ccrma.stanford.edu/~grace/gurevich.pdf

Welcome

2005-09-26T17:34:00.000-07:00

This is the blog page for Music 270D, UCSD, Fall 2005, led by Shlomo Dubnov.

In the first few weeks we will make an overview of recent publications in major journals on Computer Music and related technologies.

The journals are:
CMJ - Computer Music Journal: http://mitpress2.mit.edu/e-journals/Computer-Music-Journal/
JNMR - Journal of New Music Research: http://www.tandf.co.uk/journals/titles/09298215.asp
OS - Organised Sound: http://journals.cambridge.org/action/displayJournal?jid=OSO
TSA - IEEE Transaction on Speech and Audio Processing: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?puNumber=89
JASA - Journal of the Acoustical Society of America: http://scitation.aip.org/jasa/
JAES - Journal of the Audio Engineering Society: http://www.aes.org/journal/search.html

Also special issues of interest:
JASIST Vol. 55 #12: http://www.asis.org/Publications/JASIS/jasis.html