Virtual Loudspeakers over Headphones using Head Tracking- Issued Patent - PRIOR ART REQUEST

Question

10 minutes of your time can help narrow US patent applications before they become patents. Follow @askpatents on twitter to help.

TITLE: Personalized headphone virtualization

Summary: A method for a listener to experience the sound of virtual loudspeakers over headphones.

Publication Number: US7936887

Priority Date: Sep 1, 2004

What is claimed in the patent is:

An audio system for personalized virtualization of a set of loudspeakers in a pair of headphones, the system comprising:

1. an audio input interface for receiving a loudspeaker input signal;

2. a speaker output interface for driving each of a set of loudspeakers with an audio signal;

3. a headphone output interface for driving a pair of headphones with an audio signal;

4. a microphone input interface for receiving response signals from one or more microphones positionable near each ear of a listener;

5. a head tracking system for detecting an orientation of a listener's head;

6. an excitation signal generator coupled to the speaker output interface, wherein when the audio system is in a personalized measurement mode, the excitation signal generator is configured to provide excitation signals to the speaker output interface for driving one or more of the loudspeakers to generate audio responses at a location near each of a listener's ears;

7. a measurement module coupled to the microphone input interface to receive signals from the microphone input interface for the audio responses, the measurement module configured to generate personalized response functions for the audio responses for a plurality of head orientations, and associate each personalized response function with a particular loudspeaker, a particular ear, and a particular head orientation of the listener; and

8. a virtualizer coupled to the headphone output interface, wherein when the audio system is in a normal mode, the virtualizer is configured to transform the loudspeaker input signal using a set of response functions that is based on one or more sets of the plurality of personalized response functions, and provide the transformed loudspeaker input signal to the headphone output interface.

Here are some sample prior art:

http://old.iem.at/Members/noisternig/science/msc

http://interface.cipic.ucdavis.edu/sound/research.html

http://www.umiacs.umd.edu/publications/rendering-localized-spatial-audio-virtual-auditory-space-0

Can you help find more?

---

What is good prior art? Please see our FAQ.

Want to help? Please vote or comment on submissions below. We welcome you to post your own request for prior art on other questionable US Patent Applications.

---

Answer 1

If I'm reading this right, this is a system which can measure how listener head position affects sound received at the ears (or, more properly, a set of microphones very near the ears), then reproduce those effects in a pair of headphones.

The manner in which head position affects the sound a listener hears has been studied for at least eighty years. In 1933, AT&T had a dummy head named Oscar at the Chicago World's Fair that allowed fair-goers to hear what it heard; it attracted a great deal of attention, and similar models were used for years to research the way that the ears and head affect the sound an individual perceives. This gets you 1-2 and 4-7.

From http://www.sony.net/Products/vpt/overview/index.html, about halfway down the page (bolding mine):

When a human-shaped microphone called a dummy-head microphone is used to record 2-channel stereo sound, the sound reproduced does not remain within the listener's head (the sound image is not established in a "fixed position in the head"), bringing an enhanced sense of a live presence. This technology is called binaural recording/reproduction*5. Recognizig the potential of this technology, Sony continued research and development in this area. In parallel, Sony made a number of proprietary technological developments. Among them was a signal processing technology that incorporates the head-related transfer function (HRTF), which indicates the characteristics from the sound source to the listener's ears, using digital signal processing and head-tracking technology*6. As a result, in 1994, we introduced a first-of-its-kind innovative headphone system VIP-1000 ("Virtualphones") that gives the listener the sense that he/she can hear two front L,R speakers, even from a regular 2-channel stereo source.

The link to note 6 describes a gyroscopic system contained within the headphones. This gets you 3 and 8, after a fashion.

So here are the various components of the system described. What I'm not seeing is all of them combined into one system. I suspect that at some point in developing their headphones, Sony had to have built such a system to check their results, but without having a backlog of industry magazines to root through from that era, I have no evidence of prior publication.

Hopefully this will at least be able to point you in a direction for further research.

--Edit: You may want to take a look at US6738479. It appears to have a number of similarities to the above patent, though it uses loudspeakers rather than headphones for the second stage reproduction, and deals with a single sound source rather than a stereo source. Also look at this paper, which describes the system given in this patent in 2001.

Answer 2

In the 1990s Lake DSP developed and marketed a commercial solution for performing head-tracking stabilised spatialisation/auralisation. The DSP processor was called the Heuron. It was used for virtual reality and architectural auralization. Although I don't have a specific reference to someone placing virtual "speakers" with the Heuron, it would have been trivial. Certainly placing virtual sources to map stereo to an ambisonic B-format soundfield is an obvious technique.

For the head-tracked part, here's a quote from a 1996 paper from Lake:

"a new method recently developed at Lake for processing the B-format signals for playback over headphones, with head-tracking used to maintain a stable acoustic field around the listener, even as the listener's head turns" [1]

Lake may well hold a patent on this too.

[1] McGrath, D.S.; Reilly, A., "Creation, Manipulation and Playback of Soundfields with the Huron Digital Audio Convolution Workstation," Signal Processing and Its Applications, 1996. ISSPA 96., Fourth International Symposium on , vol.1, no., pp.288,291, 30-30 Aug. 1996 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=615735&isnumber=13406