Wave Field Synthesis and Holophony

Helmut Oellers, Germany

Summary

The animation shows the propagation of the (black drawn) direct acoustic wave of a sound source and its first reflections in the (large outer) recording room. Their starting points seem to be the (multicoloured) mirror sources behind the recorded room's walls. Her spatial distribution is essential for our spatial perception of the sound event. However, conventionally loudspeaker reproduction procedures cannot reconstruct this complex spatial structure. The reduction into a few transmission channels inevitably causes a significant loss of spatial information.

This page describes a different way to restore the mirrored sources closely at its correct positions. Just like the recording room creates all reflections from the source, a sound field synthesis restores the reflections from the dry recorded source and information regarding the recording room's properties. This “Holophoy” approach relies on the principles of Wave Field Synthesis, which was described by Berkhout at the University of Delft in the 1980s. Kirchhoff- Helmholtz- Integral proves the possibility to restore the complete sound field in theory, but some practical constraints have made that goal unfeasible until today. A number of scientific institutes have implemented this theory very successfully, but have reduced it to only one single horizontal loudspeaker row around the listener. This limits the procedure to the horizontal plane of the listener.

The “Holophoy” solution is based on a large-scale frontal WFS Loudspeaker screen - colored magenta in the animation. Within the near field of such a huge resulting diaphragm the playback room's acoustics become a trivial matter. Its reflections no longer must be eliminated by strong damping, but are purposefully included in the synthesis. Differently all conventional audio procedures, which merge together all signal components, during synthesis is direct wave, early reflections and reverberation are discretly accessable. The extreme directive effect of the loudspeaker screen thus allows, decelerate direct wave and reflections each other. By this way becomes possible, the playback room reflections faking the recording room's reflections in time, level and direction.

1. Conventioal audio

Many of the stereo freaks believe devoutly, as long as all components in the transmitting chain are perfect, the reproduction would hardly differ regarding genuine event. In fact, we are able today for create subjectively a very satisfying playback, though in conventionally approaches each reproduction still differ utterly from the spatial structure of genuine sonic field.

Not in any case thats really disturbing, especially actual studio productions doesn't align at Alain Blumelines “being there” goal. Most recordings are a “Product of Art” which provide, regardless of all limitations of the phantom source based procedures, a subjectively and emotionally excellent Playback.

But albeit, despite increasing amount of transmitting channels until today we are not really capable for recreating the acoustic impression of an famous concert hall. Tonal accuracy is all, what we can expect from traditional audio, true spatial reproduction never happens. The signals on our eardrums differ seriously, compared with the signals which occur if we really placed in the concert hall. We are simply not able until today to reproduce such acoustic impression. In the home Environment the acoustics of the home cinema mainly determine our perception. Besides, the recording process are causing a lot of wrong cues, which sometimes controverse each other. Above all, the phantom acoustic source disappear between the loudspeakers, if our ear approximating. Describing the complete disaster would overrate this short description, please klick the chapter Headline for more detailed statement.

2. Wave Field Synthesis

2.1 Virtual acoustic source

In contrast regarding Phantom acoustic source, virtual acoustic sources doesn't rely at psychoacoustic effects. Virtual acoustic sources show the same behaviour as real sound sources. That becomes comprehensible by the animation:

In centre of spherical loudspeaker alignment an virtual sound source originate. As easily imaginable, those sound source perceived in middle of the sphere in any case, independent from listener position. In this respect such sound sources fundamentally differ regarding the phantom sound source perception. If we would able for constituting such firmly locatable sound sources at all starting points of direct wave fronts and all of its reflections in the recording room, spatial sound field would indistinguishable from the genuine. In smooth loudspeaker alignments we can establish such virtual sound source from delayed source signals, according the principle of Wave Field Synthesis.

2.2 The WFS- loudspeaker rows

In the late 1980´s on the Delft University of Technology was developed a procedure for constitute such virtual sound sources by a line- array of discrete steerable loudspeakers [1]. That "Wave Field Synthesis” principle rely on Huygens Principle. Christiaan Huygens discovered, each point of a wave front established starting point of an elementary wave. More as 300 years ago, the Dutch Mathematician explains wave diffraction effects by that principle. Huygens Principle is one of the most important cognition in range of physics. In range of acoustics today the knowledge delivers the possibility, restore genuine like sound waves from such elementary waves:

In this animation we consider the holes in baffle, respectively the loudspeakers, such elementary wave starting points. As far as dimension and spacing of the holes remain small regarding wavelength, the sound pressure doesn't differ between both sides of the hole. The superposition of sufficient amount of such elementary wave's completely restores the genuine wave front. All we need is the source signal and the distance regarding each starting point of the elementary wave. Unfortunately, the sound field in the recording only particularly established from the direct wave front. The major fraction of the sound energy include in the reflections. For true spatial audio we cannot set equal the reflection starting points by main source direction, as often done during conventionally recording process. The difference in angle of the first reflections regarding direct wave starting point deliver the most important cues regarding source distance and distances regarding the walls, what imply the room dimensions. The huge amount of later reflections which arrange the reverberation tail are less important regarding the direction but provide information's regarding fine structure and properties from recording room surfaces.

The WFS - loudspeaker alignment is able for establish more as one independent virtual sound source. Its signal content may originate by different sources. Congruent signal content radiate from different source positions fake reflections from arbitrary starting points. The genuine sound field in recording room is established from huge amount of such starting points with same signal content. If we reconstruct all of its positions, the spatial sound field would be recreated from one single, dry recorded mono signal. After all, the recording room doesn't work different. The only difficulty for restore the genuine sound field is appointing all of the starting points of all of the reflections in the recording room.

Wave Field Synthesis provides two different ways in this matter. The more simply method is the model based approach. According the mirror source model the starting points become calculated simply from recording room geometry. The calculated distance of each virtual sound source position regarding each of the Loudspeaker positions determinates runtime and level. The reflection factors of the walls are including in that calculation. Such procedure is practicable for restore direct wave and first reflections in the recording room, but the huge amount of discrete reflections in reverberation tail make impossible the correct reconstruction of the complete sound field by means of the model based approach in practise.

By that reason common practise in the scientific institutes, which refining Berkhout´s idea, the application of the impulse response based approach. In prearrangement of the transmitting process become captured the spatial impulse response of the recording room. In that purpose a line array of microphones arranged in the recording room comparably as the loudspeakers arranged in the playback room. For capture the spatial impulse response, a short impulse induced on the later position of the primary sound source catched from the microphone array. That signal will hit the nearest microphone at first. The align loudspeaker on the same position regarding source will radiate the audio signal ahead all other loudspeakers during playback. The other Microphones in the recording room later strike in turn from the impulse. Thus, the signal convolution in the captured impulse responses of the microphone array will deliver coincident radiation. The appropriate loudspeakers restore the correct direction of the wave front. Though, all reflections in the recording room keep different starting points, what cause its signal will hit the Microphones different in time and sequence. The playback convolution restores that signals accordingly in time and direction. By that reason the impulse based method restore the entire acoustics of the recording room.
Yet isn't possible recording the spatial impulse response on all possible microphone positions in the recording room for all possible positions of the sound source. The measuring results must extrapolate during playback for all different positions. This calculation needs to include all of the mirror source positions. That task is hardly solvable for currently available computing power, especially for moving sound sources. In principle though, the wave field synthesis today has the ability for produce a virtual copy of the genuine sound field. All sound sources and all of its reflections in the recording room may produce virtually at any points inside the horizontal plane of the listener. That's different regarding conventionally audio reproduction procedures, which trying transmitting the whole information in some separate audio channels. The synthesis from dry recorded source signal, in the same manner as the recording room establishes the spatial sound field seems to be only possible way for a true spatial loudspeaker rendition as volume solution. Only such volume solution doesn't capture in narrow sweet spot, the wave field synthesis restores the field, changes of the listener position in the playback room cause the same changes in perception as listener movements in the recording room.

Over and above, Wave Field Synthesis provide the possibility for align the virtual sound source in front of the loudspeaker alignment. In the animation wouldn't appear any difference for delay times and levels, if the virtual source behind or in front of the microphone row. Thus we would perceive the starting point in any case behind the speakers. But becomes the delay times inverted by “Time Mirror Approach”, the loudspeakers produce concave wave fronts. In that case the virtual source appears in the focus point inside the playback area. We can walk around yet in certain degree.

Wave Field Synthesis isn't limited on plane in principle; the procedure would able for restore the sound field in all three room dimensions. But in practice single horizontal loudspeaker rows around the listener are realized until. Such solutions are possible today with hundreds of loudspeakers without unbearable problems. But in home market a broad based breakthrough doesn't seem within reach. Besides the acceptance factor, some additional problems haven't been solved until. The horizontal limitation remains clearly audible, especially in damping environments, which need for suppress the playback room acoustics for such WFS- approach. Other procedures, like Ambisonics or Vector Base Amplitude Panning (VBAP) have shown a really three dimensional reproduction of the sound event is essential. But they are missing the main advantage of the WFS - a large listening area inside the “volume solution”. The Ambisonic and VBAP techniques reduced in more limited subset of the playback rooms.

Besides, the main problem of each reproduction isn't solved by the loudspeaker rows. They cannot solve the playback room impose additionally acoustic behavior above the sound. In order to produce acoustics of recording room, the playback room acoustics must get completely suppressed. Horizontal rows of speaker's doesn't really produce parallel wave fronts, they radiate cylindrical waves. Such wave fronts lose 3 dB of its volume every time the distance is doubled. Since the listener is relatively near the speakers, the increasing volume of the nearby speakers becomes disturbing. Over and above the losen sound energy comes back from the playback room walls in case of insufficient damping.

Because of the remaining problems the realised installations until remain clearly apart from the goal of the indistinguishable reproduction. Most notably seems the reduction onto the horizontal plane and aliasing effects, caused by insufficient amount of elementary waves. But the problems seem solvable in foreseeable time, the first plants of tightly assembled two dimensionally WFS- Loudspeaker fields indicate promising results. Such speaker fields would be applicable for put into practice a WFS- “Holophony” approach, described in the next chapter.

3. Holophony

3.1. The virtual 3D-copy of the sound field

Since the invention of stereophony there have been attempts to improve the spatial reproduction of a sound event with an increasing number of channels. Yet the sound source itself doesn't reproduce any spatial sound field; each arbitrary source may be regarded, at least from a certain distance, as a Mono source. That source isn't radiating equally in all directions and there is no way for an entire spatial sound field to be reproduced. The spatial sound field is mainly the result of the source signal's reflections in the recorded room. This complex reflection pattern would be restored completely if we start from such thought:

If we build a closed cabinet around an ideal place in a concert hall, but perforate it close-packed, the acoustic impression at the listener inside would hardly be disturbed. If each of the holes contained an inward facing loudspeaker, steered from its own microphone on the other side of the wall, the acoustics effects would remain unchanged inside the box. Placed in our living room at home these speakers would provide a perfect copy of the original sound event.

Yet a problem would arise because such a number of discrete channels is hardly transferable. Every speaker radiates a slightly different signal, but when examined more closely we find that the waveform doesn't differ from hole to hole; only the arrival time is different. Therefore all of the speakers' signals are producible from a Mono signal of the sound source if you know its position in the recording room to calculate the delay time regarding each single loudspeaker.

For the spatial reproduction we not only need to restore the direct wave of the sound source, but also all of the reflections in the recorded room must be recovered as well. Since the reflections are singing no other song than the tenor, we can synthesize all of them from the direct source signal if their starting points are known. If we know the position of Tenor in the concert hall and the concert hall's properties, we can calculate the reflecting points. Now we can produce all the loudspeaker signals by a computer synthesis at home from the dry mono signal of the sound source.

However, the spouse acceptance factor would be very poor for a solution which wants to populate all living rooms' walls with loudspeakers. But if we use the reflecting surfaces of the living room, combined with established sound projection principles, a flat loudspeaker screen in front of the listener, possibly behind the picture screen, would be sufficient. But for Holophony the loudspeakers no longer would emulate other loudspeakers, but the source itself including the recorded room's spatial sound field. In the near field of the huge resulting loudspeaker, the playback room's acoustics has very little influence on listening conditions, so the recorded room's acoustics may be recovered in an untreated room.

3.2 Near field solution at“acoustic curtain”

Besides wearing headphones the only solution for avoiding the unwanted impact of living room acoustics is near field reproduction. Two possibilities exist for that. Either the speakers are placed very near at listener or the radiating diaphragm becomes very large. In normal living room environments, the distance separating speakers and listener is around three to four meter.

Dependent from loudspeaker bunching factor and playback room reverberation time the playback room reflections exceed the direct radiation in normal dwellings at less than one meter. For including a three meter distanced listener in near field we would need a anechoic living room or a diameter of the diaphragm exceeding 1, 5 meter. That's impossible for a single speaker, however using the wave field synthesis principle, the loudspeakers work altogether as unit. Such two dimensionally loudspeaker screen perform the known principle of the acoustic curtain. More as a half century bygone such solution was a dream of the inventors, at the WFS principle such approach becomes feasible today:

Each of the membrane excursions in such speaker field is simply calculable in terms of distance regarding the virtual starting point. The bending of the resulting common diaphragm is depend by virtual starting point position and frequency, per eaxmple 440 Hz , virtual source 3 meters behind the screen of 48x27 Loudspeakers:

For decreasing frequence the curvature disappear, all loudspeakers executing common plunger motion. If used two inch distance between the single transducers, that diaphragm would have a size of 2.43m (~8') x 1.38 m (~4.5'); therefore, the listener position include in near field range. Yet spatial aliasing effects would occur, if not provide sufficient amount of loudspeakers. Dependent from ingoing and outgoing angles regarding the radiating surface such effects arise above this frequency:

For instance, a 30 degree angle difference would result in aliasing above 13.5 kHz for two inch spaced speakers. That's an acceptable value; it is known that our perception isn't very sensitive regarding spatial aliasing in this frequency range. But such alignment would cause total amount of 1296 single speakers. Technically such field is feasible today; some alignments already in existence contain comparable numbers of speakers. Actually, the development goes away from the horizontal rows towards “speaking screen”. The straightening effect of such large loudspeaker screen avoid supply of the playback room mirror sources unwantedly by sound energy. That substantially degrade the disturbing influence of playback room acoustics. On the other hand, targeting supply of the first reflection sources in playback room by sound energy open a way for include playback room reflections on purpose. That describe in the next chapter.

3.3 Inclusion of the playback room acoustics in Holophony approach

There are a lot of scientific publications regarding Wave Field Synthesis principle, though by which reason the principle really is a revolution in audio reproduction hardly mentioned until today . In difference regarding all other audio reproduction principles wave field synthesis provide access at each component of the sound event. No other principle would be able for manipulate during playback direct wave, first reflections and reverberation in different manner. All conventionally procedures merge together the components inseparably already during the record.

On account of that advantage Wave Field Synthesis opens unimagined possibilities for eliminate problems of audio reproduction, which was estimated unsolvable until today. For implementing this advantage we have to abandon the prevalent system approach. Traditionally, the transmission chain start at the microphone and ends by the loudspeaker. But mostly significant signal changes arise, if the loudspeakers have done its work. For congruent perception the inclusion of the playback room properties into the system approach is indespensable. The described Holophony approach doesn't aim providing perfect signals in the loudspeakers. The main goal is the signals at the listeners ears at home match the signals on dedicated virtual listener position in the recording room. For pursuit that goal, we need to introduce such dedicated listener position in the recording room and a default listener position in the Playback room. By using this new system design, we no longer need supress the playback room acoustics. Different sound detours and sound level changes in playback room become equalize during synthesis by means of both listener positions set in congruence. The model based approach subtracting additionally detours of the first reflections in the playback room. Thereby the arrival angles and reflection factors between recording and playback room get equalize.

In very simple model we can calculate direct wave and first main wall reflection for that purpose. That's the most significant cues regarding time and direction. If lock sufficient initial time delay gap, all later reflections are less important regarding time and direction. For common model must be known the recording and the playback room geometry. Point of origin in the common geometrical system the both superimposed listener positions. The outer room 3a in the scetch is the recording room, inside depict the playback room 3b. In the centre is the common listener position in recording and playback room, 3i:

The source 3c become restored by the “acoustic curtain” of the frontal speakers nearly perfect. The first reflection mirror source in recording room resides outside that range. We cannot radiate directly. But if the position in range of the loudspeaker fields mirror source in playback room, from item of listener position, we can employ the playback wall reflection in that purpose. Thus, in order to simulate the first reflection correctly, we have to shift the recording room's mirror source position in common system in that range. This is accomplished by a simple rotation in a circular path inside the common coordinate system. Mirroring this position at playback room's geometry deliver the final starting point (4b) coordinate of the virtual source. Calculating the runtimes and levels regarding each speaker is very easy in the common model, because only the distance regarding the respective speaker determines the delay time for the corresponding audio signal.

The animation sketch shows the principle for restore the perceived altitude of recording room in playback room. Corresponding procedure regarding all main surfaces restore all room dimensions which originally perceived in recording room.

3.4 Combination of model based and data based approach

Restore all of the reflections correctly was out of reach during early stage of Wave Field Syntesis. By reason of insufficient computing power was reduced the principle at the horizontal level of the listener. Until today, effective calculation procedures for extrapolation and interpolation of the measured values regarding the given virtual source and loudspeaker positions [3], or field related solution approaches as described in [4], matter of development in the scientific institutes. The high level of effort in this matter underscores the relevance of the Principle of WFS as spatial audio solution. Though, until today the WFS isn't realised in practise in all three room dimensions.

Potentially, the combination of the model based and the data based approach would solve remaining problems: The reverberation contains important cues regarding the recording room properties. The fine structure of the surface, which determines the timbre of the room, becomes reproduced by the reverberation tail. Convolution into the impulse response of the recording room is an approved method for reconstitute the reverberation very authentic. Yet from what direction arrive the wave fronts of the reverberation tail at the listener of subordinate relevance for perception. Also in recording room, the reverberation comes from all possible directions; we cannot pinpoint the origin of second or later reflections. On the other hand, the direct wave front and its first reflections in the recording room hardly determine the timbre, but contain the most important localisation cues regarding source position. By means of optically relationships we have got the skill for determine position of those acoustic sources very accurately. Humble distinctions in arrival time or amplitude of those wave fronts permit its localisation in all three room dimensions.

The sound level of the first reflections often hardly differs regarding direct wave. Thus superposition results in deep comb filter effects. In case time relations are correct, resulting notches in frequency response are  meaningfully cues regarding room impression. But wrong detours result in different pattern regarding hills and notches. The level of the resulting misguiding cues often above 20 dB. But equalising in frequency domain cannot diminish those time domain caused faults as far as all signal components already merge together. Because of the different conditions in perception doesn't meaningful finishing direct wave and its first reflections in same manner during the wave field synthesis, like widespread practise in WFS. The completely impulse response based approach delivers perfect results, but overcharge current available computing power, especially for moved sources in three dimensionally environments.

Perception of reverberation hardly change, if source or listener change its position in the recording room. Yet level changes and direction changes of the direct wave and first reflections result in substantially alter perception. The calculations of those wave fronts feasible much more easily in the model based approach. The following screenshot show a way for combine the model based approach for first reflections with the impulse based method for reverberation: 

Combination of Imp/ Model based WFS; Please klick the picture for enlarge

The model based treatment on left side of sketch via MADI or LAN connected at the Engine part at the right side. Such breakup in computing and engine was proposed by Wolcott in [5]. The interface transmits all audio source signals and all data regarding delay time and Level for each single loudspeaker position. But for those high amount of data values – each speaker needs for each primary source and for each of its first reflections in the recording room a separate delay and level value – according [5] are eight updates per second sufficient for an smooth movement of the source.

Because of directed radiation inside loudspeaker near field the sound waves hardly cause playback room reflections. Such reproduction is able for render possible sound sources very near at the listener. The divergence problem solution according 102006054961A1 already is include in calculation. The model based calculation constitutes really three dimensions. The combination with the impulse response method for the reverberation tail is very practicable in the described procedure. In the example the channels 01 and 02 are reserved for constitute the reverberation. That signal create by convolution of the summary signal from all input sources into the impulse response of the recording room. The model calculation deliver a time value, during the first frames inside those impulse response must become suppressed in order to avoid a double production of first reflections.

Another way would be simply recording the reverberation in a discrete channel with an omni- directionally microphone apart all direct sound sources. During synthesis reverberation level only depended from overall volume. The later reflections come from all directions, what match its spatial distribution in the recording room. Huge advantage of the described proceeding the dispensability of spatial impulse response, recorded by a team of highly qualified technicians in preparation of a wave field synthesis recording. Conventional impulse response, which existent for all interesting environments today sufficient. Together with a crude geometric model from recording room each dry recorded signal can render as would it recorded in most attractive acoustic environments around the globe.

3.5 Compatibility

The Holophony approach convenient for reproducing conventionally recordings. The steerable wave fronts deliver some important advantages in comparison to solitary speakers:

¦ Mono: The speaker field generates parallel wave fronts. Its high direct sound portion and the low remaining influence of the playback room acoustic ensure excellent speech intelligibility.

¦ Focus Mono: Focused wave fronts effectuate high volume nearby the focus point. For other ranges in the playback room the signal hardly audible. This allow different signal contents in common room.

¦ Stereo: The most important advantage of the described Holophony procedure the ability costomize playback room acoustic during reproduction. The directed radiation widely avoid its wrong reflections. On the other Hand, synthesized early reflections can provide large playback room impression. That allay most audible deterioration during conventionally reproduction, caused by fact, the playback room acoustic widely differs regarding acoustics of recording room.

¦ Surround: Virtual sound sources beyond real playback room enlarge sweet spot. The huge effective diaphragm ensure near field for lessen influence of the playback room acoustics. Synthetic virtual sources outside horizontal level of listener provide 3D illusion. Increasing amount of canals for future possible by software update. Faked reflections resize the playback room virtually. And the most important fact: Excelent spouse acceptance factor for the unvisible speakers.

¦ Ambisonic: Grinding in ambisonics decoder suitable approach regarding those very realistic audio reproduction procedure. The number of virtual Loudspeakers at virtual positions may equivalent the number of input canals of the wfs- processor. That makes the loudspeaker screen applicable for high order ambisonics.

¦ Wave Field Synthesis: Suitable recordings for the WFS- procedure deliver a spatial sound field, hardly distinguishable from genuine. For studio productions suitably virtual environments from stored libraries deliver adapted playback environments. The playback room acoustic subtract during playback synthesis, in acoustic matter the listener become placed in recording room.