Author: Jürgen Herre
Background information: Spatial perception
Although spatial perception is far from being fully understood, it is
known that directional localization of sounds depends on the evaluation
of so-called spatial cues by the human auditory system (see e.g. Blauert's
book on spatial hearing [1]), the most important cues being
- interaural level differences (i.e. the differences in levels received by both ears)
- interaural phase differences (i.e. the differences in signal phase received by both ears)
- preservation of fine temporal signal envelope structure
Consequently, the fidelity of the stereo image of
a coded signal depends on the coder's ability to preserve these critical
cues appropriately.
Background information: Joint Stereo Coding
For coding of high quality stereophonic (or multi-channel) audio signals
at low bit rates, joint coding techniques have proven to be extremely
valuable. On one hand they provide mechanisms to account for binaural
psychoacoustic effects, on the other hand the required bit rate for the
stereophonic signals may be reduced significantly below the bit rate for
separate coding of the input channels.
Currently, the most common joint stereo coding techniques are Mid/Side
(M/S) stereo coding [2] and Intensity Stereo coding [3] [4]. While the
first method can account for binaural masking effects and achieve a certain
amount of signal-dependent gain, the intensity stereo method provides
a high potential for bit saving. Coders based on the intensity stereo
principle have been described in the past for stereophonic and multi-channel
coding under various names (e.g. "dynamic crosstalk" [5], "channel
coupling" [6]).
Intensity stereo exploits the fact that the perception of high frequency
sound components (e.g. above 4 kHz) mainly relies on the analysis of their
energy-time envelopes [1] rather than the waveform itself. Thus, it is
assumed sufficient to code the envelope of such a signal instead of its
waveform. This is done by transmitting one common set of spectral coefficients
("carrier signal") that is shared among several audio channels
instead of separate sets for each particular one. In the decoder, the
carrier signal is scaled independently for each signal channel to match
its original average envelope (or signal energy) for the respective coder
frame. The scaling information is calculated and transmitted once for
each group of spectral coefficients (scalefactor band). Effectively, the
stereo image is recreated at the decoder side by a pan-pot-like operation
for each spectral coder band.
Figure 1: Basic principle of intensity stereo coding
As a consequence of the intensity stereo coding / decoding process, all output signals reconstructed from a single carrier are scaled versions of each other, i.e. they have the same envelope fine structure for the duration of the coded block (e.g. 10-20 ms). This does not present a major problem for stationary signals or signals having similar envelope fine structures in the intensity stereo coded channels.
For transient signals with dissimilar envelopes in different channels, however, the original distribution of the envelope onsets between the coded channels cannot be recovered. Figures 2 and 3 show an example for this constellation: In a stereophonic recording of an applauding audience, the individual envelopes will be very different in the right and left channel due to the distinct clapping events happening at different times in both channels (see Figure 2 for left and right channel envelopes).
Figure 2: High frequency envelope structures of left and right original channel signals. Excerpt from "applause" item
After the intensity stereo encoding / decoding process, the fine time structure of the signals is mostly the same in both channels as can be seen in Figure 3. In particular, there is a structural "cross-talk" between the channels, such that perceptually important signal onsets propagate to the other opposite channel (e.g. L->R at 15 ms, R->L at 47 ms, L->R at 57 ms, L->R at 70 ms).
Figure 2: High frequency envelope structures of left and right channel signals after Intensity Stereo encoding/decoding (red). Envelopes of the original signals are shown in dotted yellow lines. Excerpt from "applause" item
Consequently, the stereo image quality of the intensity stereo coded / decoded signal will decrease significantly in such cases. The spatial impression tends to narrow down and the perceived stereo image collapses into the center position. For critical signals, like the applauding audience example, the achieved quality cannot be considered as acceptable anymore.
Sound examples
The following sound excerpts illustrate the discussed effects:
 |
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The provided example sound files demonstrate the original applause recording as well as three sound examples with increasing deficiencies in stereo imaging quality, as would be produced by a coder without a proper control of the intensity stereo coding mechanism. Please observe the increasing loss of people applauding in the outer left and outer right seats as well as the overall lack of spatial impression and distinct reproduction of the single clap events.
References:[1] J. Blauert, "Spatial Hearing", MIT Press, 1983
[2] J. D. Johnston, A. J. Ferreira: "Sum-Difference Stereo Transform Coding", IEEE ICASSP 1992, pp. 569-571
[3] R.G.v.d. Waal, R.N.J. Veldhuis, "Subband Coding of Stereophonic Digital Audio Signals", IEEE ICASSP 1991, pp. 3601 - 3604
[4] J. Herre, K. Brandenburg, D. Lederer, "Intensity Stereo Coding", 96th AES Convention, Amsterdam 1994, Preprint #3799
[5] G. Stoll, G. Theile, S. Nielsen, A. Silzle, M. Link,
R. Sedlmayer, A. Breford, "Extension of ISO/MPEG-Audio Layer II to
Multi-Channel Coding: The Future Standard for Broadcasting, Telecommunication,
and Multimedia Applications", presented at the 94th AES Convention,
Berlin 1994, Preprint # 3550
[5] Mark Davis, "The AC-3 Multichannel Coder",
95th AES Convention, New York October 1993, Preprint # 3774