PadhaikoAll subjects

Frequency Domain and Non-Linear Effects (part 2: 4.4–4.6)

Audio Processing (ENCT 389)

Open in the Padhaiko app

4.4 Basis of non-linear effects

Linear time-invariant (LTI) filters can only scale and phase-shift frequencies already present at their input; they can never create new frequencies. Non-linear audio effects—such as overdrive, tube saturation, fuzz, hard clipping, and wavefolding—intentionally violate superposition by applying a non-linear transfer characteristic (waveshaping) or signal-dependent gain, enriching the spectrum with new harmonics.

Definition

Memoryless Non-Linear Waveshaping and Harmonic Generation

A memoryless non-linear processor maps each instantaneous sample through a static function , which around can be expanded as a power series:

When excited by a pure sinusoid :

When two tones enter simultaneously, cross-terms produce intermodulation distortion (IMD) at sum and difference frequencies .

Common Non-Linear Characteristics and Aliasing Control

Common Audio Waveshaping Transfer Functions
EffectTransfer Function Spectral & Sonic Character
Hard ClippingSharp corners generate dense, harsh high-order odd harmonics (transistor fuzz)
Soft Clipping ()Smooth saturation knee; rapid harmonic roll-off (warm analog tape / tube feel)
Cubic Soft Clipper for , otherwiseStrictly band-limited to rd harmonic before clamping; smooth overdrive
Asymmetric Bias / Rectifier or Introduces both even () and odd () harmonics (triode warmth)

4.5 Dynamic range control: Envelop follow, compressor and limiting, expansion and gating, ADSR, modulation effect

Dynamic range control automatically adjusts the gain of an audio signal in response to its own changing amplitude envelope: . Because gain depends on signal level, dynamic processors are inherently non-linear and time-varying.

1. Envelope Follower (Level Detection)

An envelope follower extracts a smooth, slowly varying amplitude contour from the rapid oscillation of using either peak detection () or RMS power detection (), followed by a one-pole leaky integrator with separate attack () and release () time constants:

A short attack lets the detector catch sudden drum transients quickly; a longer release lets gain recover smoothly without low-frequency waveform distortion ("pumping").

2. Compressor and Limiter

A compressor reduces dynamic range by attenuating peaks that rise above a Threshold () according to a compression Ratio ():

Definition

Static Gain Curve of a Hard-Knee Compressor

Let be the detected level in . For threshold and ratio :

The applied gain reduction in is:

Adding Make-up Gain () afterward raises the compressed track's average perceived loudness. A Soft Knee smoothly curves the transition around .

A limiter is a compressor with a very high ratio (); a brickwall limiter sets ( for all ) with fast attack and look-ahead delay so output peaks never exceed (preventing DAC inter-sample clipping in mastering).

3. Expansion and Noise Gating

While compressors act above threshold to shrink dynamic range, expanders and gates act below threshold () to widen dynamic range and suppress background noise:

gently pushing room noise or microphone bleed downward during quiet pauses.

4. ADSR Envelope Generator

Used in synthesizers and samplers, an ADSR generator shapes a note's amplitude (or filter cutoff) over four sequential stages:

  1. Attack (): Time for level to rise from to peak () upon Note-On.
  2. Decay (): Time to fall from peak down to the Sustain level.
  3. Sustain (): Steady level () held continuously as long as the key remains pressed.
  4. Release (): Time for level to fade from Sustain back to after Note-Off.

5. Modulation Effects

Modulation effects vary amplitude, delay time, or all-pass filter phase periodically using a Low-Frequency Oscillator (LFO) (typically ):

Core Audio Modulation Effects
EffectModulated ParameterDelay Range / TopologyCharacteristic Sound
TremoloGain / AmplitudeNo delay ()Rhythmic volume pulsing
VibratoDelay time (Doppler pitch shift) wet short modulated delay ()Periodic pitch wobble without comb filtering
ChorusDelay time + dry mix modulated delay summed with dryRich ensemble thickening (simulates multiple players)
FlangerVery short delay + feedback + dry mix modulated delay with feedbackSweeping harmonic comb-filter "jet plane" notches
PhaserAll-pass filter center frequenciesCascade of all-pass stages + dry mixSmooth non-uniformly spaced sweeping notches

4.6 Time segment process: Time scratching and pitch shifting

Modifying an audio recording's playback speed/duration independently of its pitch—or scrubbing it interactively like a vinyl turntable—requires segmenting and manipulating time frames.

1. Time-Scale Modification (Time-Stretching)

Simply changing sample playback rate changes duration and pitch together (speeding up halves duration and raises pitch by octave). Time-stretching alters duration by stretch factor while keeping pitch fixed:

2. Time Scratching (Turntable / Varispeed Scrubbing)

Time scratching emulates physically manipulating a vinyl record under a stylus. Rather than maintaining constant pitch, it applies time-varying fractional resampling at a user- or gesture-controlled read pointer :

where instantaneous playback speed is :

Digital DJ controllers compute at fractional sample positions using cubic / sinc interpolation to avoid harsh aliasing noise.

3. Pitch Shifting

Pitch shifting scales all frequencies by factor (for semitones) while keeping total duration unchanged:

  1. Time-Stretch + Resample Method: First time-stretch the signal by factor (leaving pitch unchanged, duration ), then resample by factor (interpolating samples at step ). Duration returns to while every frequency scales by .
  2. Direct Frequency-Domain / PSOLA Pitch Shifting: In PSOLA, keep output grain timestamps fixed (preserving duration) while spacing pitch-period grains closer () or farther apart ().
  3. Formant Preservation: Simply scaling all frequencies by also shifts vocal tract formants, creating unnatural "chipmunk" () or "monster" () voices. Dividing out the original spectral envelope (via LPC or cepstral liftering), shifting the fine harmonic spectrum by , and re-applying the original spectral envelope preserves natural speaker identity.

Worked Examples

Example 4.4

4.4

Consider the cubic soft-clipping waveshaper ().

  1. Evaluate for an input sample .
  2. If a pure cosine passes through the purely cubic distortion term , find the frequencies and amplitudes of all output harmonics.
  3. At , will the rd harmonic alias if ? What if ?

Evaluating the waveshaper at :

Harmonic expansion for :

Thus the output contains (amplitude ) and (amplitude ).

Aliasing check at ():

  • For : no aliasing.
  • For : aliases to (preventable by or oversampling).
Answer:
  1. .
  2. Harmonics: with amplitude , and with amplitude .
  3. does not alias (); aliases its rd harmonic to .
Example 4.5

4.5

A hard-knee digital audio compressor has Threshold , Ratio , Attack time , and Make-up Gain , operating at .

  1. When an audio signal with detected level arrives (steady state), calculate the compressed output level before make-up gain, the gain reduction , and the final output level after make-up gain.
  2. Compute the one-pole envelope follower smoothing coefficient for .

Output level before make-up gain and gain reduction: Since , the overshoot is :

Final output level with make-up gain ():

Attack smoothing coefficient at ():

Answer:
  1. Compressed level ; gain reduction ; final output after make-up gain .
  2. Attack coefficient .
Example 4.6

4.6

A guitar track sampled at ( duration, samples) is pitch-shifted up by semitones (a musical perfect fifth) without changing its duration, using the time-stretch + resample pipeline with analysis hop samples.

  1. Compute the pitch-shift frequency factor .
  2. Find the required phase-vocoder synthesis hop (rounded to nearest integer) for the intermediate time-stretch stage.
  3. Explain how the subsequent resampling stage restores the original duration while achieving the semitone pitch shift.

Pitch factor for semitones:

Phase-vocoder synthesis hop : To raise pitch by factor without changing duration, we first time-stretch the audio (at constant pitch) by factor :

This lengthens the audio to ( samples) while keeping pitch unchanged.

Resampling back to original duration: Finally, resample (speed up) the stretched signal by stepping through it at increment input samples per output sample (using cubic/sinc interpolation) and play back at :

  • All frequencies scale up by ( semitones).
  • Total output samples return to samples ().
Answer:
  1. Pitch factor .
  2. Synthesis hop samples.
  3. Time-stretching by at fixed pitch and then resampling by scales all pitches up by semitones while restoring the exact duration.

Exercises

Practice Exercises — Non-Linear Effects, Dynamics & Time-Pitch Processing

Question 1

Distinguish between harmonic distortion and intermodulation distortion (IMD) in non-linear audio processing. If two tones and pass through , list all new frequencies generated in the output. Also state why oversampling is needed in digital distortion effects.

Harmonic distortion vs. IMD:

  • Harmonic distortion consists of integer multiples () of each individual input frequency.
  • Intermodulation distortion (IMD) consists of sum and difference frequencies () created when two or more tones interact inside a non-linearity; IMD frequencies are generally inharmonic and sound harsh.

Expanding for :

New frequencies generated (besides DC and originals ):

  • 2nd harmonics: , .
  • IMD products: , .

Why oversampling is used: High-order non-linearities create harmonics above that fold back (alias) as non-musical grit across the audible band. Raising the internal sample rate by provides Nyquist headroom so generated harmonics can be low-pass filtered before downsampling.

Answer:

Harmonic distortion produces multiples , whereas IMD produces cross-products . For and , adds DC (), , , , and . Oversampling prevents upper distortion harmonics from aliasing back into the audible spectrum.

Question 2
  1. A compressor has Threshold and Ratio . Find the output level and gain reduction for an input signal at .
  2. A downward expander has Threshold and Expansion Ratio . Find the output level for a quiet background signal at .

Compressor calculation ():

Downward expander calculation (): Distance below threshold is . Multiplying by :

Answer:
  1. Compressor: , gain reduction .
  2. Downward expander: .
Question 3

Compare Chorus, Flanger, and Phaser modulation effects in terms of signal processing topology, delay/filter parameters, and audible spectral effect.

Topology and parameter comparison:

  • Chorus: Mixes dry signal with an LFO-modulated delay of (usually without strong feedback), creating pitch/time micro-detuning that sounds like multiple instruments playing in unison.
  • Flanger: Mixes dry signal with a much shorter LFO-modulated delay of plus feedback, producing evenly spaced harmonic comb-filter notches () that sweep up and down like a jet plane.
  • Phaser: Mixes dry signal with a cascade of all-pass filters whose notch frequencies are swept by an LFO, creating fewer, unevenly spaced notches for a smoother, vocal-like swooshing texture.
Answer:

Chorus uses modulated delays for ensemble thickening; Flanger uses modulated delays with feedback for sweeping harmonic comb notches; Phaser cascades LFO-modulated all-pass filters to produce sweeping non-uniform phase notches.

Question 4

Explain the four stages of an ADSR envelope generator and describe how an envelope follower uses separate attack and release time constants in dynamic range processors.

ADSR stages:

  • Attack: Rise time from to peak upon key press (Note-On).
  • Decay: Fall time from peak down to the Sustain plateau.
  • Sustain: Constant amplitude level maintained while the key is held down.
  • Release: Fade-out time from Sustain back to after key release (Note-Off).

Attack and release in an envelope follower: An envelope follower tracks or via a one-pole filter that switches between when the input rises above the current envelope (Attack) and when the input falls (Release). Fast attack catches sudden peaks before clipping, while slower release prevents gain from bouncing inside individual low-frequency cycles.

Answer:

ADSR shapes synthesized notes through Attack, Decay, Sustain level, and Release stages; an envelope follower smooths rectified audio with a fast attack coefficient to catch rising transients and a slower release coefficient for distortion-free gain recovery.

Question 5
  1. Distinguish between time-stretching (WSOLA / Phase Vocoder) and time scratching (turntable scrubbing) in terms of pitch behavior.
  2. A vocal sample is shifted down by semitones at constant duration using analysis hop . Compute the pitch factor and the intermediate time-stretch synthesis hop , and explain why formant preservation is important for vocals.

Time-stretching vs. time scratching:

  • Time-stretching changes duration independently while preserving original pitch (using WSOLA phase-aligned overlap-add or phase-vocoder phase propagation).
  • Time scratching varies the sample read-pointer speed dynamically (forward, stopped, or reverse), causing pitch to vary proportionally with instantaneous scrubbing speed.

Pitch factor and synthesis hop for semitones:

(The signal is shortened by at fixed pitch, then resampled at step to restore duration while lowering frequencies by semitones.)

Why formant preservation matters for vocals: Human vowel identity and perceived vocal tract size depend on fixed resonant formant frequencies (). Scaling all frequencies down by shifts formants downward too, creating an unnatural giant/monster timbre unless the original spectral envelope is preserved.

Answer:
  1. Time-stretching alters duration at constant pitch, whereas time scratching couples speed and pitch dynamically like a turntable.
  2. , samples; formant preservation keeps the vocal tract's spectral envelope stationary so shifted vocals still sound natural.
← Frequency Domain and Non-Linear Effects (part 1: 4.1–4.3)Music Information Retrieval →