SVS by example

Singing voice synthesis (SVS) methods that I have tried, with audio examples
2026-09-01    4 minute read

There are lots of different methods of writing a singing voice synthesizer (SVS), and they each have their benefits and drawbacks.

To compare them, I'm using Ievan Polkka as a benchmark because it's:

Also if I hear Daisy Bell as an SVS demo one more time I'm going to become the Joker

Source-filter

Source-filter synthesis is a sort of physical modelling where the vocal cords are replaced with a buzzy waveform (usually an impulse or a sawtooth wave) and the vocal tract (throat, mouth, tongue, lips) is replaced with a filter that resonates at different freqeuencies depending on the target "shape".

In many cases, the source and the filter are largely independent, and can be mixed and matched.

Formant filter

Examples:

TODO: example

Tube waveguide filter

Examples:

  • Pink Trombone

TODO: example

Source models

Waveforms

  • cosine
  • impulse train
  • sawtooth
  • TODO examples of each

In meatspace, the wave propagation in the vocal tract feeds back to the vocal folds, and therefore affects the pitch and timbre of the source. However, this interaction can be simplified to parametric controls in the source, like changes to its pitch or waveshape.

TODO example of parametric changes

When I said earlier that any source could be combined with any filter model, I lied. The exception to this is using a physical model for the vocal folds, because it needs to know the pressure at the base of the vocal tract, so only filters that support that (like a tube waveguide) can be used with that particular source model.

Sample-based

These involve recording utterances in meatspace, analyzing them, and using those analyses to generate synthesized audio. In some cases, sample-based SVS models can have a source-filter architecture, but it's much less common, so I've branched it out into a separate category.

Examples:

  • Vocaloid
  • UTAU
  • Animal Crossing Animalese & K.K. Slider

Because sample-based models are more end-to-end (compared to source-filter models, which are more modular), I found them less useful for my research into different forms of phonation and articulation. That being said, they are generally more intelligible and higher fidelity.

when you do utterance-based synthesis, the nature of the recordings are of utmost importance. youd think you want natural, fluid pronunciation in order to make the final result sound smooth, but you actually want emphasized consonants and very resonant vowels because the formants get a little muddied up in the conversion process.

Time-domain Concatenation

super simple concept. get a sample of every syllable and just stitch them together. Basically YouTube poop.

TODO: example

Spectral methods

Articulation

articulation of a vocoder (short for vocal encoder/decoder), there are many levels of abstraction. each abstraction aims to make less common things difficult or impossible, and more common things easy.


  • get f0 with large fft buffer
  • find harmonicity by using autocorrelation output properties
    • probably range between peaks and valleys? or area of later half minus area of earlier half?
  • high pass original signal
  • use zero crossings from high passed signals
    • calculate area between zero crossings and merge when areas are very very small
    • create a wavetable using zero crossing segmentation resize and center
    • normalize each wave
  • rubato resample to fft size
  • use resized fft to get formants

fft formants vs filter formants (people under a blanket analogy) finding formants