Table of Contents

Installing codecs

VisualSubSync is built around its waveform audio display. For it to work correctly you need, if it's not already done, to install an additional piece of software called a codec.

1) Video file structure

Let's start by the beginning: the video file. Most video file have the following data layout:

Video file layout

The yellow part is called the container, it's the glue between the video and the audio track.
It contains information needed to play the video and the audio in a synchronized way.
It also contains more general information, like the video title, the duration, or the language of the audio track for example.
To name a few container format, there is AVI, MKV (Matroska) or MP4.

The blue and orange parts are blocks of video and audio data. As you can see, those data are interleaved in the file.
You have one piece of video, then for these piece of video you have the corresponding audio data.
This way you can read the file in a sequential way, without reading one piece of video there
and then having to seek to another position to read the audio. It wouldn't be possible to play a video file from a DVD if it wasn't stored this way.

2) Playback

Now let's see how playback works in VSS. It's based on a Windows software component called DirectShow.
The following diagram illustrates what happen behind the scene:

Video file playback

The playback process is divided into different software blocks called filters. Each filter has a small task to do.
The source filter read the data from the media. This raw data need to be sorted before processing, that's what the splitter filter do.
Once data are sorted, those can be decompressed by the decoder filter.
And finally the decompressed data are rendered to the screen or to speakers by renderer filters.

3) So what's a codec ?

What is usually called a codec, is the decoder filter, and it's what you need to install.
You also need to install a splitter filter if your file is in a container like Matroska for example.
Even if Windows already come with a set of filters, it doesn't support everything out of the box,
and also some Windows filters don't play nice with third party applications.

4) Getting information about a video file

To know what's inside your video file you can use MediaInfo

For example, we can see that the file below is using the AVI container, the video track is in DivX format and the audio track is in AC-3 format.

An AVI file in MediaInfo

Let's try another file. This one is using the Matroska container (MKV), the video track is in AVC format and the audio track is in AAC format.

An MKV file in MediaInfo

You can also use GSpot as an alternative to MediaInfo.

5) What to install ?

Here is a list of codecs to install for most common formats:

Note that by installing Haali Media Splitter and ffdshow you cover most usages.

6) ffdshow configuration

ffdshow can be seen as an all in one codec and so it's configuration is quite extended. To change the configuration you will find the following icons in your start menu:

ffdshow start menu icons

Here is the most important part of the video decoder configuration:

ffdshow video decoder configuration

For each format, you can select the decoder type or even disable it by clicking in the Decoder column.

7) Last resort : re-encoding

If you still can't play your file, or can't find any available codec for it, there is a last resort solution: re-encoding your file.
Here we will use ffmpeg to do that, it has the big advantage of being available as a standalone executable so it's very stable.
You can download a Windows version here: http://ffmpeg.arrozcru.org/autobuilds

The following command will extract the audio track to a WAV file, reducing the number of channels to 1:
ffmpeg -i myvideo.avi -ac 1 myvideo.wav

And to re-encode a video in an AVI container with video in MPEG4-ASP (equivalent to Xvid or Divx) and audio uncompressed:
ffmpeg -i myvideo.mp4 -vcodec mpeg4 -b 2000000 -acodec pcm_s16le -ac 1 mynewvideo.avi

8) Known problems and workaround

This problem usually happen on Windows 7 (and later) when decoding AAC.
It's seems there is a limitation in the Microsoft audio decoder when used with third party program like VSS.
To fix the problem you can use Win7DSFilterTweaker to set another audio decoder (ffdshow) as the preferred decoder.