Installing codecs

VisualSubSync is built around its waveform audio display. For it to work correctly you need, if it's not already done, to install an additional piece of software called a codec.

Let's start by the beginning: the video file. Most video file have the following data layout:

The yellow part is called the container, it's the glue between the video and the audio track.
It contains information needed to play the video and the audio in a synchronized way.
It also contains more general information, like the video title, the duration, or the language of the audio track for example.
To name a few container format, there is AVI, MKV (Matroska) or MP4.

The blue and orange parts are blocks of video and audio data. As you can see, those data are interleaved in the file.
You have one piece of video, then for these piece of video you have the corresponding audio data.
This way you can read the file in a sequential way, without reading one piece of video there
and then having to seek to another position to read the audio. It wouldn't be possible to play a video file from a DVD if it wasn't stored this way.

Now let's see how playback works in VSS. It's based on a Windows software component called DirectShow.
The following diagram illustrates what happen behind the scene:

The playback process is divided into different software blocks called filters. Each filter has a small task to do.
The source filter read the data from the media. This raw data need to be sorted before processing, that's what the splitter filter do.
Once data are sorted, those can be decompressed by the decoder filter.
And finally the decompressed data are rendered to the screen or to speakers by renderer filters.

What is usually called a codec, is the decoder filter, and it's what you need to install.
You also need to install a splitter filter if your file is in a container like Matroska for example.
Even if Windows already come with a set of filters, it doesn't support everything out of the box,
and also some Windows filters don't play nice with third party applications.

To know what's inside your video file you can use MediaInfo

For example, we can see that the file below is using the AVI container, the video track is in DivX format and the audio track is in AC-3 format.

Let's try another file. This one is using the Matroska container (MKV), the video track is in AVC format and the audio track is in AAC format.

You can also use GSpot as an alternative to MediaInfo.

Here is a list of codecs to install for most common formats:

• Containers:
  • AVI, WMV: included in Windows
  • MKV, MP4, OGG/OGM: Haali Media Splitter
  • FLV: FLVSplitter
  • RM: Real Alternative
  • MOV: Quicktime Alternative

• Video:
  • DivX, Xvid, MPEG4 ASP: ffdshow
  • AVC, H264, X264: ffdshow

• Audio:
  • MP3: included in Windows or ffdshow
  • AC-3: AC3Filter or ffdshow
  • Vorbis: ffdshow
  • AAC: ffdshow

• Subtitles:
  • SRT, SSA/ASS: DirectVobSub/VSFilter

Note that by installing Haali Media Splitter and ffdshow you cover most usages.

ffdshow can be seen as an all in one codec and so it's configuration is quite extended. To change the configuration you will find the following icons in your start menu:

Here is the most important part of the video decoder configuration:

For each format, you can select the decoder type or even disable it by clicking in the Decoder column.

If you still can't play your file, or can't find any available codec for it, there is a last resort solution: re-encoding your file.
Here we will use ffmpeg to do that, it has the big advantage of being available as a standalone executable so it's very stable.
You can download a Windows version here: http://ffmpeg.arrozcru.org/autobuilds

The following command will extract the audio track to a WAV file, reducing the number of channels to 1:
ffmpeg -i myvideo.avi -ac 1 myvideo.wav

And to re-encode a video in an AVI container with video in MPEG4-ASP (equivalent to Xvid or Divx) and audio uncompressed:
ffmpeg -i myvideo.mp4 -vcodec mpeg4 -b 2000000 -acodec pcm_s16le -ac 1 mynewvideo.avi

• VSS display an incomplete audio graph.

This problem usually happen on Windows 7 (and later) when decoding AAC.
It's seems there is a limitation in the Microsoft audio decoder when used with third party program like VSS.
To fix the problem you can use Win7DSFilterTweaker to set another audio decoder (ffdshow) as the preferred decoder.