Subtitle tracks in files produced by MakeMKV contain errors

Please post here for issues related to Blu-ray discs
Post Reply
xaviour
Posts: 58
Joined: Fri Mar 25, 2011 2:02 pm

Subtitle tracks in files produced by MakeMKV contain errors

Post by xaviour »

The Matroska Foundation provides a couples of command line tools to check compliance of MKV files ("mkvalidator") and optimize them ("mkclean"). When used on files
created by makemkv, "mkvalidator" outputs a lot of warnings and error. For example:

Code: Select all

WRN0C0: First Block for video track #1 in Cluster at 32359660555 is not a keyframe
WRN0C0: First Block for video track #1 in Cluster at 32363072132 is not a keyframe
WRN0C0: First Block for video track #1 in Cluster at 32368701614 is not a keyframe
[...]
ERR0B1: SimpleBlock at 212916 track #3 is not a keyframe
ERR0B1: SimpleBlock at 458327698 track #3 is not a keyframe
[...]
First of all, it would be nice if MakeMKV would be better at clustering frames. It would save an lengthly optimization phase. Isn't there a minimum number of I frames per seconds for H.264 tracks in a bluray?

Then, in the particular example I am using, the third track contains PGS subtitles. Aren't subtitles always encoded independently from previous frames and as such always matroska keyframes?
xaviour
Posts: 58
Joined: Fri Mar 25, 2011 2:02 pm

Re: Subtitle tracks in files produced by MakeMKV contain err

Post by xaviour »

The disc used as example is:
Star Wars I: The Phantom Menace Region A, B, C
from The Star Wars The Complete Saga bluray set.

A testdump is available upon request.
Attachments
title01.txt
Output of mkvalidator v0.3.7.
(706.31 KiB) Downloaded 446 times
MakeMKV_log.txt
Log file produced while ripping the Phantom Menace.
(3.53 KiB) Downloaded 479 times
mike admin
Posts: 4065
Joined: Wed Nov 26, 2008 2:26 am
Contact:

Re: Subtitle tracks in files produced by MakeMKV contain err

Post by mike admin »

xaviour wrote: First of all, it would be nice if MakeMKV would be better at clustering frames. It would save an lengthly optimization phase. Isn't there a minimum number of I frames per seconds for H.264 tracks in a bluray?
Generally, no - on some files clusters put only on I-frame boundaries will be too large and produced file will have seek issues. Can you remux this file with mkvmerge in any way that clusters start only on I-frames with default 0.5sec cues?
xaviour wrote: Then, in the particular example I am using, the third track contains PGS subtitles. Aren't subtitles always encoded independently from previous frames and as such always matroska keyframes?
Not for PGS subtitles. PGS subtitles come as pair of frames, first "show subtitles" frame followed by "hide subtitles" frame. The first one is a key frame, the second is not. As far as I know, there is no hard requirement for simple block to be always a key frame. The current logic in mkvalidator (all non-video simple blocks must be keyframes) is apparently wrong.
xaviour
Posts: 58
Joined: Fri Mar 25, 2011 2:02 pm

Re: Subtitle tracks in files produced by MakeMKV contain err

Post by xaviour »

xaviour wrote:Isn't there a minimum number of I frames per seconds for H.264 tracks in a bluray?
mike admin wrote:Generally, no
This statement is inconsistent with this post and the output of the following command:

Code: Select all

mkvinfo -v Videos/The_Phantom_Menace/title01.mkv | grep key | grep "track number 1" > keyframes.txt
The result clearly shows a keyframe at least every second. I attached the result to this post. However, I guess you have analyzed more blurays than I have so you are probably right in the general case.
mike admin wrote:on some files clusters put only on I-frame boundaries will be too large and produced file will have seek issues.
I do not understand this argument. If clusters start with an IDR frame and the position of start of the cluster is stored in the cue table, then to jump to a random frame, the player needs first to look up the clusters in which the frame is located. Then it needs to decode all the frames preceding the desired frame. In 24p with 1 IDR per second, that's up to 23 frames. This takes between 1/3 second (for modern chips decoding at 60 fps) and 1 second. If clusters last for 1/2 second then either the desired frame is in the cluster and after an IDR frame, in which case it can be decoded fast or it is not (no IDR in cluster or IDR follows the desired frame) in which case the frame cannot be decoded.

So I really think that the warning from mkvalidator makes sense. To help with seeking, clusters should start with a keyframe. Having 0.5 sec clusters is not going to fix the seeking latency created by a video encoded with 1 keyframe per second.
mike admin wrote:Can you remux this file with mkvmerge in any way that clusters start only on I-frames with default 0.5sec cues?
Sure. Tried with mkvmerge v5.0.1, mkclean v0.8.6 and ffmpeg 0.7.2-4:0.7.2-1ubuntu1. Only ffmpeg generates clusters starting with video keyframes.
Also all 3 tools generate files that contain errors as far as mkvalidator is concerned.
mike admin wrote:Not for PGS subtitles. PGS subtitles come as pair of frames, first "show subtitles" frame followed by "hide subtitles" frame. The first one is a key frame, the second is not. As far as I know, there is no hard requirement for simple block to be always a key frame. The current logic in mkvalidator (all non-video simple blocks must be keyframes) is apparently wrong.
I do not know the specifics of PGS subtitles. But if the "hide subtitles" frame can be decoded independently of the "show subtitles" frame then it is a keyframe. If the PGS decoder is reset and fed a "hide subtitles", nothing wrong is going to happen. It will just clear its already cleared frame buffer. That would explain mkvalidator policy.

Either way, I just care about MakeMKV and mkvalidator to agree with each other. If mkvalidator is wrong then either I need arguments to report the bug or you can report it yourself.
Attachments
keyframe.txt
Output of mkvinfo showing that keyframes are spaced every second.
(767.97 KiB) Downloaded 438 times
mike admin
Posts: 4065
Joined: Wed Nov 26, 2008 2:26 am
Contact:

Re: Subtitle tracks in files produced by MakeMKV contain err

Post by mike admin »

For PGS you are right - this has to be looked at. For clustering, LOTS of MKV decoders will fail if cues are less than 0.5 sec - see what happened when MakeMKV (briefly) switched to 1 sec cues - http://www.makemkv.com/forum2/viewtopic.php?f=1&t=3627
xaviour
Posts: 58
Joined: Fri Mar 25, 2011 2:02 pm

Re: Subtitle tracks in files produced by MakeMKV contain err

Post by xaviour »

mike admin wrote:For PGS you are right - this has to be looked at.
Glad to hear it.
mike admin wrote:For clustering, LOTS of MKV decoders will fail if cues are less than 0.5 sec
You mean, more than 0.5 sec right? Moreover, what I am suggesting is:
- Create a new clusters every IDR frame which should end up being about every second.
- For each cluster, have a cue entry pointing to the start of a cluster (the IDR frame). Do not index frame within a cluster (provided that Mastroska allow
such a thing).
mike admin wrote:see what happened when MakeMKV (briefly) switched to 1 sec cues - http://www.makemkv.com/forum2/viewtopic.php?f=1&t=3627
Thanks you for the reference. This looks like bugs in the players.

Is there a way for me to generate files with 1 second cues so that I can test it against a bunch of software and hardware players?
I might be able to convince the authors to fix the bugs.
xaviour
Posts: 58
Joined: Fri Mar 25, 2011 2:02 pm

Re: Subtitle tracks in files produced by MakeMKV contain err

Post by xaviour »

The problem is still present with 1.6.16.
xaviour
Posts: 58
Joined: Fri Mar 25, 2011 2:02 pm

Re: Subtitle tracks in files produced by MakeMKV contain err

Post by xaviour »

mike admin wrote:For PGS you are right - this has to be looked at. For clustering, LOTS of MKV decoders will fail if cues are less than 0.5 sec - see what happened when MakeMKV (briefly) switched to 1 sec cues - http://www.makemkv.com/forum2/viewtopic.php?f=1&t=3627
There is a message in the Matroska Development mailing list archive that might explain the problem. Maybe the solution is to trade total file size against small clusters and fast seeking.

Some clusters would start with a keyframe and would be indexed while others would not have any keyframe in them and would not be indexed. An updated Matroska verification tool would then ignore clusters without a keyframe and warn about clusters that contain a keyframe which is not the first frame.
xaviour
Posts: 58
Joined: Fri Mar 25, 2011 2:02 pm

Re: Subtitle tracks in files produced by MakeMKV contain err

Post by xaviour »

The problem is still present with 1.7.0.
xaviour
Posts: 58
Joined: Fri Mar 25, 2011 2:02 pm

Re: Subtitle tracks in files produced by MakeMKV contain err

Post by xaviour »

The problem is still present with 1.7.1.
xaviour
Posts: 58
Joined: Fri Mar 25, 2011 2:02 pm

Re: Subtitle tracks in files produced by MakeMKV contain err

Post by xaviour »

The problem is still present with 1.7.2.
xaviour
Posts: 58
Joined: Fri Mar 25, 2011 2:02 pm

Re: Subtitle tracks in files produced by MakeMKV contain err

Post by xaviour »

The problem is still present with 1.7.3.
xaviour
Posts: 58
Joined: Fri Mar 25, 2011 2:02 pm

Re: Subtitle tracks in files produced by MakeMKV contain err

Post by xaviour »

The problem has been fixed in version 1.7.4.

Moreover mkvalidator v0.4.0 has been released and does not output a warning for clusters
without a keyframe. It display a warning only for clusters that contain a keyframe which is
not the first frame of the cluster.
Post Reply