More mencoder for the iPhone; AppleTV coming soon.

I posted previously about my process for transcoding video for the iPhone with mencoder. In addition to the base write-up, I started encapsulating that information into a python script to automate some of the settings. It's quite simple right now, though it does have some basic options for different source types (progressive DVD vs. telecined, etc).

What you'll see in there as well is the beginning of an attempt to define a target profile for the AppleTV. What's in svn isn't ready for prime time yet for appletv encoding (should be fine for iphone), though I've got that coming. The main challenge was the fact that, as mencoder kindly tells you, muxing the output with libavformat (such as used for mp4 output) is broken when using B-frames. The result is a jittering in the video when played back with something other than mplayer.

The traditional solution to the problem is to transcode to AVI, dump the raw video and audio streams with mencoder, then re-mux them with something like MP4Box, like so:

mencoder -ovc copy -oac copy -nosound -of rawvideo input.avi -o video.h264
mencoder -ovc copy -oac copy -of rawaudio input.avi -o audio.aac
MP4Box -new remuxed.mp4 -add video.h264 -add audio.aac

The problem was, this wasn't working for me; the raw h.264 and aac bitstreams dumped by mencoder appeared to be invalid. If I tried to mux just the h.264, MP4Box gave me "BitStream not Compliant". Trying to mux just the AAC resulted in a floating point exception. I wasn't able to play either stream with mplayer, either. I thought this may have been a problem with the win32 builds of mencoder and MP4Box I grabbed, but I built from source on my home server, and had the exact same problems.

I considered chucking B-frames and just using a higher bitrate and CABAC for better quality, but B-frames are pretty key for getting better quality. So I pressed on.

I still haven't figured out why extracting with mencoder doesn't work, but I discovered that I can encode to mp4 with mencoder, extract the streams with MP4Box (which, conveniently, do properly play back with mplayer), and then re-mux them. Note that a simple remux from mp4 -> mp4 does not work.

MP4Box -raw 1 input.mp4
MP4Box -raw 2 input.mp4
MP4Box -new remuxed.mp4 -add input_track1.h264 -add input_track2.aac -fps 23.976

Note the -fps command line argument - MP4Box can't identify the frame rate from the h.264 file, and assumes 25 fps if you don't tell it otherwise. This framerate business is part of why I'm doing the wrapper script - you know the framerate when running mencoder to generate the output, so it's easy to pass along. Another benefit of re-muxing with MP4Box is that it puts the 'moov' atom of the mp4 file up front, which most streaming mp4 players require (e.g. if you want to stream this file to your iPhone).

I'll update the google code svn project soon, which will have full support for AppleTV via remuxing with MP4Box. After that, I'm going to work on source type detection, so you won't need to tell it if the video is progressive, interlaced, telecined, etc.

For reference, here's what my AppleTV encodes look like now (using the dsize option Chris pointed out on in the comments to my last post):

c:\program files\mplayer\mencoder.exe steep_sample.avi -o c:\windows\temp\mencoderwrapperkmvuek.mp4 -vf dsize=854:480:0,scale=-8:-8,harddup -lavfopts format=mp4 -ofps 24000/1001 -faacopts mpeg=4:object=2:raw:br=128 -oac faac -ovc x264 -passlogfile c:\windows\temp\passlogfilejg_rsh.log -sws 9 -x264encopts cabac:level_idc=31:bframes=6:global_header:threads=auto:subq=1:frameref=1:turbo=1:pass=1:bitrate=1500 -of lavf
c:\program files\mplayer\mencoder.exe steep_sample.avi -o c:\windows\temp\mencoderwrapperkmvuek.mp4 -vf dsize=854:480:0,scale=-8:-8,harddup -lavfopts format=mp4 -ofps 24000/1001 -faacopts mpeg=4:object=2:raw:br=128 -oac faac -ovc x264 -passlogfile c:\windows\temp\passlogfilejg_rsh.log -sws 9 -x264encopts cabac:level_idc=31:bframes=6:global_header:threads=auto:subq=5:frameref=6:partitions=all:trellis=1:chroma_me:me=umh:pass=2:bitrate=1500 -of lavf
c:\bin\MP4Box.exe -raw 1 c:\windows\temp\mencoderwrapperkmvuek.mp4
c:\bin\MP4Box.exe -raw 1 c:\windows\temp\mencoderwrapperkmvuek.mp4
c:\bin\MP4Box.exe -new steep_appletv.mp4 -fps 23.976088 -add c:\windows\temp\mencoderwrapperkmvuek_track1.h264 -add c:\windows\temp\mencoderwrapperkmvuek_track2.aac

Hope this helps someone. And if anyone has any idea how to make this stuff format better on blogger, please let me know.