Converting "unsupported" Video files for Apple TV 2

Let's explore the least painful way to convert incompatible files like mkv and avi to Apple TV compatible mp4 videos. Also, the solution I'm using is a Mac only application, so if you're on another platform, I'm afraid you'll need to keep searching.

Hello again and welcome back to my nearly abandoned blog!  A couple posts per year ain't bad, right?

For my last post of the year, I wanted to share some of what I learned about beating various video formats (.mkv, .avi, etc) into something that both imported to iTunes and would play on the Apple TV 2. The main thing we want to do is avoid re-encoding the video if it's already stored in the original file with the correct codec.

I want to try and break this post up into sections to make it a little easier to digest since I tend to be a bit verbose sometimes.  I'll break this up into sections for an overview of my experience, technical details on codecs, transcoding and conversion tools and finally audio formats (which is the main problem I had with conversion).  If you don't really want the back story or the details on codecs and tools, jump to the summary section at the end of the post that explains the process.

My Foray into Apple TV
This Christmas, I found that Santa left me a shiny new Apple TV under the tree.  Interestingly, I found it does stuff that my TiVos already also do, but the Apple TV is much more elegant, and streaming content located on my Mac super fast and much more user friendly.  That is to say once I was able to actually get content loaded into iTunes and playable on my new device.

Since I have been using PyTivo  for ages to get various video copied over to the TiVo boxes in the house, I've amassed a fair collection of content in a number of different formats.  Of course all of this content is perfectly legitimate, I would never condone any sort of piracy or circumvention of the DMCA, that is just wrong, mmmkay?

At first, I just started dropping my files into Handbrake using either the built-in presets (high profile was the recommended one on the forums) or by downloading and using specific presets for new Apple devices.  That worked great, the converted files played just fine on the Apple TV, save for two problems.  First was that since I was re-encoding stuff that was already compressed, there was a loss in video quality (though it was slight) and second was the process took an eternity.  With my dual core iMac it would take a few hours to convert a full-length 720P video.

Being an impatient person, I decided to school myself in the world of video conversion, so with some quality time with Google and some trial and error I was able to come up with a solution that will save countless hours of encoding, I hope.  Now is a good time to point out that none of this stuff is really original though, just things I found online. I will try my best to provide links wherever possible, but if I missed something, please let me know.  So let's move on to some of the technical details.

Tech Overview (or really basic things I should have known but didn't)
To understand the most efficient way to get video compatible with the Apple TV, I found it was important to learn about some basics like the ATV's capabilities, the difference between codecs and containers and some of the more common formats for (totally legit) files out there.

Supported Video Formats for the Apple TV 2:  There are basically a few highlights to take away from the spec details.  The only videos ATV can play must be MP4 files (.mp4, m4v, or mov), have to be encoded as h.264 and have an AAC soundtrack with a bit rate no higher than 160kbps.  Alternately, the files can have an AC3 soundtrack ONLY IF your Apple TV is connected to a receiver or TV that can decode that format.  That last part is where I ran into most of my trouble, more on that later.

Ok, let's dissect the supported format details a little bit more to try and get a feel for what we need to look out for.  The first part of the specs are what I tend to refer to as the container, which is probably not technically correct, but it's my blog dammit and I can call it whatever I want.  So by container I'm referring to the actual file type that stores the video and audio information that makes up a video file.  There are a number of containers running around out there in the wild, there are of course the ones directly supported by Apple like mp4, then there are the more popular and unsupported ones that we are mostly interested in like mkv and avi.

As mentioned, these container files contain video and audio streams, in many cases, the video stream included in a container unsupported by Apple TV is actually compatible with the Apple TV.  If a video file includes a stream encoded with h.264 (you may also see it called x264) then assuming it's not encoded at an outrageous bit or frame rate, it will be playable without transcoding into a different format.  If your file uses a different codec like XviD or DivX, you'll have no choice but to re-encode the entire file.

The same thing applies to the audio codec stored in the file, if it's compatible, it won't need to be converted.  The bad news is that I haven't seen a lot of mkv files with stereo AAC.  This particular point is where I ran into issues, especially finding a utility that could do video pass through while re-encoding the audio.  The good news is it takes WAY less time to just transcode audio when converting a video to an mp4.

ProTip: Most video players (like VLC or Quicktime) have an option to bring up information on a particular video which should tell you exactly what codecs are being used as well as bit rates and frame rates.

Get info in Quicktime X (using Perian). We would need to transcode the video in this file since it's DivX.  The audio is compatible, however.

Ok, that's great but...
So let's cover some of the applications and techniques required to actually make this thing work.  There are a bunch of different things you can use, depending on what you need to do and your individual preferences.   Tools that I personally use a lot are Handbrake (for full on encoding), Quicktime X (with Perian installed to allow me to load most types of videos) and finally, MKVtools.  Of the three applications listed here, MKVtools can actually do everything and will be the focus of the rest of this post.

Using the information I had up to this point, I was using QuickTime to simply convert the container to MOV for existing h.264 video by using save as and selecting the movie format.  This resulted in a file that imported into iTunes and played just fine, hooray!  Then I tried to open it up on my Apple TV, it attempted to stream it, but quickly produced the dreaded "An Error Occurred Loading this Content. Try Again Later" message.  Researching the error message, I found this blog post that indicated the ATV might have trouble detecting the AC3 audio stream.  I switched the AC3 setting to on, but alas, my super cheap-o house brand TV couldn't handle an AC3 signal apparently as I continued to get the error message when attempting to play the video.

Create Apple TV magic with MKVtools
Ok, so I need to find a tool to allow me to pass through the video stream while transcoding the audio stream to a compatible AAC format.  I keep a number of tools around, but I found none of them really offered the ability to pass through video, most could pass through audio, but not video.  Some searching revealed that the old version of QuickTime Pro would let you do that when saving as, but that wasn't available to me.  I also have a copy of VisualHub laying around, which lets you use the pass through flags for both video and audio.  For some reason the resulting video files were unplayable in anything.  Since VisualHub has been abandoned for years, I decided to keep looking.

I stumbled across this thread on the Mac Rumors forum that was discussing methods of using a bunch of different command line open source tools to manually extract/convert the video and audio streams and then re-combine (mux) them back into a playable mp4 file.  Too much work for me :)  However, the thread was 18 pages long, and in the latter pages there was mention of a graphical front end for these tools called MKVtools.  

MKVtools is available with an unlimited trial that limits a few minor features (like queuing) but otherwise is fully functional.  The author wants to make sure the application works for you before you pony up the princely sum of... 5 bucks!

MKVtools can take a variety of formats as input and produce avi and most importantly mp4 files as the results.  It can do full encoding of video and audio or selectively let you select either or none for conversion to an mp4 file.  As an additional bonus, you can also keep (or convert to) an AC3 sound track and just add your AAC stereo stream as a secondary stream.  This solution worked for me and allows me 5.1 compatibility with my TiVo that is connected to a proper AV receiver.  As an added bonus you can also work with subtitles, though if you want to hard code a subtitle stream from an mkv file to your mp4, you will have to re-encode your video file.

The main MKVtools window with our sample file loaded

Once your video file is loaded, the main window will show you the specifics of the streams included in your video file.  Note the track info for the video stream.  It is listed as mpeg4.  This is NOT compatible with the ATV, if it were it would be listed as h.264.  Therefore, in this particular example we would need to do a full re-encode, this tool would work, or you could always use your favorite tool to re-encode.

The MP4 section

Here's the mp4 section where we'll be creating our compatible mp4.  We'll ignore the incompatible format for the moment and pretend we have a real h.264 stream.  To convert, check the boxes next to the audio and video streams in the top window.  Then simply click on the pass thru button in the video section, this video already has an aac stream, so we'll also select pass thru for audio and allow the application to just repackage the file as an mp4 file.  If the packaged audio were AC3, you could also leave pass-thru selected and simply check the add 2 Ch. track button, it will leave the existing audio alone and add an AAC secondary track which could be read by the Apple TV.  Finally if you have some other type of 5.1 soundtrack, like DTS, you could select either AAC 2ch or AC3 (5.1) and then check the add 2 Ch. track button.

Click convert and the resulting file will by default be saved to the same location as the original (along with the log file and intermediate audio and video files).  You can change the save location and the retention of the extra files in the preferences.

Thanks for sticking with me this far.  The summary of all this is below if you need a quick reference (or didn't want to read this massive article).

Summary
Ok, so here's the basic steps for conversion happiness in numbered list form.  The moral of the story is if you have a video already encoded in h.264, you want to avoid re-encoding the video if at all possible.

1. Download MKVTools
2. Open your video file in MKVTools and click the MP4 tab
3. Select the video and audio streams check boxes, note the codecs of each
4. If video is listed as h.264, select pass thru in the video section.  Otherwise, select one of the options to re-encode the video (or use your favorite other video encoding tool, you can't pass through the video for this file).
5. If the audio is listed as aac 2ch. no futher steps are required, select pass thru for audio then click convert.
6. If the audio is AC3, click the add 2 Ch. track button then click convert
7. If the audio is something else, select AAC 2ch and click convert, you may be able to convert 5.1 sound to ac3 5.1 but make sure to click the add 2 ch. track option, click convert.
8. import the resulting mp4 file into itunes when it's done.
9. View the file on the Apple TV.

TiVo's Late Adopter Penalty

Hello friends, and welcome to today's Grandpa Simpson rant!

So TiVo's next big thing has finally been announced.  The TiVo premiere will be shipping in April and is being touted as the one set top box you need to provide a way to get all kinds of content on your living room TV.  Cool!  I've been a proponent of using a TiVo as a multi-purpose media box for ages.  The current TiVO HD units have a lot of options for content besides cable programming, TiVo has just never done a great job of beating that drum. 

Now the company has started figuring out they should really get into the integration game and push that existing stuff (and some new) front-and-center.  Even better, there will be a real, honest-to-goodness HD interface.

Hooray! I'm sure some of these things will get pushed to the existing HD units since they have that capability, no more ugly stretched SD interface.

Wait... What's that? Oh, so the new platform has a lot more processing horsepower, and they're not going to be refreshing the UI on the HD units?  Sure, it's understandable, the thing uses Flash after all, that's not gonna cut it with the HD's geriatric computing power.  After all, the HD and Series 3 units have been around for years, time for them to go.

Based on the way TiVo has dropped support for the previous platform as soon as the next new and shiny platform comes up, this doesn't bode well for those of us sitting on Series 3 and HD units.  I can understand that, except for one small thing, I just bought a unit in December for Christmas, so I'm not over here like Grandpa refusing to give up my Windows 98 box and dial up modem because it hasn't caught on fire just yet.

Fine, my fault for not waiting, for being a late adopter and all.  My unit will keep working no harm no fail. 

As long as I keep paying TiVo that monthly fee.

Gee, no new features, no update to the UI that was pretty much a stop-gap to begin with, and I get to pay for that privilege?  Awesome!  It's like the mob is running things over there.  We're sorry we can't really give you new features, but you should pay up, we'd sure hate for something to happen to your ability to access your videos over the network or your season passes.

Sure, stuff gets obsoleted all the time, and someone has to get stuck with holding the bag on legacy hardware, but even Microsoft and Apple usually keep supporting a platform at least past the damn warranty expiration.

I'm still a fan of my TiVos, mostly because of the convergence, I can stream my DVD collection, or online content right along side of the stuff I record from old school TV.  So far there's not really much that can compete with it (HTPC maybe, but it's still a bigger pain in the ass, and has a way lower wife acceptance factor.)

More and more I become filled with dread over the prospect of getting a new TiVo.  First there's buying the unit, then the monthly service charge which eventually will be to pay for nothing other than to be able to use the thing.  And that's after the dreaded 7th ring of hell which is getting the cable company to figure out how the hell to make the stupid cable card work.

I'm thinking maybe it's time just to live with a so-so DVR experience paired with a second, internet-enabled set top box to get all the modern content.

Play any Shoutcast stream through TiVo (OS X and iTunes)

I discovered this feature by total accident the other day.  This may be well known, but who knows, I figured it might be worth sharing to anyone that stumbles across this.

Those of us with TiVos and Macs are kind of left out in the cold when it comes to the built-in functionality of the TiVo supplied TiVo Desktop for Macs.  You get only photo and music sharing (and some basic video publishing if you hack it).  If you want anything more advanced you have to spring for commercial software from Roxio (or go with one of the unsupported, but excellent community apps like PyTiVoX).  Anyway, there's one thing the stock software offers that as far as I know the Windows version doesn't: sharing your iTunes library complete with playlists to your TiVo units.

The fact you can share iTunes playlists with the Mac version of the default desktop is nice to begin with.  But, it turns out if you have a ShoutCast (or similar streaming audio) location saved within iTunes, you can select it from your library and the stream will play. I noticed this simply because a streaming station ended up in a smart playlist and just started playing along with my regular MP3 files.

It's pretty simple to setup, here's the details:

The first step is to download and install the TiVo Desktop then configure iTunes Sharing.

Next, add the address for a stream in iTunes, which can be found under the Advanced menu.  As an example, I'm using the audio feed from TWiT, which steams live and recorded audio from the podcasts being recorded by Leo Laporte's TWiT network.

Enter the URL:

When you click OK, a link to the stream will show up in your music library like any other audio file.  To make it easier to find on the TiVo side you can add it to or create a new playlist so it's listed on the top level of your iTunes listing on your DVR.

Once you have the file in iTunes, go over to your DVR and select your shared iTunes library from the Music, Photos and Showcases menu.  From there you can navigate to the location of your newly added stream and just hit play to begin streaming.

Keep in mind this is just simple streaming, as far as I can tell it won't send across any metadata such as current track or play times, but it's a nice little undocumented feature to have.

I almost want an iPad (Or NO FARMVILLE FOR YOU!)

Quick math quiz - solve for X: Old Dell Laptop + X > iPad - X

The solution in just a minute, but first some background. I, like most of the free world spent a great deal of time following the live coverage of the new Jesus Pad powered by Unicorn Tears presented by Steve Jobs himself (instead of being a productive contributor to my employer). At first I was wondering what was wrong with me. I just wasn't beside myself with crazy fervor over the iPad. I must be running low on Apple Kool-Aid, surely they couldn't have totally whiffed what was supposed to be their most revolutionary gizmo ever. It was just a big ass iPod Touch, no wild hand flailing like a futuristic Tom Cruise in Minority Report, no wireless HDMI, and not even a single camera (let alone two of them).

At least the price was a bit of a surprise. Sure, it still out costs most serviceable netbooks and even some full notebooks, but for a first gen Apple product to be under 7 bones, that was pretty impressive. Still I kept wondering what was wrong with me, why wasn't I trying to figure out the best way to make this thing a part of my life? I simply decided I must be in the process of losing my cool, I am a parent after all, and by the time the kids hit middle school the cool will be completely gone anyway.

So I did something odd, I simply went on with my life and got back to work. Then my thoughts wandered off into the realm of how my wife and I use our computers at home. We each have our own laptops (I inherit her old ones, she gets the new ones.) My wife's MacBook is getting to be of a certain age, so soon it shall be mine, and my old Dell can find a new life on a farm where it can frolic and play with other decrepit computers.

Then thoughts of the iPad and the $499 entry price sunk in. Hey, you know you could buy TWO iPads for the price of a MacBook and each have your own shiny new Unicorn Tear powered device. No more stupid arguments over who's the most downtrodden second-class citizen in the family. Being at work, my mind shifted to use case and workflow analysis.

My wife and I pretty much do the same stuff on our laptops; we just sort of fart around with them after the kids go to bed. You know the same thing everyone does, e-mail, Facebook, web surfing and so forth. Hmm... the iPad does that, and it offers a seemingly better form factor (and no wobbly lid and loose hard drive cage like my Dell). Holy cow, now I get it... NOW I'M COOL AGAIN!

I was ready to start pitching my wife on the idea when we got home from work. And I started thinking a little bit more about the specific sites we visit on our laptops. Then I had a horrible thought: No Tiki Farm! No Farmville! CRAP! My wife more than me is addicted to one of these silly Facebook games, and then I realized, it's Flash based. Early reviews of the iPad confirm there's no Flash. After a quick inventory of the sites I visited frequently, I realized there was a lot of Flash involved. It's kind of a big deal

Sure we've all lived with iPhones without Flash for a long time now, but that's a different animal in my mind. I use the phone differently, sure it's sometimes a bummer to not have Flash support, but I still prefer to do a lot of things on a larger screen. Not having it on the iPad on the other hand is a serious problem. Basically, I would need to have the iPad AND still hang on to my rickety old Dell so I can whip it out when I feel the need to watch Hulu or any number of other tasks. The old laptop still can surf the web, and check e-mail and do everything else the iPad can do, except I can't turn it upside down, the screen would probably fall off or something.

Yeah that seems kinda stupid to me too. So the answer to our math quiz is flash: Old Dell + Flash > iPad - Flash. It can do all the web surfing, watching videos e-mail fairly competently as well as play stupid little games where I have to try and annoy enough friends into joining my quest to plant virtual pineapples.

Ok, well it's early in the game the iPad isn't in anyone's hands yet, maybe Flash can make it into the iPad at or shortly after shipping time. Also, perhaps pigs will fly.

But then again, Steve's reasoning was that the resource hog nature of Flash would melt the poor little iPhone into a quivering mass of molten mobile phone processor bits. The iPhone just couldn't give Flash the necessary resources. Ok, understandable I've seen how some apps simply make my poor phone whimper in pain. And of course being a dedicated fanboi, I'd never question His Jobbiness. Well, except there's one thing I don't quite understand Mr. Jobs - unlike the iPhone, by all accounts, the iPad runs like a stripe-assed ape. Imagine I said that last part like detective Columbo.

So if the iPad is a speed demon, is there actually a reason (other than it is kinda crappy) Apple will put forward as to why Flash won't work on it this time?

While I wouldn't mind cashing in my old Dell for my wife's gently used MacBook, I'd much rather have other options. I burn my incense and swing the dead chicken over my head at my little shrine of the great Apple (while facing Cupertino of course) and pray that there may be a lasting peace made between the houses of Adobe and Apple and the iPad could indeed become a respectable citizen of my leisure time.

I promise you Oh Great Jobs, I won't be greedy, I won't ask for Flash on the iPhone. I will trust in your wisdom in such magics, but please please let me live a peaceful existence of technical equity with my significant other.

Until then, you can find me at the store buying duct tape for my trusty old Dell.