Knights Of Serenity

Here's my first music video edit in a couple years. Frankly, it started out with my doing some experiments with editing in HD quality. Ever since I'd heard the song "Knights Of Cydonia" by MUSE, I'd always thought it would work well for doing a music video of Firefly clips. One I'd gotten started playing around with the HD editing process, I started thinking about what source material to play with, and the Firefly video idea popped back into mind (and the full Firefly series and Serenity feature film had been released to blu-ray). So, I started ripping the episodes in their raw HD quality to mess with. I'll detail the full process after the video (an alternate version is embedded at the bottom of this page)...

That embedded encode above is at a fairly low bitrate and quality level, but is at least at 1280x720 resolution (which you'll take advantage of if you view it in full screen). I mastered the video in full 1080p high def (and have full 1080p renders of the final product, but they're a bit more than a gig in size and unwieldy to present in an online form). Anyway, here's the process used, blow-by-blow. And for the record, none of this is done with malicious copyright infringement intentions. It's just a tech nerd messing around with one of his fanboy obsession TV shows (contender for the greatest series in TV history).

• Using AnyDVD HD (an app I've been a happily registered user of for some time), I ripped all the discs to a 1TB hard drive (using my handy eSATA drive dock) to start work. This drive would hold all my materials for this project, and would end up being 2/3 full by the end of the project.
• From there, I found that it was best to use Clown BD to pull the episodes out of the raw BD disc rips that AnyDVD made with just the video track I wanted and to transcode the audio from the DTS HD codec to AC3 (as the app I describe in the next step didn't like DTS tracks). Clown BD has a nice wizard approach to extracting the streams you want and writing them out to a raw TS file to use.
• Trying to get these blu-ray ripped files to open in Sony's Vegas Video 8 (my preferred editing app) was problematic at best. And when I could get something to load, the raw codecs involved were horrible to work with in terms of editing performance and accuracy. After tons of messing around and research online, I stumbled across Cineform's Neo Scene encoder. This was a major milestone in the experimentation process. It transcodes the videos into Cineform's proprietary codec. While the files jump from approximately 7gb per episode in their raw blu-ray form to 30-35gb per episode in their new Cineform format, they become a breeze to work with in an NLE such as Vegas Video. The files are fast and responsive in the editor, and spot-on perfect in frame accuracy. And here's where a bit of luck came into play. Cineform has a 7 day trial on the encoder, and the decoder is free. So I had 7 days to convert all my ripped source files, then I could continue with the editing beyond that 7 days with the free decoder. The Cineform output is so nice that I might even consider ponying up the $130 for the app were I to have more use for such a thing down the road (which is unlikely - as I'm not going to be attempting a project like this again for a while).
• Fox's Firefly set is encoded in AVC format, while Universal's Serenity is VC-1 encoded (surely the result of it having been previously on Universal's favored HD-DVD format, where VC-1 encodes were more the norm). The upshot of this is that Neo Scene could work with the AVC files from the Firefly set, but not Serenity's VC-1 codec.
• I spent a lot of time finding the best way to get the Serenity rip from the VC-1 format to an AVC format so I could use it. I ultimately ended up using HDConvertToX to get it into an h264/AVC format that Neo Scene would like. Actually, after having typed that, I'm doubting myself on which tool I used to accomplish that conversion. It might have been HDConvertToX, or it might have been something else.
• Anyway, once I had all my source files piped through Neo Scene, I could get them all into a project in Vegas. If you think the above info sounds confusing, you don't know the half of it. Or the tenth of it. I experimented with so many encoders, demuxers and rippers that I couldn't possibly remember even half of them that I experimented with. I spent more hours than I care to consider sifting through forums and wiki's hunting for useful info, too. If I had known what I was getting myself into, I probably would have never started the whole experiment.
• So with more than half the primary effort taken care of, I could spend some time in Vegas. I went through every episode and created subclips of any scenes or shots I wanted to use and semi-organized them in Vegas. From there, I began assembling the video. The assembly process is a highly experimental one. For some shots, I know where in the song I wanna place them. With other shots, it's a matter of just dragging it around in the time line and seeing where it fits best.
• Once I got, say, 80% of the song's timeline populated with clips, the "fun" part of finding clips amongst the one's I've yet to use that can work in the gaps in the timeline that I have yet to fill. Frankly, any bits and pieces in the video that I still don't quite like are the result of this part of the process. Those last 20% gap-fill clip choices are the ones that I'm never ultimately happy with, but they do what they need to - filling the gaps.
• Then came the part that I knew would be a particular challenge with this video. And quite frankly, I was kind of looking forward to this part, as I knew it would be a requirement when I started the project. That part would be adjusting aspect ratios. See, Firefly is shot for HDTV in a 16x9 aspect ratio. Serenity was shot in the anamorphic 2.35 theatrical aspect ratio. So if I left the clips in their natural ratio, the clips would be randomly alternating between 16x9 and 2.35, causing the black letterbox bars to randomly come and go, which looks awkward. While I ultimately rendered out a version in that alternating form, just for amusement and comparison use, I set about unifying the clips to work together. The most obvious target was getting the Serenity clips into the 16x9 format so the whole video is in 16x9 aspect (as presented in the embedded version above). This involved doing cropping and pan/scanning within the 2.35 frame for the most useful 16x9 frame of picture in the clip. In many instances, I actually have the cropping moving horizontally tracking the action in the frame, though you'd likely never know it watching the video if I hadn't revealed that fact. One downside to having to do this cropping/scanning of the frame is that I'm not getting the full native 1080p resolution in the clips from Serenity for the video, since I'm scaling them up to fit the frame. Still, those clips would be at least native to a 720p rendering. Regardless, I did my master render in 1080p, since the Firefly clips were still in their native 1080p format. The irony is that the fancier, bigger budget Serenity feature film is the one with lower resolution clips in the video. The untrained eye wouldn't spot the difference anyway.
• Once I had that done, I reversed the concept to create a fully 2.35 version of the video. What that involved was falling back to my Vegas project with all the clips in their native aspect ratio again. Then rather than doing horizontal panning and scanning across the wide frame of Serenity clips, I did 2.35 cropping of the Firefly clips and did panning and scanning in vertical directions as needed. Going into the project, I didn't think that making the 16x9 Firefly clips into 2.35 ratio would work as well, but I was surprised to find out that I could make most of the clips work very well in a 2.35 cropped ratio. And ultimately, I'm a huge fan of the anamorphic 2.35 ratio. Were I a film director, I'd shoot all my films at 2.35. So I was rather pleased to make the video also work in that form. That version of the video is embedded at the bottom of this page.
• So as you can see, this was an insane process. I must say that I'm quite happy with the final result. Like every video I do, I'm never completely happy with it, and wish I'd tweaked this or that, or replace some clips that don't quite work where they are with other clips, etc. Ultimately, there comes a point where I just have to stop screwing with it and just declare it "done enough." The quality of the video turned out very nicely. There's a little bit of edge aliasing I wish wasn't there (mostly in clips from Serenity that were scaled to fit the frame), but it looks far better than any previous standard def videos I've done - by a long shot. The master 1080p renders I did of the video are of much higher quality than the embedded version above. And like I said, there were countless other applications and tools I used as part of this process, even if they ultimately weren't needed or ended up not being part of the final workflow I used. More of them than I could remember, let alone list.
• Oh, I will put in a plug for RipBot264, which does a fantastic job rendering/compiling the material out to an AVCHD build (AVCHD is the disc format that HD camcorders use). The AVCHD discs play back beautifully in a blu-ray player (or a PS3, etc) in lovely HD quality. RipBot264 is also the app I used to render out the MP4 version that's embedded on this page.
• What do I figure the total number of man hours were that were involved? I don't know. Probably 40-50, all said and done. That's not counting the hours of disc ripping, file transcoding or video rendering, where I would have been doing other things while the computer did its thing. That time was spread in random intervals over the last 3-4 weeks. I'm just happy to be done with it. Well, "done enough."
• Here's a few random facts:
• The bit of music at the beginning is from Greg Edmonson's score for Firefly, but not from the episode/scene that it is paired with in the video.
• The opening scene is one I chose just to have something to kick off the video, as the beginning of the song itself didn't work to well as a start - plus it book-ended nicely with the shot I used of River at the end of the video.
• There are a couple speed-ups I did in parts of the clips of River during the opening and closing clips of the video for timing reasons - and I'm amazed that I can't see the speed-ups I did no matter how hard I look for them, even knowing they are there.
• There is one clip that is in reverse, where Mal points his gun towards camera (the clip just before the stunt dude is thrown over the bar into the glass). That's a clip from Serenity where Mal is actually lowering the gun from camera, but I found it worked better reversed.
• My favorite section of the video turned out to be the quick fading clips during the chorus of the song (at the 2:06 mark).
• And I couldn't help but include a few of the amusing clips like Wash with his flashback scene mustache, or a classic clip of him with his dinosaurs. Not to mention having to end it with Joss' Mutant Enemy "Grrrrrr, aaaargh!"