codeblog

I was too lazy to walk over to my TV, so I decided to watch my MythTV recordings on my desktop without having installed a MythTV frontend. Via the magic of MythTVfs, I started watching a recent Stargate episode.

Before the opening credits had finished, I knew I was already going to miss MythTV’s commercial flagging. So started the investigation into where in the world MythTV stores that information. I was imagining adding “.edl” files to MythTVfs automatically, etc.

In MythTV 0.19, using “mythcommflag”, you can get the “Cut” list, but not the “Commercial Skip” list. (Think of the latter as a “Cut Hint” list.) The command for this is:

mythcommflag –getcutlist -c 1059 -s 20060728210000

In MythTV 0.20, you’ll be able to use “–getskiplist”. Since I’m still using 0.19, I had to go directly to the “mythconverg” database to get the details. The marks are stored in the “recordedmarkup” table. The mark types I care about are: 4: Commercial Start, 5: Commercial End. This SQL query gets me somewhere:

SELECT mark, type FROM recordedmarkup WHERE chanid = “1059″ AND starttime = “20060728210000″ AND (type = 4 OR type = 5) ORDER BY mark;

However, mplayer’s EDL file format expects time, not frame number. MythTV stores frame number. Luck for us, it’s all NTSC MPEG2, so we’re at 29.97 frame per second, and I can modify the SQL:

SELECT mark/29.97, type FROM recordedmarkup WHERE chanid = “1059″ AND starttime = “20060728210000″ AND (type = 4 OR type = 5) ORDER BY mark;

Now I just have to get the pairs on a single line with a trailing “0″ for mplayer to know to skip that time frame:

echo ‘SELECT mark/29.97 FROM recordedmarkup WHERE chanid = “1059″ AND starttime = “20060728210000″ AND (type = 4 OR type = 5) ORDER BY mark;’ | mysql -B –skip-column-names | xargs -l2 | awk ‘{print $0 ” 0″ }’

Combined with some logic to extract the channel and starttime for a given recording, I’ve now got a really crazy wrapper script that’ll let me mplayer a recording after generating an EDL cutlist.

(With thanks to Ken’s excellent collection of “merge pairs of lines into single lines” short cuts.)

Today, Randy let me borrow his awsome presentation remote. It’s basically an RF remote, whose other end is a USB fob, that acts as a keyboard. The two “next” and “previous” buttons on the remote map to “PageUp” and “PageDown” keys, which worked great for the Crucible and Xen presentation Bryce and I gave this afternoon.

Tomorrow, for my MythTV presenation, I’m using OpenOffice.org, which makes a distinction between “Space” and “PageDown”. “PageDown” goes literally to the next slide, where as “Space” triggers the next animation within a slide. Since my presentation slides have a ton of “reveals”, I need to be pressing “Space”, not “PageDown”. A search of the Logitech page yielded no info on changing the remote’s config, so I opted to using “xmodmap” to get the job done for me.

Using “xev”, I found that the keysym for “PageDown” on my keyboard is 0xff56, and the keysym for “Space” is 0×20. So, I blew away my PageDown key, and replaced it with Space:

echo keysym 0xff56 = 0×20 | xmodmap -

Now the remote works just how I need it. :)

Every once in a while, I hear something in a movie and yell, “Hey! That’s in a Frontline song!” This time it was while catching a portion of the The Abyss, where a big chunk of gear misses falling on their habitat, and they kind of laugh nervously. Well, that nervous laugh was looped for about for about 30 seconds at the start of “Victim of a Criminal” from Millennium. I found it at about the same time I thought to put “Frontline Assembly samples” into Google, which gave me this page, which is a list of all the samples that they used. Robocop, Aliens, and the one that haunted me for a while before this Abyss incident: Stargate and “send in the probe”.

This is what my brain is filled with: sound effects. Great. :)

Over the weekend, I coded up a protocol dissector in Ethereal for the TiVoConnect Discovery Protocol. The protocol is very simple, but I still wanted the satisfaction of seeing it listed by name when scanning through my home network captures while debugging Galleon/TiVo traffic.

Ethereal has great developer documentation. It was easy to find and got me coding right away with a skeleton dissector. I just love the projects with these kind of to-the-point examples. The only thing I felt was missing from their README.developer was something showing that the dissector routine could return gboolean, letting a dissector reject being attached to a given packet.

There were other clearly written dissectors that I used for reference: DNS, Yahoo, and Syslog. They seemed to answer most of the more subtle questions I had about rewriting column text, scanning the packet, and dealing with other special cases.

Hopefully the patch will get accepted. I even did the randomized testing the wiki recommended. :)

A few months ago, zap2it’s renewal survey included a set of questions that were not questions, but rather pleas that people move their database updates to a non-standard time. The default install of MythTV has a hard-coded time in the middle of the night to contact the zap2it servers for TV listing updates. Unfortunately, this means zap2it’s servers were (are?) getting crushed on the hour, across 4 time-zones, in the middle of the night.

To solve this little problem in MythTV, I turned off the built-in “mythfilldatabase” execution, and moved to doing it via crontab. zap2it said they had virtually no load during the day-time, so I moved the update to a little after noon, sticking the following in /etc/cron.d/mythtv-backend:

09 12 * * * mythtv mythfilldatabase –quiet

It dawned on me today while messing around with the “at” scheduler that I could actually randomize when during the day the mythfilldatabase runs. If you wanted to run the job anywhere in a 12 hour (720 minute) period after 9am, you could do it this way:

0 9 * * * mythtv echo “mythfilldatabase –quiet” | at + $(( RANDOM % 720 )) minutes

(My bash manpage says the low-order bits of $RANDOM are as random as the high-order bits, so this is “safe”. If you don’t trust your version of bash, you could use $(( 720 * RANDOM / 32767 )) instead. IANAMG*, YMMV, OMGPONIES.)

*I Am Not A Math Geek

I ran into a nasty bug while attempting to downgrade a MySQL database. I had been running my MythTV machine on Debian Unstable, but recently reinstalled to Ubuntu. This has the unfortunate consequence of going from MySQL 4.1 to MySQL 4.0. The “mysqldump” option “–compat=mysql40″ kind of forgets to include the “auto_increment” flag for tables creation. This caused my subsequent MythTV 0.18 to 0.19 upgrade attempt to instantly bomb, since all the INSERTs expecting the PRIMARY KEY to increment as new stuff was inserted … didn’t.

Once I split the dumps into tables (-t) and data (-d) with different “–compat” levels and hand-edited the tables, everything was “fine” again. I actually got the whole system up and on its feet again, with no loss of Stargate SG-1 episodes. ;)

So, now all I have to fight with is Xv on an old S3 card. Looks like new versions of Xorg don’t aim Xv to the right place. And, mysteriously, the S3 card’s Xv implementation lacks the XV_SWITCHCRT attribute, so I can’t just use “xvattr” to fix it, like I do I my laptop. Aaagh.

I wonder if something like the xorg.conf’s Option MonitorLayout “TV,CRT” might help it? I’ll try that tomorrow.

From the time-lapse construction webcam I set up with Brian, I built a pair of AVIs. These were made with ffmpeg so they use the FMP4 codec. Windows folks can find ffdshow in a number of places.

Left view movie (31M), set to Delerium’s “Silence” (with Sarah Mclachlan).
Right view movie (24M), set to a remix of Everything But the Girl’s “Like the Deserts Miss the Rain”.

I tried to build these movies showing only day-light hours, on work days. A few holidays sneak in, though. (There’s a longer section of a few days where no one shows up for work across Thanksgiving, for example.)

The AVI frame rate is 25fps, with each frame jumping 10 minutes. The effective speed is 14400:1. They span the time from July 26 2005 through January 21 2006. (The right-side camera was added on August 29.)

My least favorite easter egg is where the room-light-shield I taped up to keep room glare off the camera peels off the window and hangs in front of one of the cameras during Christmas and New Year’s vacation (when no one was around to fix it).

My most favorite easter egg is near the latter half when a pile drilling rig is parked in the foreground. Over the course of the day, the hydraulics holding the drill up at an angle bleed out, and the drill slowly pitches forward. I just love it; it’s exactly the kind of event no one notices at the time because it’s happening so slowly. Seen in time-lapse, though, it’s very obvious — it’s the only thing moving at all! :)

Enjoy!

Netflix accidentally lets you download movies for free. I reported this on March 18th, but they still haven’t replied. It’s been 2 weeks, so I’m posting the details now.

While digging through Netflix’s javascript I found a function named “startDownload“. I was originally just curious about the AJAX responsible for the movie info popup boxes, but this proved much more interesting.

I’m guessing they must be beta-testing this for some accounts because nothing visible through my account ever calls “startDownload“, but I could still use it.

Turns out the function handles a bitrate selection, and then just rewrites the URL a little. You can get the same affect by just adding “&download=avi&br=4” to the end of a movie info URL. For example, this is the URL to get info about Ice Age, and this is the URL to download Ice Age. This even seems to work without being logged in.

I haven’t had time to check if everything in their library is downloadable, but of the 6 or so I tried, they all worked. If anyone finds a cut-off date, let me know.

Woohoo! I got accepted to present at OSCon again! I’m really excited about this one, too — I get to present about something non-work-related. The title of my presentation is “DVR Happiness: Gluing MythTV and TiVo together with Galleon“. Here is my proposed outline:

1. Intro to DVRs
• TiVo: have you been under a rock?
• MythTV: learn all about video standards.
2. TiVo Gets You A Lot
• Hacked TiVos can do great things

Is your TiVo a tool or a toy?

• Stock TiVos can do cool stuff too

ToGo: move video from TiVo to PC

GoBack: move video from PC to TiVo

MP3s: streaming from anywhere

Image Galleries: beyond just snapshots

• Galleon Gets You More
• Implements the server-side of TiVo features
• On-the-fly format conversion
3. MythTV Gets You The Most
• Making Tivo recordings available to MythTV

Format conversion

• Making MythTV recordings available to TiVo

Mounting a MythTV filesystem with FUSE

• Making your MythTV remote make noises

Short-cuts with the Linux IR daemon

EDIT: WordPress pisses me off so very much when it comes to lists, indenting, and code snippets. Some day, I will switch to something that just lets me type in HTML and doesn’t try to “fix” it for me afterwards. *fume*

In the back of my head, I’ve been wondering about various ways I could use my $20 camcorder. Making cheap movie shorts, like SNL’s “Lazy Sunday” come to mind. There are no fancy zooms or special effects. It’s all editing and audio. Bryce kicked some ideas my way, most of which include using the dogs and/or cats as the primary actors (I can pay them in kibble). He also suggested renewing my father’s Rottweiler Camcorder with a much smaller device. I think the images would be mostly obscured by my dog’s chin. We’ll see. I also worry it may suffer from being summarily ingested.

While playing Name that Tune 80s DVD Edition, the a-ha video for “Take On Me” came on. The rotoscoping used to create the animated parts made me think of Inkscape’s autotracing function, and I lept up to go investigate the possibilities.

So far, I’ve played with two styles. One leaves the autotraced fill areas (which makes a video look like a really freaky cartoon), and one that reduces the fill opacity, and adds line density so it looks more like a regular outlined cartoon. That one tends to be distracting, though, since the edges keep jumping all over the place.

The autorotoscope script requires mplayer, ImageMagick, autotrace, and ffmpeg.

Here are the results:

• Original (364KB)
• Filled (478KB)
• Outlined (495KB)

These AVIs use xvid for their video codec. If you don’t already have it, you can get them from here.

My first working crack at getting a filesystem overlay working between MythTV, Galleon, and my TiVo is finished. If you’re brave, check it out.

Every few years, I end up relearning Java for some project or another. Today, that project is Galleon, the fantastic TiVo Home Media Server. I’ve been sort of mildly involved in another person’s attempts to get MythTV and his TiVo working nicely together.

The basic situation is this: I want to be able to watch stuff I’ve recorded on my MythTV box via my TiVo. Galleon already does this.

Using the “GoBack” TiVo feature, Galleon acts like another TiVo on the network, and sends a show to the TiVo. Normally this is used to send a .TiVo file from your computer back to the TiVo. In the case of TiVo-to-TiVo communication, they include the metadata about the show (title, air date, duration, description, etc) before starting the data transfer. Galleon stores the metadata to a local database when it downloads shows via the ToGo feature.

It is possible to send any valid MPEG stream to the TiVo, but unless the Galleon database has metadata for the show, there will be nothing but the filename on the TiVo end when it transfers. In the case of MythTV shows, the metadata is contained in the MythTV database. I’m hoping to create a MythTV “application” for Galleon that will connect to the MythTV database, and populate the TiVo with the needed metadata.

Since I’m so green on Java, I’m doing something else as a “proof-of-concept”. It was suggested that some of the metadata could be encoded into the filename. This requires two halves: a parser in Galleon to extract the data, and the files to be named with their metadata encoded.

On the Galleon side, I’m digging around with the StringTokenizer, and generally getting my feet wet with the Galleon source and banging my head on the java compiler.

On the filesystem side, I’m going to use FUSE to create an overlay filesystem that queries the MythTV database, and builds a list of files based on an NFS mount’s contents. (Which I’ll NFS mount from my MythTV box.)

Mostly I just wanted to write a FUSE application. :)

My friend Brian just had his article-writing debut on O’Reilly’s ONLamp site. He wrote about how to use XMLTV to help manage your TV viewing schedules if you don’t have a DVR doing it for you (or not doing it well enough).

I have been really unhappy with MythTV’s visibility of program “Original Air Date” information, which as far as I could tell is only visible through the Program Finder. I wanted to be able to see original air date while I browsed my recordings. Digging through the MythTV code has proven very difficult. The documentation has been minimal, and I haven’t found any tutorials on theme creation, which seems to be where all the visible components of the mythfrontend get their details from.

While looking for the bleeding-edge code, I did find http://cvs.mythtv.org/ which is actually a Subversion repository, bug tracker, and wiki. There a nice start to information there, including doxygen output. Also, the #mythtv-users channel on freenode has a nice MythTV FAQ.

The bulk of the display stuff I was looking from takes place in programs/mythfrontend/playbackbox.cpp (thank Bryce). The “cursorDown” function led me through to the “update*” functions, and eventually ToMap/SetText calls, which load program information into a hash, and then pass that hash to the theme engine.

libs/libmythtv/programinfo.cpp has ToMap defined, and all the various hash keys are visible, including the original air date variable I was looking for:

progMap[”title”] = title;
progMap[”subtitle”] = subtitle;
progMap[”description”] = description;
…
progMap[”originalairdate”]= originalAirDate.toString(dateFormat);

SetText is in libs/libmyth/uitypes.cpp. Hash items are uppercased to match %-enclosed words from the themes. The first “|” seen is to identify “what appears in front”, and the second is “what’s after”.

I modified the ui.xml from my theme (G.A.N.T. currently) from:

  <value>%SUBTITLE|”|”
%%STARS%%DESCRIPTION%</value>

to:

  <value>%SUBTITLE|”|” %%ORIGINALAIRDATE|(|)
%%STARS%%DESCRIPTION%</value>

So now, when I scroll down to Smallville, I see in the description box:

“Aqua” (2005-10-20)
During a beach party Lois hits her head when she jumps into the lake, and …

Ta-da! Original Air Date in parens. Now, being able to see the year is important, so I had to change my date format to one that included the year, but it’s ugly. To fix this, I need to actually change code. In “MythDateFormat” from programs/mythfrontend/globalsettings.cpp, I added:

gc->addSelection(sampdate.toString(”ddd MMM d, yyyy”), “ddd MMM d, yyyy”);

Now I just have to get it compiled. :)

Well, I’ve discovered that my “slow to respond” UI was entirely due to the lirc “repeat” settings. I’ve eliminated (”repeat=0″) the repeat settings for the Esc, Up, Down, Left, Right, Space, and Return buttons. I discovered this only after seeing that ircat was just as slow to respond. I’ve hacked together a “make noise” script (named “irnoise”) that runs along with mythfrontend:

#!/bin/bash
export SOUNDS=~mythtv/sounds

ircat mythtv | while read NAME; do
  case “$NAME” in
   Return|Space)
    SOUND=select.wav
    ;;
   *)
    SOUND=default.wav
    ;;
  esac
  #echo “$NAME: $SOUND”
  aplay -q “$SOUNDS”/”$SOUND”
done

This gives me my “boop” and “click” noises for all remote buttons. Yay! Since I couldn’t find the official TiVo noises, and I don’t feel like taking my TiVo apart right now, I just grabbed some noises I found online. From the “MiscWAVs.zip”, I used “THUD.WAV” and “BTN_DWN.WAV”:

sox src/THUD.WAV -t wav -c 1 -s -w -r 48000 default.wav resample
sox src/BTN_DWN.WAV -t wav -c 1 -s -w -r 48000 -v 2 select.wav resample
normalize-audio -m default.wav select.wav

Not a lot of complaints left with my MythTV installation. :)

I accidentally fixed my “fonts are too small” problem. While reading the MythTV development notes, there was mention of everything being designed for a 100dpi screen resolution. As it turns out, my Xorg config was defaulting to 75dpi, so I forced it with the DisplaySize option. Since my video card uses 640×480 for it’s NTSC output, I had to modify the recommended settings to use:

DisplaySize 162 121 # 100 DPI @ 640×480

I was surprised to discover that this solved my font size issues. I had so totally given up on the font size problem I didn’t even list it was a problem in the prior MythTV blog entry. :) I was actually expecting fonts to get even smaller, but I guess this changes how font selection is done, and as a result, everything appears sane now. Neato!

Also yesterday, in the hopes of reducing the effects of the “crappy audio” problem, and allowing multiple programs to have the soundcard open, I figured out how to get ALSA working natively within MythTV. In the MythTV configurations, the sound device should be “ALSA:default” instead of “/dev/dsp”, and the mixer should be “default” instead of “/dev/mixer”. The start of this was gleaned from the link I mentioned earlier. Mixer settings were found through trial and error. Strangely, MythMusic had a separate playback configuration, so I had to change that to “ALSA:default” as well.

Now that ALSA is being used, the audio choppiness has not returned. I can still get desync’d A/V, but I think that’s entirely due to disk latency issues, or something like that. Usually when it happens, if I pause or restart playback, it goes away. Also, since multiple programs can open the ALSA device and play sounds at the same time (thank you ALSA dmix), I can start looking at how to add a tool to play “button press notification” sounds. I’ll initial probably use something like “ircat” piped to a reader just to get a proof-of-concept. Then I’ll find some hooks in mythfrontend to attach it to instead. Eventually, I was a themeable visual notification. I should probably join the dev list to make sure other folks aren’t already working on this.

I’ve also found mention of “show type priority bumping”, where “New Episode” can trigger a bump in the priority of a recording. This may be a good step towards recording new Stargates at high priority, but reruns at very low priority. In the priority adjusting tool, I can see the third row for this kind of priority bump, but I can’t find the UI elements to adjust it.

I was preparing myself to deal with the “pain” of moving to a multimedia system that didn’t have all the feature tweaks I need. MythTV has surprised me in that after only a week, and I’ve solved almost all my issues through just finding the right configuration options.

As of about Tuesday (2 days after the initial “commitment”), I was ready to call it “better than my existing system”. It had a few glitches that bugged me, but overall, it had many many more features than I was expecting. After watching Smallville Thursday night, I’m a total freak for MythTV. Smallville was basically the first “Production” show MythTV recorded for me. I had recorded “Medium” earlier in the week, and that served as a good way to feel out the interface. Smallville is the real test because it’s at position 1 in my TiVo (and MythTV) recording priorities. I had no irritations while watching it. Nicely done, MythTV.

Rewind to Saturday. Bryce and I spent about 12 hours straight digging through KnoppMyth both on his new system and my machine that I brought over to his place. By the end of it, I had entirely reinstalled my system with Debian Sid, and installed the most current ivtv drivers, with the apt-able myth binaries. We had figured out how to get KnoppMyth running with the newer tuner chip, but Bryce’s HD audio card wasn’t supported in that version of the kernel. Let me just say, everyone should just start with the latest ivtv driver. It detected everything correctly right off the bat. On Sunday, Bryce installed Gentoo, and got the latest ivtv, etc, everything was happy.

Early this week I toyed with the remote control settings, and discovered a whole mess of MPlayer commands I didn’t know about that let you control playback speed (including fast audio!), OSD text (so I can have a visible indication that I’m paused), etc. After restoring my other MPlayer defaults (16M cache, readable font, etc), MPlayer stuff was in great shape again, including DVD playback. I also programmed my spare TiVo remote to control my stereo power and volume. I’m down to 1 remote finally! (I was surprised that the TiVo remote programming codes aren’t online anywhere. The only guide seems to be in the TiVo itself.)

Yesterday I discovered MythWeb. I must have been blind to miss it before. That’ll teach me not to read the entire EFF MythTV guide first. Full scheduling, guide data, recorded show lists, and most importantly, the ability to adjust the keybindings for the various MythTV modules. In the MythMusic module, I was infuriated that “PgDn” would skip to the next song, instead of (wait for it) paging down in the list of songs. I just can’t understand why such a massively counter-intuitive setting is the default.

The commercial detection system is greate so far. It’s already painful for me to go back to using my TiVo where I have to press the “skip 30 seconds” button ten times to get past all the commercials.

Current issues:

• Interface is slow. Everything (especially the video browser) is slow to respond. I miss not having an audible notification that a remote button was received, but there should at least be SOME kind of visible change if a button is pressed.
• Intermitant crappy audio playback. Something goofy happens on playback sometimes where the audio is just totally trashed. I just have to quit the playback and try again. I wonder if switching everything to using ALSA would make things better.
• No way to record the same show with two priorities. I want to have “New Episodes Only” for “Stargate: SG-1″ at a high priority, but “Any time, any channel” for it at a very low priority. I haven’t figured out how to do this yet. I think I will have to write special recording rules for it in SQL somewhere secret.

Jimmac you are my hero! I was just complaining about needing to tag a lot of my early music rips so they would be sensible in MythTV (and on my iPod), and poof, there’s the answer in my RSS feed reader. tagtool is exactly what I was looking for. It’s going to take me about 10 minutes to tag a little over 5000 mp3s. And for 9 minutes and 30 seconds of that, I’m going to be looking up publication years, which is the only thing not in the pathname that’s going to get sucked into the ID3 tag. Great, great tool!

As Bryce has already detailed, a few of us have started examining MythTV. While TiVo has served me pretty well, it sounds like now that the recording restriction flag has been tested it won’t be too long now before the content providers start actually using it. Supposedly the latest alarm is unfounded, in that it was a mistake. Whether it was a test, a mistake, it does show that my TiVo is now prepared to cripple itself.

MythTV has come a long way now. It seems that it’s a viable alternative for a DVR. Within our MythTV Club, we’re each going to have different problems. Doug will be fighting DVD playback via his PVR-350, Bryce will be playing with double recording sources, and I will be fighting with the video library playback capabilities. The EFF has a great write-up on installing MythTV for broadcast HDTV support. I think HD broadcast recording is going to come pretty late in our designs.

My existing video navigator (not the TiVo) is a very simple filesystem browser that just launches “mplayer” or “xmms” on the files it finds. (Holy crap, I set that up in 2002?!) MythTV’s “mythvideo” plugin is a much smarter browser for both video and audio, but its interface is wildly different from my current system. Especially for audio, which is very ID3-tag heavy. Very few of my early CD rips have any ID3 tags at all. The built-in MythTV player is also very very broken for seeking, and for unusual formats. I have a lot of movie trailers in Quicktime that MythTV refuses to play. I suspect that the video library issues will be the easiest to solve, though, since I can select a different viewer, etc. All that I have set up already on my existing navigator.

The nice thing about MythTV’s video/audio browser, though, is all the metadata support. I can pull down DVD and CD covers, etc. It’s way prettier than my existing system too. My goal is to get MythTV set up on my existing TV computer this week. Once it has replaced the filesystem browser, I’ll have the same functionality as the old system, and gain the ability to move on to real DVR activity in the future. I want to match my TiVo’s recording schedule, and then probably run in parallel until TiVo really gets DRM going.

I’m curious to see what the Netflix/TiVo joint venture is going to look like. I suspect the recent DRM testing is for the Netflix offering. If that’s true, I’ll wait until I see it, and then cancel my service. I’d really like to support TiVo, but not if they’re trying to cripple my time-shifting/long-term storage capabilities. I wonder how long it’ll take me to kernel-monte the TiVo into a MythTV frontend. All I really need is a cross-compiler. Sure would be nice to reuse my TiVo as a MythTV frontend if it comes to that.

I was able to get my hands on a CVS camcorder this past week. The unit is very cool. It doesn’t have any features of a “real” video camera, but I view it like a super-version of my Digital camera which can take 15 seconds worth of video at a time. This one is only limited by available storage (about 20 minutes). The resolution can be turned up to 640×480, too.

Initially I couldn’t get a cable built for it. I had a spare USB cable and a spare Palm Pilot cable to merge, but after soldering it all together, the device wouldn’t take an address (but my laptop noticed it was getting plugged in). After a hint from Linus (”plug-ins are detected via power-draw, everything else is over the data lines”), I realized that perhaps the USB data wires were reversed (the cable has got to flip them on at least one end of the cable… I just got unlucky).

The software for downloading the videos had been ported to Linux, and I worked on some more code to have a one-shot ability to download all the videos from the camera. Now I’ve got autoconf/automake working for it, and have gotten myself added to the Saturn Tools project where we can all work on the code through SourceForge’s CVS. Much easier than endlessly trading patches. :)

So far, I’ve managed to wreck the splash-screen image during camera bootup. Something is busted with the usb_bulk_write calls, and the device stops taking writes after 4k worth of data. Hopefully I can get that repaired so I can be greeted by the Laughing Man. I’ve also got a JPG all ready to replace the logo screen, “powered by pure digital technology”, with my oh-so-clever “powered by pwnd digital technology”.

I’ve just published srcfilter. It runs under Wine, which is the only reason I kept working on it. Coding in the Windows style has this sickening allure, like an overly sweet smell. I know I don’t want it, but it seems so tasty. (Oh, DWORD, how I hate thee. All your instance names start with “dw”.)

Anyway, srcfilter uses the DirectShow API to initialize a SourceFilter from a DLL you supply. It then pulls a specified source file through the DLL, writing the output to a file. Similiar things can be done with the GraphEdit tool that comes with the DirectX SDK. If you wanted to set up a Graph that uses a SourceFilter to read a file and a Dump sink to write output, you can use srcfilter to do the same thing. And you wouldn’t need Windows to do it.

Well, in the last few weeks, after finally getting MPEG2-PS streams out of my series 2 TiVo, I’ve been actively using gopchop again. It had stagnated, but it seems that people with hardware MPEG2 encoders continue to use it. I should have realized sooner that they are gopchop’s audience. It’s traditionally been a rather fragile bit of code for software-generated MPEG2-PS streams, but pretty stable for the more regular hardware-encoder streams. Between my recent increase in its use, and two large patch bundles I was sent, I’m trying to pick up development again.

One thing that has been particularly frustrating when dealing with MPEG2 video has been dealing with the rendering of the pictures. MPEG2 stores its pictures “out of order”. And by “order” I mean display order. They’re in order for decoding, but not for display. libmpeg2 handles all this for me, except for the part where I need to stop on a specific picture. For an accurate GOP splitting UI, I need to display the last picture of a GOP. Depending on the stream, libmpeg2 may not have rendered the last picture, since it’s waiting for more B or P frames before it knows the decoding is done. (For example, B frames depend on frames in the future, so you can’t render them until you see further down the stream, etc.)

My plan to deal with this is to force-feed libmpeg2 with End-Of-Stream packets, and not render stuff until I’m on the picture I want. For example, to display the initial B or P frames from a GOP, I may need to process the entire prior GOP first. For some types of streams, this may end up being very CPU expensive, since they may have hundreds of pictures in each GOP. But, this is why MPEG2 editing is hard. :)

I’m surprised that anyone still uses referer headers as a “security” measure. I’ve come across this several times recently. I’ll select a URL out of firefox, and paste it onto a curl -O command line, only to end up with a 0-sized file. And usually if I just add -e [site URL] to the command line, poof there’s my file. Most recently, I found this when trying to download the freely available Nine Inch Nails samples.

Seriously, what’s the point of doing this test? I don’t understand at all. If you want people to download a file in their web browser, do you think they can’t figure this out?

Well, it doesn’t actually make a lot of sense to figure out the encryption used by the TiVoToGo DLL, since TiVo could at any moment change it, the TiVo box’s routines, etc, and force all of us to upgrade. This ends up turning into an arms race, like iTMS. So, a better approach is to just use the DLL itself. Wine and Mplayer both have perfectly well written DLL loaders. All that has to be done is write a DirectShow Pull Pin implementation, following all the public published specs.

After I found some details on the .tivo file format, and cooked up some code to produce the same output. So far, there isn’t a lot of detail. It’s mostly just strange stuff. mpegcat finds the MPEG2 stream right where it should be, but supposedly the non-I-frames appear to be encoded.

All the files I’ve downloaded from my TiVo have the same “fingerprint” listed in the XML header, and a different “salt”. The interface for downloading recordings is very simple. Connect to https://[tivo IP]/ and it’ll prompt you for your username and password, which is “tivo” and your “Media Access Key”. After that, you’re all set; browse away. Download speed is rather slow: 500kB/s. Better than 2400 baud. :)

Last week a friend of mine told me about the Java HMO (Home Media Option) server. I had no idea it existed and immediately got it set up. This provides your TiVo with a place to get MP3s, streaming audio, and images over your local network through whatever other computer you have around to do the work for it. The regular Windows TiVo HM “Desktop” software isn’t nearly as cool. With the Java HMO, you can get movie listings, web cams, etc. It’s terrific.

Anyway, this was supposed to be about code, but it’s more about a growing itch to patch. It seems that the “Internet Images” module is either very fragile and/or doesn’t like having spaces (or maybe commas?) in the image titles (which it uses for caching in /tmp/javaHMO.cache/)

The overall system is a great design, and I’m currently enjoying watching some great webcams in places that are in daylight while I’m in the dark:

Antarctic Gentoo Penguins
Hanmer Springs, New Zealand
Tembe Elephant Watering Hole, South Africa

I can see elephant butts! :)

TiVo published their networking specs, but only included the broadcast and “desktop” specs. I think it would be cool to have the TiVo share movies with my computer too. Would be great for movie trailers and DVDs. My DVD player is in the other room, so I can’t watch movies in the TiVo room (at least until NetFlix and TiVo get their movie-delivery system finished). It still might be tricky to resample the DVD on the fly. TiVo normally records stuff at half-resolution, so dealing with a full DVD MPEG2 stream would be exciting to say the least. I’ll just have to see what’s on the wire between two TiVos configured to record shows from eachother.

(Edit: Holy crap! My wish is granted in the TiVo->PC direction, at least!)