05 Jan 2004
MacRaven is On

I am happy to report that my system is fully operational. I have all the attributes of Radio and none of the slow silliness that seemed to plague their servers and the local system.

I ditched AmphetaDesk as the aggregator and went to Blagg, with some modifications. I did this for a number of reasons:

  • Amphetadesk was not really an "aggregator" - it read and stored the latest RSS feeds available from the specified sources. If you didn't stay on top of things and read it religiously you missed a bunch of news.
  • It wouldn't allow you to delete a particular item - and if you hacked it to delete items, you have to hack it again to keep it from adding them back in on the next run
  • Worst of all, AmphetaDesk was fully standards compliant. Why was this a bad thing? Because the rest of the world treats standards more like guidelines than hard and fast rules. And consequently the feeds would break: especially the BBC and Slashdot. Totally annoying ...

I will have to say that AmphetaDesk was very nice code: well thought out, structured so folks could get at the guts easily enough and generally well commented, well written code. It would serve very well as a desktop news aggregator - which is, of course, what it really is! It didn't do so well in duplicating the functionality I've become accustomed to in Radio - it just wasn't designed to do those kind of things. I was inspired to purchase the book by one of the authors of AmphetaDesk:Spidering Hacks - it really got me going on the redesign of the aggregation system. Very useful book - frankly, all of the "hacks" series from O'Reilly that I've acquired (covering Google and OS X as well as this one) have been useful to me.

Basically, I have three machines that I run on a regular basis. Big Mac is the Power Mac G4 - it runs my local mail server, web server, and other things, as well as serving as my wife's desktop platform for her digital photo work. I have my PowerBook up in the library, which is where I really like to work, and I have a Sony Windows XP Pro box on my desk, with all the associated crap piled on it for writing C++ in Windows. Radio was running on the Windows box - it has it's own built in web server, and I'd access it over the internal network via the browser. Once I decided to run Blosxom as the blog software, my first task was to write a little program (using Borland's C++ Builder) for Windows that would take the output of a Radio exporter script and let me add titles, categories, and otherwise clean things up in the code. I did a little hacking on the script itself, but most of the work was done thru my little program. (I'll be cleaning it up and posting it on my company server for download under the GPL as soon as I get around to it).

That got the archives over and taken care of - next I set up two copies of Blosxom - one local on BigMac, and one on the FreeBSD server that runs haxton.org. It took about ten minutes to get it running, and another 6 hours to get the "look and feel" the way I wanted. Blosxom is rather easy to write to: everything's a text file - you just use your favorite editor and drop it in place (I'm writing this in BBEdit on my PowerBook). So I post locally, and then use rsync to mirror the local directories to the server, once an hour with a crontab entry. Instant upstreaming.

Futzing with the aggregator was no fun - I couldn't get AmphetaDesk to quit pooping out on XML errors, even though I had an acceptable posting scheme concocted. So I looked at other options and found Blagg. Blagg, in it's normal state, just read RSS feeds and dumps them into a Blosxom blog. I set up a second Blosxom blog locally on BigMac (I even renamed the Perl program to keep things straight) and hacked Blagg with a bit o'Perl to get it to put the articles from each feed into a separate directory, under the main dir of my new "aggregator blog". I added this to my crontab to run hourly with the posting routines, and I have a working aggregator.

But I still couldn't post or delete - and Blosxom didn't really like the idea of being fooled into going outside of what it thought was it's main data structure to edit entries. So I whipped up a PHP script to replace the Blosxom editing plugin, and added a delete feature while I was at it. I don't really "unlink" the entries - I just rename them, and then have Blagg check for the existence of either the correct name file or the renamed file. This prevents Blagg from aggregating again items I've already "deleted". Once a day, I got thru the directories with a shell script and delete any of the renamed files that are over two weeks old (some of the blogs and news sources I have keep stuff pumping thru RSS for that long).

The PHP posting program brings up an editing screen for me to add my comments to the item, change the title if desired, and put it into a category (I even went so far as to automatically read my categories - the directory structure of the main Blosxom installation - and fill in the HTML select options automatically).

It looks a whole lot like Radio on this end, it's easier to configure and maintain, and I don't have to shell out $40 to keep my blog next year. About the only thing I haven't got added in yet are email notification of writebacks, and pinging weblogs.com on an update. But those are both Blosxom plugins, so adding them in should be reasonably easy.

All in all, it's been an interesting trip. I sure did miss blogging during the transition - could you tell? (Just count the number of posts I made yesterday afternoon).

00:00 /Home | 0 comments | permanent link