ShrimpWorks

Heh, I guess there are already plenty of tools out there which do this sort of thing already (never seen them personally, but then I’ve never looked either, heh), but this only took me half an evening to throw together anyway.

Basically, it’s a Python (uses youtube-dl) and PHP-powered web-based YouTube video downloader and converter, you just stick in the URL to a YouTube clip you want to save, and it will download it and offer it for download as an MPEG which you can save on your PC and play in all it’s low-quality glory whenever you want.

Basically it automates the following, which can be run on any Linux PC:

# youtube-dl.py -o myvid.flv http://www.youtube.com/watch?v=123abc # ffmpeg -i myvid.flv -ab 56 -ar 22050 -b 500 -s 320x240 myvid.mpeg

As an added benefit, it stores a complete history of downloaded clips, so you and others can re-download them at any time without having to do the whole fetch/convert process over again. Plus it uses a nifty fake AJAX waiting effect :P.

Requires Linux with ffmpeg, Python 2.4+, and PHP 4.3+.

Hmmmmm, long time no update. That’s not to say I haven’t been busy recently.

Last month, we released “UnWheel R5”, which seems to have become the (hopefully) final release. I’m pretty happy with it at the moment, all the major bugs are gone, multiplayer is working wonderfully and the online record system is churning records around at a mean rate (and those records still need a monthly rotation system applied, so still some work to be done there). I still haven’t decided if I want to do this all again in Unreal Torunament 2007 or not :).

Elsewhere, I’ve been playing around with DynaBar, and it’s grown a lot. The plugin system has been tweaked to allow better customisation options from the developer side, as well as having options added to improve the user interaction side of things. There are a whole crapload of other options available as well, multiple layers (supporting PNG graphics with alpha transparency), different scanline styles, text prefixes and suffixes, better caching options, etc. In addition, you can choose to have the background be a gradient blending between any two colours, horizontal or vertical, and you can create “groups”, which is a bunch of userbars animated (with fading/blending between bars), and they all remain fully dynamic. Speaking of dynamic, I’ve also added a whole load of plugins, from XFire, to more Last.FM options, to Battlefield 2 and TrackMania, and even RSS headlines and live game server status via Qstat.

I’ve put up a test system here as a sort of sandbox, so feel free to try out all the options and plugins, and if you have any suggestions or ideas for plugins, please let me know. In addition to the designer, there’s a browser available, which lets you easily build the animated groups mentioned above. Also, it all works with Internet Explorer now, which I didn’t bother fixing with the previous version (wasn’t meant to be such a “big” project :)). Source code package will be available as soon as some more testing is done.

In addition, I’ve been re-writing my online Dosage-powered comic viewer - Injector

• again, this time it’s going fully “Web 2.0” (ZOMG!), so everything’s quite nice and quick. This project still needs a bit of work on the administration and installation side of things before it can see a release.

Aside from all that, I’ve also been slowly building a new UnrealZA site, using the Python-powered Django framework. It really is a wonderful thing. Please excuse me for a minute while I run away from a horde of crazy, twisted, Nevow fans (among others). Anyway, I’ll happily recommend Django any day of the week to anyone looking for a Python web framework.

I’ve also decided I don’t like the look of this site anymore, so I guess that’s another thing to go on my to-do list for the near future.

Alright, so I’ve been getting more and more spam in recent weeks, and they’ve been getting harder and harder to build basic filter rules for.

My mail works in a pretty round-about way:
I have multiple POP accounts all over the place, which have sort of accumulated over the years. It becomes a bit of a mission to always set up and check all these accounts, so what I have now is a small Python script that connects to each of the servers, grabs the mail, sorts them based on some simple filters (like, containing a [mailinglist] type subject), and places them within a Maildir structure based on that sorting. In addition, it does the same thing for deciding if it should delete a message - extremely basic spam filtering rules can be set up to check out certain headers for possible spam flags, etc.

The downloaded mail is then served via IMAP, using the Dovecot mail server. The great thing about that, is then every time I re-install any of the machines I use for mail access, or install a new one, I instantly have all my neatly sorted folders, all my mail from all my accounts, and only one IMAP account I need to set up.

Anyway, basically, the spam filtering of the above system was rather lame, so I went on the hunt for something a little more useful. Enter SpamBayes - a mail proxy application written in Python.

It’s already “in Debian”, so installing was as fun as always (aptitude install spambayes), after which I only needed to start the service, and then it’s off to a browser to configure it. Actually there wasone step before that - since I’m running this on my server, and SpamBayes is meant for use by a single user on their own PC, it doesn’t allow connections do it’s browser-based configuration from other hosts. Which is a bit of a problem when running a server which I have no interaction with beyond a command shell. Thanks to Lynx I was able to configure it to allow connections from my local network.

For starters, you need to tell it which POP3 servers you want to connect to, and assign local ports to each one, which will be stand-ins for port 110 when connecting to servers. The interface for this is a bit troublesome however, requiring you to enter each server into a single input field, separated by commas. The associated ports for each server are then entered into another input field in the same manner. It took me a while to get both the fields synced due tot he number of servers I intended using.

Next up, I fed it a few emails for training (saved emails out of Thunderbird as EML files, and these can be uploaded to the server for training via the browser interface) both ‘ham’ and spam.

Once it knew the basics, I simply updated the list of servers in my Python script to “localhost”, and whichever port each one was set to. Shortly thereafter, mail started passing through the system. Most of it was identified as “unsure”, as it hadn’t seen enough examples of ham or spam yet. Quite smartly, it keeps a record of each message that’s passed through, and you can easily train ham or spam from these.

Around 50 mails later, it was identifying almost every message perfectly. I’m going to leave it running for a day or two more, training everything that arrives, then I’ll just add a single filter to my mail fetching script, looking at the “X-Spambayes-Classification” header for “spam” (delete), or “ham” take no action.

I’m quite happy with this setup, looks like it’ll work quite well :D.

Guess it shouldn’t have taken so long for me to get around to doing this, but at least it’s done now.

Attached to this post you’ll find a zip file, containing a small example application which allows you to spawn PyODE physics-enabled cubes with the middle mouse button into a PyOgre world. You can then bounce and roll the cubes around by holding the left or right mouse buttons.

The code is fairly straight-forward, and I’ve included quite a number of comments. Should be easy enough to follow what’s going on if you’ve been through the PyOgre tutorials.

A note of performance and stability - you can safely spawn loads of cubes as long as there are not too many collisions going on at once (after around 50 cubes, things start to get really sluggish if there are too many inter-cube collisions going on). In practice though, I doubt you’d need that many collisions happening at any one time. Also, If you make a large pile of cubes, lift them all up, and let them fall down together, it seems to bomb out as there are too many collisions happening when they all land on top of eachother at once. I haven’t debugged this very much, so I’m not sure yet if it’s a ODE limitation, or something bad I’m doing in the code. If anyone works it out, I’d be interested to know.

Please don’t ask for advice on stuff like per-polygon collisions, terrain collision and the like, I have not really messed with this beyond the state of this example. Once you get the basics going after checking out the example, I’m sure a few questions shot off at the PyOgre Forums would turn up more useful results than asking me :).

Have fun ;).

My idea for implementing non-physics physics into my little game framework didn’t work out too well, so I gave in and took a look around for options.

It seems only ODE is available to Python, via PyODE. Not many [open source] physics engines seem to have Python bindings, which I find rather odd.

As it turns out, it isn’t actually all that of a mission to get ODE and Ogre working together, and the results I’ve got so far are quite acceptable. I can spawn loads of cubes (of varying sizes) and throw them around the scene and they bounce and jump around in a suitable fasion.

I haven’t tried with balls or polygon-accurate stuff yet, that’s next on the to-do list. I also intend writing a short how-to for PyODE and PyOgre integration at some point, as I was a little confused to start with, not knowing quite where or how to begin, and there is no PyODE/PyOgre example code floating around to reference.

EDIT: Example using PyODE and PyOgre now available -

Yes, so everyone’s obsessed with checking their BF2 stats these days ;).

Anyway, I wanted to give my Supybot IRC bot (“Nooblet” on Shadowfire) the ability to check my own and other people’s stats whenever they felt like it. I came up with something like this:

import urllib2
from string import split
from time import time

# the columns you want to request data for. comma-separated string.
info = 'per*,cmb*,twsc,cpcp,cacp,dfcp,kila,heal,rviv,rsup,rpar,tgte,dkas,dsab,cdsc,rank,cmsc,kick,kill,deth,suic,ospm,klpm,klpr,dtpr,bksk,wdsk,bbrs,tcdr,ban,dtpm,lbtl,osaa,vrk,tsql,tsqm,tlwf,mvks,vmks,mvn*,vmr*,fkit,fmap,fveh,fwea,wtm-,wkl-,wdt-,wac-,wkd-,vtm-,vkl-,vdt-,vkd-,vkr-,atm-,awn-,alo-,abr-,ktm-,kkl-,kdt-,kkd-'

# this is my BF2 ID. You can also query the stats server with "nick" rather than "pid", but I've had problems with some characters
pid = '43595724'

opener = urllib2.build_opener()
opener.addheaders = [('User-agent', 'GameSpyHTTP/1.0')]  # otherwise GameSpy's servers will block your request
webData = opener.open('http://bf2web.gamespy.com/ASP/getplayerinfo.aspx?pid=%s&info=%s&nocache=%s'%(pid,info,round(time()))).read()

statsData = split(webData, "\n")

cols = split(statsData[3], "\t")
data = split(statsData[4], "\t")

stats = dict(zip(cols, data))

# you now have a nice dictionary with a few hundred bits of stats data.
print "%s has %s kills and %s deaths and a score of %s" % (stats['nick'],stats['kill'],stats['deth'],stats['scor'])

Thanks to korpse and mithrandi for showing me the “zip” function. It takes the list from the first parameter, and uses those values as keys in a dictionary, the values from the second parameter are then used as values in that dictionary. I was using map(None, keyList, valueList), but zip seems cleaner.

Anyway, if you’re looking for more info on stats querying, try the BF2 Technical Info wiki, or check out SaladFork’s Guide to Creating a BF2 Stat Signature - although it’s in PHP, he does give a nice list of column names and their meanings, you can also grab lists of ranks, weapons, vehicles, etc.

Update: Also see: Battlefield Stats in PHP

Updated: Since this was written, some things changed with the stats system, and the GameSpy application requires you to pass a bunch of columns you want info for. This can help customise the data you get back, so you only request what you need. I’ve included all the columns in the info variable, which you can customise. Make sure it contains only valid columns, or you won’t get any data back at all.

As above, for info on what all the columns etc. mean, check out the BF2 Technical Wiki.