ShrimpWorks

Yes, so everyone’s obsessed with checking their BF2 stats these days ;).

Anyway, I wanted to give my Supybot IRC bot (“Nooblet” on Shadowfire) the ability to check my own and other people’s stats whenever they felt like it. I came up with something like this:

import urllib2
from string import split
from time import time

# the columns you want to request data for. comma-separated string.
info = 'per*,cmb*,twsc,cpcp,cacp,dfcp,kila,heal,rviv,rsup,rpar,tgte,dkas,dsab,cdsc,rank,cmsc,kick,kill,deth,suic,ospm,klpm,klpr,dtpr,bksk,wdsk,bbrs,tcdr,ban,dtpm,lbtl,osaa,vrk,tsql,tsqm,tlwf,mvks,vmks,mvn*,vmr*,fkit,fmap,fveh,fwea,wtm-,wkl-,wdt-,wac-,wkd-,vtm-,vkl-,vdt-,vkd-,vkr-,atm-,awn-,alo-,abr-,ktm-,kkl-,kdt-,kkd-'

# this is my BF2 ID. You can also query the stats server with "nick" rather than "pid", but I've had problems with some characters
pid = '43595724'

opener = urllib2.build_opener()
opener.addheaders = [('User-agent', 'GameSpyHTTP/1.0')]  # otherwise GameSpy's servers will block your request
webData = opener.open('http://bf2web.gamespy.com/ASP/getplayerinfo.aspx?pid=%s&info=%s&nocache=%s'%(pid,info,round(time()))).read()

statsData = split(webData, "\n")

cols = split(statsData[3], "\t")
data = split(statsData[4], "\t")

stats = dict(zip(cols, data))

# you now have a nice dictionary with a few hundred bits of stats data.
print "%s has %s kills and %s deaths and a score of %s" % (stats['nick'],stats['kill'],stats['deth'],stats['scor'])

Thanks to korpse and mithrandi for showing me the “zip” function. It takes the list from the first parameter, and uses those values as keys in a dictionary, the values from the second parameter are then used as values in that dictionary. I was using map(None, keyList, valueList), but zip seems cleaner.

Anyway, if you’re looking for more info on stats querying, try the BF2 Technical Info wiki, or check out SaladFork’s Guide to Creating a BF2 Stat Signature - although it’s in PHP, he does give a nice list of column names and their meanings, you can also grab lists of ranks, weapons, vehicles, etc.

Update: Also see: Battlefield Stats in PHP

Updated: Since this was written, some things changed with the stats system, and the GameSpy application requires you to pass a bunch of columns you want info for. This can help customise the data you get back, so you only request what you need. I’ve included all the columns in the info variable, which you can customise. Make sure it contains only valid columns, or you won’t get any data back at all.

As above, for info on what all the columns etc. mean, check out the BF2 Technical Wiki.

I decided it might be a good idea, from a debugging and administrative point of view, to save the reports people were viewing in my application at work. Since users are distributed all over the country, and I have to communicate with them over the phone with on-the-spot problems, it’s hard and very time consuming to get them to tell me all the parameters etc they’re using to generate a report which they are having problems with (mostly, the data looks like something they weren’t expecting).

Anyway, I thought saving the exact report they are viewing is much easier for me to simply call up while someone’s on the phone, than doing the whole ‘which options did you use?’ thing.

Option one was saving each report to a PDF file on the server, but, that’s pretty inflexible, and also, people can view more than just PDF files with my reporting options (emails can be send direct, zip files can be downloaded, and data can be downloaded in csv format), so that wouldn’t always work out.

So I looked into serializing the actual report object used to generate any of the above report types, since that object contains absolutely all data, totals, titles, info, etc.. Unfortunately, PHP’s built-in serialize() and unserialize() functions don’t work too well with complex data structures (arrays within classes, multi-dimensional arrays, etc, etc), and I couldn’t really work around all that without writing a few GBs of code onto my report class.

SO, I turned to XML. As it turns out, the PEAR ‘suite’ of scripts contains a rather useful XML serialization class -XML_Serializer, which can turn any data structure into an XML string, and for reading those strings back to usable PHP variables and objects.

It’s really quite simple:

$object = new SomeClass();
$object->var1 = "Hello World";
$object->etc();

$serializer = &new XML_Serializer(array("indent" => "  ", "typeHints" => true));
$serializer->serialize($object);
$xml = $serializer->getSerializedData();

$xml now contains the full definition of $object in XML. You can write $xml to a file, save it in a database, whatever you like.

You can then come back later, and load the file/database record/etc into a string, and deserialize it…

$unserializer = &new XML_Unserializer();
$unserializer->unserialize($xml);
$object2 = $unserializer->getUnserializedData();

echo $object2->var1;
$object2->etc();

Quite fun actually :D. You can obviously do a lot of error checking, and other things, but that’s just the basics.

Ok, so this is such an insanely simple thing to do. I once tried looking for a PHP class or package which could write Excel .xls files from data in an array. Well, I found a really ugly class which simply saved a .csv file as .xls. Anyway, I ended up writing my own 4-line function for that:

function arrayToCSV($data) {
    $csv = implode(',', array_keys($data[0])) . '\r\n';
    for ($i = 0; $i < count($data); $i++) {
        $csv .= implode(',', $data[$i]) . '\r\n';
    }
    return $csv;
}

You’ll have to do your own validation for empty arrays and things elsewhere. It returns a string with each record from the array on it’s own line, separated by commas. $data is expected in a format something like this:

$data[] = array('name' => 'Bob', 'age' => 12);
$data[] = array('name' => 'Jack', 'age' => 15);

Which looks something like this: print_r($data):

Array
(
    [0] => Array
        (
            [name] => Bob
            [age] => 12
        )
    [1] => Array
        (
            [name] => Jack
            [age] => 15
        )
)

If you’re using ADODB, you can use this little function to convert the recordset to a suitable array (there are some built-in functions to convert recordsets to arrays, but they all end up giving you pretty useless data).

function rsToArray($rs) {
    $arr = array();
    while (!$rs->EOF) {
        $arr[] = $rs->GetRowAssoc(False);
        $rs->MoveNext();
    }
    return $arr;
}

Since I’m bored at the moment, I might as well post some code :D

PHP has 2 nice functions, explode() which breaks a string into an array using a separator, and implode() which takes an array and glues it back into a single string using a separator. I use it regularly, so eventually ended up needing one for Delphi too.

I cannot take full credit for the Explode function however, I found it somewhere and can’t for the life of me remember where that might be. It is somewhat modified from the original though…

Also note that these work with TStrings, rather than arrays.

function Explode(const str: string; const separator: string): TStrings;
var
  n: integer;
  p, q, s: PChar;
  item: string;
begin
  Result := TStringList.Create;
  try
    p := PChar(str);
    s := PChar(separator);
    n := Length(separator);
    repeat
      q := StrPos(p, s);
      if q = nil then q := StrScan(p, #0);
      SetString(item, p, q - p);
      Result.Add(item);
      p := q + n;
    until q^ = #0;
  except
    item := '';
    Result.Free;
    raise;
  end;
end;

function Implode(const Strings: TStrings; const separator: string): String;
var
  i: Integer;
begin
  Result := Strings[0];
  for i := 1 to Strings.Count - 1 do
    Result := Result + separator + Strings[i];
end;

I’ve been adding some nifty things to my PHP project at work (cellphone starter pack invoicing, usage tracking [big bro is watching you earn him money every time you use your pre-paid airtime :P], etc). It’s got a lot of pretty nice features. Using client-side JavaScript, it actually pretty much behaves as a normal desktop application, only it lags like hell when you visit other pages (slow web server :P).

Anyway, I can’t claim I’ve written all the snazzy features myself unfortunately. Due to time and pressure I’ve been relying on good old Open Source to help me out. There’s actually a helluva lot of stuff out there to make things easier for just about anything you need to do…

• ADOdb - http://adodb.sourceforge.net/

  Obviously the first thing to get going on any database-aware application, is to actually interact with the database. PHP has an Interbase module, with a whole bunch of Interbase functions. These all work fine, and there’s really no problem with them. ADOdb however makes working with SQL databases so much easier. No need to remember PHP’s randomly named module functions, and it gives you access to a huge assortment of database drivers (of corse all dependent on normal PHP modules). It’s even available for Python now!

• Smarty - http://smarty.php.net/

  Next you’d presumably like a fancy presentation for your application. You have three options here - painfully output every single line of HTML manually through your code, create yourself a template engine/language, or use Smarty. Smarty is a Template Engine, which lets you define “.tpl” files which are bascally just HTML files with a couple of variable placeholders and smarty function calls. But it’s not just the simplicity of creating a Smarty instance, assigning variables to it, and then just calling display(), it’s the functions you can put into your template files. Looping through customer listings, building tables or drop lists, check lists, etc, etc, etc. One of the most useful things I’ve found is the ability to define your own smarty functions, allowing you to add custom functionality to your templates and the output they generate.

• PHP pdf / PDFClass - http://ros.co.nz/pdf/

  One of the requirements of this project was that reports had to be generated in user and printer friendly PDF format. At first I looked into the option of having Smarty generate tabular reports, and get an HTML to PDF processor (there are several PHP options available) to turn my HTML into the PDFs required. Unfortunately, for the most part, these converters are heavily buggy, and didn’t give me very much control at all over the output (ok, I had full control over the output, but things like page headers and footers, page numbering, etc, etc are required for nice reports). Anyway, so I set about using PHP’s PDF functions. Unfortunately, that turned out rather risky across different versions of PHP, and was generally a pain to work with. Enter this package. It doesn’t require any of PHP’s PDF modules, so it’s free from cross-version and cross-platform bugs and stuff. It also has a bunch of nice functions for headers, footers, tables, etc. It also provides the option to save the output to disk, output direct to browser, or plop the output into memory where I can play with it. I’ve built a very nice reporting class around this package.

• Code 3 of 9 Barcode Generator - http://www.sid6581.net/cs/php-scripts/barcode/

  Another requirement of the system is the ability to group multiple items into a single item. From the code side of things, it’s fairly simple, but to users, trying to manage these million boxes of starter packs and remembering or creating their own codes for these boxes would obviously be rather difficult. Anyway, I though it’d be nice to offer a barcode people can print and stick to the boxes. Enter this little script. Couldn’t be easier to use, and with the help of a little Javascript, users can even scale the barcodes up and down (by dynamically reloading the image, not just changing the dimensions and possibly corrupting it) before printing.

• PHPMailer - http://phpmailer.sourceforge.net/

  I’m using this for sending my PDF reports via email. It allows you to easily attach files, or file content (so I don’t need to save the PDF’s to temp files before attaching, just attach the output directly from script). Also supports SMTP, so no need to rely on PHP’s mail configuration.

I’m also using a nifty little ZIP lib that allows me to zip my in-memory PDF reports, and send them direct to the browser for downloadable reports (since a normal PDF will open in the browser). Unfortunately there’s no readme or author URL in the source :)

I did get it from PHPClasses.org though, which has quickly become by first stop for anything I need in a hurry that I couldn’t be bothered, or don’t have time to write myself. Nice rating system and “top 10"s filter out the good stuff instantly, making finding stuff really simple. I highly recommend it to any PHP developers.

Dunno if anyone would have noticed but the site was blinking on and off last week, with dynamic DNS issues.

I’ve been using an application which runs as a service on my Windows machine, but it seems to often give up if it can’t get a new IP or the update fails, and sometimes it just doesn’t bother even trying :-).

Anyway I slapped up a quick Python script to be run from a cron job at 5 minute intervals to check a website which provides my IP (like http://checkip.dyndns.org), grab the first IP it finds, and updates my ZoneEdit account with the new IP.

Seems to have been running reliably the past few days now.

I’ve dumped it on the Files page if anyone would like to give it a go. It’s set up for ZoneEdit, but I’m sure it’s easy to adapt to other services as well.

*sigh* Look at me starting a million projects and never finishing any of them :P.

Well this one, I fully intend finishing at least. It’s a wxPython GUI front-end for Dosage (http://slipgate.za.net/dosage/) which basically has the sole goal of making managing your comics easier for the mouse potatoes out there (and makes Dosage more accessible to the average Joe user who doesn’t want to mess around with command lines and stuff).

At present, it doesn’t do much, it simply builds a nice tree of comic modules (virtual modules are displayed as branches) and allows you to add those to a list (which basically makes up your ‘subscriptions’), and lets you select one or more of those and have Dosage download the latest strip for them.

Seems I’ve spent most of today writing Dosage (http://slipgate.za.net/dosage/) modules - managed to find a whole load of comics, and ended up making 10 new modules.

It’s surprising how many comics have their main images sliced in half somewhere midway through the image (so it ends up being 2 images which Dosage obviously won’t handle), I found some that even have each panel in a separate HTML table cell :-/.

Not sure if it’s bad web design or whatever, but it’s a pity since they’ll never make their way into Dosage…

I’ve put my Python POP3 MailBox class up, it may be of some use to someone… Read the comments and stuff within the script for full details.

Download: http://shrimpworks.za.net/filemgmt/singlefile.php?lid=3

Well as expected, my little wxPython dict client is finished.

I decided to install the File Management plugin to let me put this stuff on here somehow… So check out pyDict in the “Files” link on the top-right of the site.

Or simply go here: http://shrimpworks.za.net/filemgmt/index.php

The ReadMe explains the basics of how it works and stuff…