ShrimpWorks

With all the talk of Unreal Tournament 4 possibly being cancelled one of these days, due to Epic’s runaway success with Fortnite, I’ve decided there’s really no reason to not be playing UT99.

Thus, we set about trying to run it on modern hardware, with a modern Linux installation.

As much as this is about setting things up on Linux, it’s also partially my own attempt at some knowledge preservation, as a lot of this stuff ends up being forgotten or lost over time (it’s been almost 20 years! a lot of the old sites and things you expect to find this info on simply do not exist anymore :()

This is part one of two, and will focus on installing and running the game using Wine.

Continue Reading ...

I recently wanted to set up a couple of rough monitoring services to keep track of simple server status, load, disk etc. While there are options available like Munin which can do this by installing agents on the machines to be monitored, I wanted something a little simpler and more portable.

I’m quite fond of the StatsD + Graphite + Grafana stack, which is quite easy to run thanks to Kamon’s grafana_grafite Docker image, and I realised you can actually quite simply write counters, gauges and timers to StatsD using nothing but the standard Linux tools nc and cron.

For example, every minute on each server being monitored, a simple cron job is executed which uses nc to write a bunch of information to my StatsD service:

#!/bin/bash

HOST=$(hostname)

STAT_HOST="statsd-host"
STAT_PORT=8215

# load average
echo "load.$HOST.avg:`cat /proc/loadavg | cut -d ' ' -f 1 | awk '{print $1*100}'`|g" | nc -w 1 -u $STAT_HOST $STAT_PORT

# memory
echo "memory.$HOST.perc.free:`free | grep Mem | awk '{print $3/$2 * 100.0}'`|g" | nc -w 1 -u $STAT_HOST $STAT_PORT
echo "memory.$HOST.bytes.total:`free -b | grep Mem | awk '{print $2}'`|g" | nc -w 1 -u $STAT_HOST $STAT_PORT
echo "memory.$HOST.bytes.used:`free -b | grep Mem | awk '{print $3}'`|g" | nc -w 1 -u  $STAT_HOST $STAT_PORT

# disk
echo "disk.$HOST.kbytes.total:`df -k --output=size / | grep -v [a-z]`|g" | nc -w 1 -u $STAT_HOST $STAT_PORT
echo "disk.$HOST.kbytes.used:`df -k --output=used / | grep -v [a-z]`|g" | nc -w 1 -u $STAT_HOST $STAT_PORT
echo "disk.$HOST.kbytes.avail:`df -k --output=avail / | grep -v [a-z]`|g" | nc -w 1 -u $STAT_HOST $STAT_PORT

# mail queues
for i in maildrop hold incoming active deferred bounce; do echo "postfix.$HOST.queues.${i}:`find /var/spool/postfix/${i} -type f | wc -l`|c"; done | nc -w 1 -u $STAT_HOST $STAT_PORT

It’s perhaps a bit inefficient in places, but gets the job done fairly well. One minute resolution may be a bit rough, but it’s sufficient for most of these data points which don’t change too dramatically over time.

Some other more specific variations include HTTP accesses, ping times, etc. Pretty much any parameter you can parse down to a single number can be published as a counter, gauge or timer to StatsD, and then neatly graphed over time.

This is a small follow-on on from the Kodi on Debian Sid guide I did earlier this year to get lirc (IR remote support) working once more, following an upgrade to version 0.9.4, which changes how the lirc services and configuration work (shakes fist at systemd).

After upgrading and following all the instructions in /usr/share/doc/lirc/README.Debian.gz, I was left with the problem of Kodi not responding to any remote input at all.

Firstly, I had to re-source my remote’s configuration (mceusb) from the lirc git repository. Place the *.lircd.conf file from there into /etc/lirc/lircd.conf.d/ and remove/rename other .lircd.conf files already in that directory.

Now, running irw and pressing some buttons on your remote should show you the button pressed and the configuration used.

Next up, Kodi fails to connect to the IR device. There are two trivial but non-obvious solutions:

Firstly, without changing any of the default configuration generated by the migration process outlined in the lirc README file, simply change your Kodi starup command as follows:

$ kodi --lircdev /var/run/lirc/lircd

Alternatively, you may change the lirc configuration, to put the device file back where Kodi expects it:

# in /etc/lirc/lirc_options.conf:
output = /dev/lircd

Then end result should be you happily continuing with your life.

I recently went through the process of reinstalling the media PC connected to my TV, which I use to run Kodi for movies and TV, and Steam in Big Picture mode, which allows me to stream Windows-only games from my desktop to the couch.

I thought it would be useful to describe my setup and the process to achieve it, in case anyone else is interested in creating their own custom Kodi/Debian/Steam builds.

Continue Reading ...

I’ve become fond of using nginx on my development machines, rather than a full Apache.

There are no explicit options built-in which allow something along the same lines as Apache’s userdir, however it’s easy enough to tweak the default configuration to support that behaviour without the need for external modules.

I also do some PHP dabbling from time to time, so need to enable that as well.

Install the required bits:

$ sudo aptitude install nginx php5-fpm

Configure nginx (the below is my customised and cleaned out server definition):

/etc/nginx/sites-available/default

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    root /var/www/html;

    # Add index.php to the list if you are using PHP
    index index.html index.htm index.nginx-debian.html index.php;

    server_name _;

    # PHP support in user directories
    location ~ ^/~(.+?)(/.*\.php)$ {
        alias /home/$1/public_html;
        autoindex on;

        include snippets/fastcgi-php.conf;

        try_files $2 = 404;
        fastcgi_pass unix:/var/run/php5-fpm.sock;
    }

    # PHP support in document root
    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/var/run/php5-fpm.sock;
    }

    # User directories in /home/user/public_html/
    # are accessed via http://host/~user/
    location ~ ^/~(.+?)(/.*)?$ {
        alias /home/$1/public_html$2;
        autoindex on;
    }
}

I also had to make a change to /etc/nginx/snippets/fastcgi-php.conf, to comment out the following line:

#try_files $fastcgi_script_name =404;

After restarting the nginx service (also make sure the php5-fpm service is running), you will be able to serve HTML and PHP files from your ~/public_html directory.

Recently, I’ve made the switch from KDE being my preferred Linux desktop environment/window manager, to i3, a tiling window manager, for both my work and private development environments (my home desktop is still Windows 7, since I do still game enough for it to become painful to dual-boot - so I do most of my development within a VM these days).

I really like it’s absolutely minimal approach - essentially it does nothing itself, it provides a simple window manager, and near limitless configurability. This has proven an excellent learning experience for me, since it’s forced me to get a lot closer to system components usually “hidden” behind sliders and widgets in KDE or Gnome, as well as a host of alternatives to applications those environments provide by default. It’s also resulted in a much cleaner and faster system, containing only the applications and services I actually want.

We recently installed fresh new desktop machines at work, so I thought I’d share some of my setup, in case it’s of some value to anyone else (and my own future reference!). The following steps assume you know how to operate a basic Debian system. I’m not going to go too deep into any usage details for i3 either, since there’s an excellent user guide and comprehensive FAQ system which should answer any questions you may have.

I’d also advocate using “aptitude” as an alternative to “apt-get” for all package installations, updates and removals.

The Basics

I always start off with a Debian “netinst”. Post-install, this provides an incredibly basic bare-bones OS with a few system utilities (during the installation, de-select the pre-configured “Desktop”, “Web Server”, “Mail Server”, etc. options, just keep the “Standard System Utilities”).

First thing to to after installing is install sudo and add your user to the sudoers group, to avoid having to be root to get things done. Now’s also a good time to install vim.

I also like seeing Aptitude’s “visual preview” of changes when doing package management, so to avoid having to call $ aptitude --visual-preview install ... on every invocation, we can edit root’s aptitude config:

/root/.aptitude/config:

Aptitude::CmdLine::Visual-Preview "true";

Upgrade to Unstable/Sid

Perhaps a bit reckless, but I’ve honestly never experienced any crippling issues running Debian Unstable (“sid”). You’ll only need to modify /etc/apt/source.list and replace references to “wheezy” or “testing” with “unstable” or “sid”, and disable the updates and security repositories, leaving you only the main deb and deb-src repositories (I’ve enabled non-free and contrib, since I want to install FlashPlayer and nVidia drivers later):

/etc/apt/source.list:

deb http://cdn.debian.net/debian/ unstable main non-free contrib
deb-src http://cdn.debian.net/debian/ unstable main non-free contrib

After saving the above changes, execute the following:

$ sudo aptitude update
$ sudo aptitude dist-upgrade

The dist-upgrade step will upgrade all installed packages to whatever’s newest in unstable.

Desktop Install

With the base system as up-to-date as it can be, it’s time to install the desktop environment.

$ sudo aptitude install xorg lightdm i3-wm i3status suckless-tools

After installation, I’d reboot and ensure a nice graphical login prompt appears. After login, you’ll be asked some initial i3 setup questions (which are easy to change later) and land in the default i3 workspace. Press Mod+Enter (Mod being whatever you selected in the aforementioned setup questions - likely “windows” key, or Ctrl) to open a new terminal window. It’s probably xterm, which is sort of OK, but I switched to lxtermial - it’s nice and lightweight but still has a fair number of configuration and convenience features (like URL detection - useful for IRC).

If you install another terminal, and opening more terminals results in more xterms rather than your installed terminal, do the following to set your preferred option:

$ sudo update-alternatives --config x-terminal-emulator

Desktop Tweaks

Before digging too deep into installing additional software, it’s a good time to configure some additional options to make life a bit more pleasant.

Look and Feel

In order to make sure your eyes are not offended by the default GTK theme which you may end up seeing a lot of, set up the GTK theme and icon theme:

~/.gtkrc-2.0:

include "/usr/share/themes/Adwaita/gtk-2.0/gtkrc"
gtk-icon-theme-name="Adwaita"

In addition, I found it a lot cleaner and space-maximising to disable i3’s window titles and thin it’s borders down, by addition the following to ~/i3/config:

new_window 1pixel
new_float normal

py3status

Install python-pip via Aptitude, and then $ sudo pip install py3status. I use py3status since it provides some nice additional modules, is more flexible, and is fully compatible with the default i3status configuration. It’s also a good time to check out the i3status configuration documentation and do some tweaks, since a couple of the default entries here are likely not too useful.

Wallpaper

Randomised (of fixed if preferred) wallpapers can easily be achieved by installing feh (which makes for a good i3-friendly picture viewer in general) then adding the following to ~/i3/config:

exec --no-startup-id feh --recursive --randomize --bg-fill ~/Pictures/wallpaper/

Incidentally, the imgur wallpaper gallery is a good place to find some wallpapers.

File Management

Sometimes a GUI file manager can be useful, and for this, a nice light-weight alternative to the bigger desktops’ Nautilus and Dolphins is PCManFM, installed as pcmanfm.

A nice companion application for (compressed) archive management is xarchiver. You may need to install additional tools (such as zip, unzip, unrar-free, etc, depending on the files you commonly work with).

Conclusion

The entire setup to this point should not have taken more than 1-2 hours, depending on download speeds (really, most time is spent just waiting for downloads…), so you can get this kind of environment running with minimal effort and downtime.

I haven’t included anything about multimedia, custom key bindings, lock screens, or others here, but there are loads of other resources around which can fill you in on those and the myriad ways you can configure your i3 environment.

Your next step, if you’re new to i3, should probably be to take a read through the i3 user guide, which is impressively comprehensive.

So, being stuck without access to Photoshop and my regular Windows PC has taught me a little about GIMP – basically it’s exactly the same as Photoshop with a less slick UI :D.

Also, since I haven’t really written anything tutorial-ish in many many years, so this is as much about brushing up on those skills as anything else.

This is a guide for quick and simple photo enhancement using GIMP.

Continue Reading ...

Alright, so I’ve been getting more and more spam in recent weeks, and they’ve been getting harder and harder to build basic filter rules for.

My mail works in a pretty round-about way:
I have multiple POP accounts all over the place, which have sort of accumulated over the years. It becomes a bit of a mission to always set up and check all these accounts, so what I have now is a small Python script that connects to each of the servers, grabs the mail, sorts them based on some simple filters (like, containing a [mailinglist] type subject), and places them within a Maildir structure based on that sorting. In addition, it does the same thing for deciding if it should delete a message - extremely basic spam filtering rules can be set up to check out certain headers for possible spam flags, etc.

The downloaded mail is then served via IMAP, using the Dovecot mail server. The great thing about that, is then every time I re-install any of the machines I use for mail access, or install a new one, I instantly have all my neatly sorted folders, all my mail from all my accounts, and only one IMAP account I need to set up.

Anyway, basically, the spam filtering of the above system was rather lame, so I went on the hunt for something a little more useful. Enter SpamBayes - a mail proxy application written in Python.

It’s already “in Debian”, so installing was as fun as always (aptitude install spambayes), after which I only needed to start the service, and then it’s off to a browser to configure it. Actually there wasone step before that - since I’m running this on my server, and SpamBayes is meant for use by a single user on their own PC, it doesn’t allow connections do it’s browser-based configuration from other hosts. Which is a bit of a problem when running a server which I have no interaction with beyond a command shell. Thanks to Lynx I was able to configure it to allow connections from my local network.

For starters, you need to tell it which POP3 servers you want to connect to, and assign local ports to each one, which will be stand-ins for port 110 when connecting to servers. The interface for this is a bit troublesome however, requiring you to enter each server into a single input field, separated by commas. The associated ports for each server are then entered into another input field in the same manner. It took me a while to get both the fields synced due tot he number of servers I intended using.

Next up, I fed it a few emails for training (saved emails out of Thunderbird as EML files, and these can be uploaded to the server for training via the browser interface) both ‘ham’ and spam.

Once it knew the basics, I simply updated the list of servers in my Python script to “localhost”, and whichever port each one was set to. Shortly thereafter, mail started passing through the system. Most of it was identified as “unsure”, as it hadn’t seen enough examples of ham or spam yet. Quite smartly, it keeps a record of each message that’s passed through, and you can easily train ham or spam from these.

Around 50 mails later, it was identifying almost every message perfectly. I’m going to leave it running for a day or two more, training everything that arrives, then I’ll just add a single filter to my mail fetching script, looking at the “X-Spambayes-Classification” header for “spam” (delete), or “ham” take no action.

I’m quite happy with this setup, looks like it’ll work quite well :D.

Yay, on Friday, I decided to take the plunge, and install Debian on my laptop. I’ve always wanted to try working in a Linux desktop environment, considering I do practically no Delphi development any more, everything’s either PHP or Python. Since Debian has plenty of support for both of these, it seemed quite ideal.

I’m not dumping the existing Windows install though, so I had to partition my drive. Now, partitioning drives is about the most nerve-racking thing I’ve ever done, no matter what software I’m using, no matter how little data I stand to lose, I’m always scared as hell something will go wrong. Doubly so on this laptop since I use it constantly for work, and it’s the default installation HP put on, which I’m not really eager to bugger up. Luckily, that all went smoothly though :D.

When it came to installing Debian though, I had a problem with the netinst installer I was using, in that a package or two would not install due to missing some authentication files or something. So, since that installer didn’t work, it was suggested that I try an older Sarge installer and upgrade to the Unstable branch from there.

As expected, that installation went without issue. Unfortunately I just had to try my best to keep the machine cool while waiting for the installation to complete and the ACPI applications to kick in so the fans would do their thing. Thanks Korpse for warning me about that in advance ;).

So anyway, after the base installation was done, it was trivial (as I have come to expect from Debian,) to get Gnome, X.org and GDM installed and running. Apache, PHP and the rest were even easier. I was very pleased to see that the Firebird SQL server (open source’d Interbase fork thingey) was available in Debian right away, so I plonked that in as well. All that was left was to install Subversion, and check out my work stuff. Once done, I was 100% ready for work :D.

There are a few things not working yet - I haven’t had much luck with the wireless LAN. Gnome’s wireless LAN configuration applet thing is missing WPA-PSK authentication options (which I’ve configured my wireless AP to use), it only supports WEP. I also can’t seem to get the WPA tools (wap_supplicant) to work correctly. Based on the output I’m getting though, it *looks* like it’s connecting to the access point - it manages to find the AP’s MAC address and everything just fine, and it reports the authentication was successful, but beyond that, I can’t actually ping anything, and all traffic still seems to be trying to go through eth0 (wired LAN) rather than eth1 (wireless). Guess I need to learn a bit more about Linux networking… heh.

Another thing that’s not quite working is Bluetooth. Again, it *seems* to be working, the hardware is detected, and is working fine. My phone can pair with the laptop fine, using the PIN I’ve defined, but I can’t seem to transfer files or anything in either direction. I’ll admit that I haven’t played with this much yet, but I’m not really sure where to go next. I haven’t even tried looking into infra-red yet :).

And yes, I do use all these things, that’s why I spent so much on this laptop :P.

I also haven’t installed the ATI drivers yet, it looks like it’s going to be a bit of a mission on it’s own though.

Along the way I’ve also discovered some interesting new applications I haven’t seen or heard of before. gDesklets seem like a nice way to waste some CPU time and memory if you like to keep your desktop busy. Beep Media Player seems like a very nice alternative to XMMS, seems a lot more stable, and the general feel integrates better with Gnome. RapidSVN looks like it’s trying to be a nice enough SVN front-end, however I find the good old command-line a lot more friendly and efficient (and it crashes less). Sylpheed is a rather nice little mail client, and works well as an alternative to Thunderbird - assuming you’re unhappy with Thunderbird though. ibWebAdmin is a neat web-based tool for managing your Interbase databases, not as feature-packed or good looking as phpMyAdmin, but it does what it needs to do pretty well.

All that’s left now is to give it a shot at work tomorrow, and see how it all goes :).

I must say, I’m rather disappointed with my “SmoothWall experience” so far. I’ve been tasked with setting up a SmoothWall firewall/proxy machine at work, and from what I’ve read, it’s like the best thing since sliced bread.

Unfortunately I cannot agree.

The installation tends to go fine, it partitions the hard disk by itself, installs fairly fast, then steps through a simple setup ‘wizard’. Here we are prompted if we want to enable or disable ADSL. Now, I want SmoothWall to connect via our ADSL line. BUT, it seems the developer’s idea of “ADSL” is in fact “USB ADSL Modem”.

Anyway, after figuring that one out, and after much shuffling of subnets and IPs between the router, SmoothWall, and my PC, I finally get it to use the router as a gateway. I try visiting some sites - DNS lookups fail. I take a look in all the log options on SmoothWall, and find the firewall is blocking DNS traffic, and is trying to route everything through the same (“Green”) NIC, rather than the second (“Red”) one.

Sooo, turns out I can fix this by running the “setup” tool again, and ‘pretending’ to change the IPs, so it resets everything (re-writes the firewall rules maybe?). Cool, everything’s working again. Not quite.

Seems after that, the proxy magically stops working altogether, so from the web interface, I just disable it, and re-enable it. Cool, everything’s working now. Riiiiight.

A few hours later, suddenly the internet is dead. Hmm, seems the firewall is blocking all traffic again and routing though the same NIC. Sooo, I repeat the whole IP change/reset, proxy reset, etc, and everything’s cool.

A few hours later I find myself repeating the whole procedure again.

This is seriously lame, having to practically reboot the entire machine every few hours. So I think maybe I’ll try to set up a PPPoE connection. So I go and configure the router correctly, test ‘dialing up’ with my machine in XP, all’s cool. Now to set up SmoothWall. Running the setup tool again lets me set the “Red” interface to “PPPoE”, and that seems done. Now where do I put my username and password to dial up?

Apparently the “ppp settings” page of the web GUI is where it’s done. Now excuse my ignorance, but this looks like a modem dial-up page, asking for phone numbers, which COM port my modem is on, etc, etc. A bit of searching around the rather un-helpful support forums, reveals that this is indeed where you need to configure PPPoE usernames and passwords. Just leave all settings alone except for login details.

I give it a shot, tell it to connect, nothing happens. Check the logs, and not surprisingly, it’s trying to connect via ttyS0 (COM1).

Now, apparently there’s supposed to be an option to select the correct interface in the drop-list where you select which port your modem is on, on the “PPP Settings” page, but for some magical reason this does not exist for me.

Unfortunately their forums are also not very helpful it seems, and even after composing a very descriptive help request, I get a rather sarcastic “RTFM” response for a subject not covered in the manual.

Basically the manuals are not up to scratch, the support forums are full of leetbois, the options in both the setup tool and web UI are obscure, and the whole thing is bloody useless, needing a darn reboot every few hours. WTF.

I’d love to send the whole thing to hell, but unfortunately I have to get it to work. *sigh*