Geoff Garside

Closedown

Please see http://geoffgarside.co.uk/ for future content

Harsh highlighting

Just read about the Harsh ERb/HAML highlighter, it looks quite cool. Also because it uses UltraViolet it can be powered by my version of the Oniguruma gem.

Initialising Cocoa NSValueTransformers

Just a quicky, I created my first custom Value Transformer this evening. Normally to register your transformer you have to do something like the following

+ (void)initialize
{
    [super initialize];
    [self initialiseValueTransformers];
}
+ (void)initialiseValueTransformers
{
    LoginStatusTransformer *loginStatusTransformer =
        [[[LoginStatusTransformer alloc] init] autorelease];
    [LoginStatusTransformer setValueTransformer:loginStatusTransformer
        forName:@"LoginStatusTransformer"];
}

which hurt that part of my brain which likes clean code because of all the repetition.

So I cooked up a nice little pre-processor macro to tidy it up

#define init_transformer(klass) \
    klass *vt##klass = [[[klass alloc] init] autorelease]; \
    [klass setValueTransformer:vt##klass forName:@""#klass]

so now I have

+ (void)initialiseValueTransformers
{
    init_transformer(LoginStatusTransformer);
}

which will be expanded out to

+ (void)initialiseValueTransformers
{
    LoginStatusTransformer *vtLoginStatusTransformer =
        [[[LoginStatusTransformer alloc] init] autorelease];
    [LoginStatusTransformer setValueTransformer:vtLoginStatusTransformer 
        forName:@"LoginStatusTransformer"];
}

it seems to be working for me at the moment, any bugs in the macro please let me know.

Tweaking RDiscount

As I mentioned in my previous post I’d had some issues with Pygments and RDiscount. I have been using Pygments to highlight my code blocks in my blog. Unfortunately right after using a code block with linenos none of the markdown was being parsed anymore.

The Discount library upon which RDiscount is based has the ability to output a debug tree which shows the different blocks within the document. Things like paragraphs, quotes, code blocks and so on. Using this I was able to determine that the HTML block which contained the highlighted code was the last block being detected.

The Pygments highlighted source with line numbers is actually rendered in a table. This table closes with

</td></tr></table>

and herein lay the problem. The Discount library detects the closing HTML tag with the following function

static Line *
htmlblock(Paragraph *p, char *tag)
{
    Line *t = p->text, *ret;
    int closesize;
    char close[MAXTAG+4];

    if ( selfclose(t, tag) || (strlen(tag) >= MAXTAG) ) {
        ret = t->next;
        t->next = 0;
        return ret;
    }

    closesize = sprintf(close, "</%s>", tag);

    for ( ; t ; t = t->next) {
        if ( strncasecmp(T(t->text), close, closesize) == 0 ) {
            ret = t->next;
            t->next = 0;
            return ret;
        }
    }
    return 0;
}

which as you might be able to tell, on line 17, checks the line for the presence of the HTML closing element. In the case of the Pygments closing element, this was not on its own line. While looking for </table> it would only read up to </td></t from the line before giving up.

Testing with a pre-pygmentised file and pushing the </table> onto its own new line confirmed that this was the problem.

But how do we fix it?

Well I started out looking for C string searching functions, of course strstr was the first candidate. Replacing line 17 of the aforementioned htmlblock() function with

if ( strstr(T(t->text), close) != NULL ) {

seemed like a good idea, but running the RDiscount test suite revealed my naivety of the solution. With my change text such as </table> was matched breaking HTML code examples, this wasn’t good enough.

I spent much longer than I really should have trying other solutions before having the critical Eureka moment. I should add that prior to this moment I’d learnt of the more suitable strcasestr() function to better match the strncasecmp() function originally used.

At any rate my Eureka was realising that I was only ever going to need to search a string of closing tags, and that these closing tags would never be preceded by white space. Initially was thinking of using the CTYPE isspace macro, but a far simpler solution also struck me.

A closing tag, or string of closing tags will always start with a <, so provided the line started with < and then contained the respective closing tag I could reasonably safely assume the HTML block was being closed.

So I next transformed line 17 of htmlblock() to

if ( T(t->text)[0] == '<' && strcasestr(T(t->text), close) != NULL ) {

and re-ran the RDiscount test suite to see if the tests passed, they did. Next I added a snippet of the Pygments table to one of the test files with some extra markdown after it, updated the expected output file as well and re-ran the test suite. Again it all passed perfectly.

Once this was all done, I committed my changes to a branch of the fork I’d made of RDiscount and pushed them up to my fork on GitHub.

As I final test I updated the Discount source as well and re-ran it over the file to output the debug tree again. I was pleased to see all the right blocks of the file being shown, lovely.

Update

My fork has been merged with the master and been refined as well.

Changing the blogging gear

I’ve recently been working on merging my two blogs together into one. One of them I host myself, the other is with Tumblr. One has stagnated quite severely with the last update April 1st, 2008, the other only seems to get periodic and occasionally small updates.

Twitter has played a large part in the lack of updates to both of these sites. Through Twitter I can post short thoughts, URLs to items of potential interest and frequently have a more live dialogue with a group of people. I also don’t plug either of my blogs to any degree really, I provide links to them, but I don’t go telling people to read them.

So what, you may ask, is the point in merging them at all? Well I still want to blog, but the point of blogging has changed. I now see blogging as a platform from which I can propose my thoughts or detail my current exploits when they merit it. My recent head injury was one such item mainly because I was asked for further details on it and it wouldn’t fit the Twitter 140 character limit. The post on MySQL Replication was asked for by a friend of mine, while it was simplistic it helped me iron the process down in my head as well.

So as the game is changing a bit I’ve been through all of my old blog posts, correcting spelling mistakes as well as changing some grammar here and there. Its a bit vain I know, but some of it read really poorly, I also went and wrapped nearly all of the code blogs so they’d be parsed by Pygments. I had some fun with that, especially with RDiscount which I am using to parse the Markdown post files. I managed to sort out those problems though and now I’m just down to sorting out some niggly little bits before I close down the other two and redirect them to one location.

Setting up MySQL Replication

Today I was asked to set up a MySQL Replication server to allow for failover on one of the services we operate.

A friend of mine also asked me to document the procedure as its been a while since he’d done it and couldn’t remember. So here goes….

The Setup

I’ll be doing all of this on FreeBSD 7.1 servers and I’ll be installing things from ports and then setting up the replication part.

Installing MySQL is pretty much a breeze, I do tend to do this first though

# cat <<EOF > /etc/make.conf
.if ${.CURDIR:M*/databases/mysql5*}
BUILD_OPTIMIZED=YES
.endif
EOF

to ensure that MySQL is built to run as quickly as possible. Then you have the simple commands to install the server, assuming you use ports-mgmt/portupgrade

# portinstall databases/mysql50-server

once that has been done on each server then we can begin setting up the replication service.

Instruct the Master

First we need to tell the master that it is a master, we also need to make sure that it’ll listen for slaves. So we need to add some things to the configuration file. You can use /etc/my.cnf which is pretty global but I’ve opted for /var/db/mysql/my.cnf

[mysqld]
bind-address      = 192.168.0.10
server-id         = 1

now, it will listen and it knows its the master.

Next we need to instruct it where to keep the binary log file for replaying to the slaves so in the same file

log-bin           = mysql-bin
expire-logs-days  = 20
max_binlog_size   = 104857600

so the binary log file will be written to /var/db/mysql/mysql-bin, entries older than 20 days should be dropped and it shouldn’t grow bigger than 100 megabytes.

Finally we tell it exactly which databases we want to replicate, can you ignore this if you don’t want to be specific. Again in the same file

binlog-do-db      = testdb
binlog-ignore-db  = mysql,test

so only our testdb table will be binary logged, additionally mysql and test databases will be ignored.

Now you can restart your MySQL master server, to check that things are working you can connect to the server and run

mysql> SHOW MASTER STATUS;
+------------------+----------+---------------+-----------------------+
| File             | Position | Binlog_Do_DB  | Binlog_Ignore_DB      |
+------------------+----------+---------------+-----------------------+
| mysql-bin.000001 |       98 | testdb,testdb | mysql,test,mysql,test | 
+------------------+----------+---------------+-----------------------+
1 row in set (0.01 sec)

and you should get something like above. You can also check to make sure the server is listening for connections with

# sockstat | grep mysql | grep tcp4
mysql    mysqld     90119 13 tcp4   192.168.0.10:3306   *:*

Creating a replication user

So we need to setup an account on the master server for the slave to use to authenticate itself. To do this we create a REPLICATION SLAVE using the following MySQL statement

GRANT REPLICATION SLAVE ON *.* TO 'slave'@'slavehost' IDENTIFIED BY 'slavepass';

we’ll set these values on the slave as well so that it can authenticate to the master.

Instructing the Slave

Again I’ll be keeping my configuration in /var/db/mysql/my.cnf, so for the slave we need to tell it that it isn’t top of the food chain and also who is. To do this we add

[mysqld]
bind-address          = 192.168.0.11
server-id             = 2

to tell the Slave to listen for connections, this is so that external services which perhaps only need read access to the database can use the slave and save the master for writes.

Next we must inform the Slave of its masters address and its authentication credentials for use when addressing the Master

master-host           = 192.168.0.10
master-user           = slave
master-password       = slavepass
master-connect-retry  = 60

Finally as we only want one database to be replicated, in my case at least, add the following line to the configuration file

replicate-do-db       = testdb

so that only the testdb database will be replicated.

Copying the database to the Slave

Firstly we need to know where in the binlog we are at the moment, so on the master server run the following command

mysql> SHOW MASTER STATUS;
+------------------+----------+---------------+-----------------------+
| File             | Position | Binlog_Do_DB  | Binlog_Ignore_DB      |
+------------------+----------+---------------+-----------------------+
| mysql-bin.000001 |   327128 | testdb,testdb | mysql,test,mysql,test | 
+------------------+----------+---------------+-----------------------+

remember the File and Position values as we’ll need these to resynchronised the databases once the slave has a copy of the data.

Using mysqldump create a copy of the master database to put onto the slave

# mysqldump --opt --databases testdb > /root/master-databases.sql

Now copy the /root/master-databases.sql file to the slave and then import it, assuming you copied it to /root/master-databases.sql on the slave as well

# mysql < /root/master-databases.sql

Now we need to synchronise the two copies. On the slave load up the mysql client

mysql> STOP SLAVE;
mysql> CHANGE MASTER TO MASTER_HOST = '192.168.0.10',
    -> MASTER_USER = 'slave_user', MASTER_PASSWORD = 'slave_pass',
    -> MASTER_LOG_FILE = 'mysql-bin.000001', MASTER_LOG_POS = 327128;
mysql> START SLAVE;

and the changes will be synchronised between the two servers.

A quick test

Now its a good idea to make some test insertions into the master database and then check to see if they exist on the slave as well. I’ll leave this part up to you…

So what happened to me on Friday the 6th of March?

As so many of these tales unravel it all started with alcohol. I’d been invited out by Claire for her brother Neils fiancé Alices birthday drinks do. This is only the second time I’ve met Alice, the first time at their engagement party. I’ve known Neil for a while as he used to work at the same company I’m at now.

At any rate over the course of the night quite a few drinks where consumed.

Towards the end of the evening, one or two people had started to go home, I felt a bit dizzy quite rapidly. Normally I get a slowly increasing buzz which I know well and use as a good indicator of when to stop drinking. I made my way to the toilets to get some water on my face, unfortunately there was a queue of people coming out of the toilets, so I decided to go back up the spiral staircase to get some air instead. The next couple of events are bit more patchy. I recall standing under the staircase, possibly letting some people past, something also makes me think I might have sat down under the stairs, maybe I was waiting to get into the toilets to splash my face. Things just sort of get a bit black and hazy from there.

The next thing I remember is seeing Claire and the woman from the coat desk standing over me. Claire was pressing some blue paper towels to my forehead. I don’t specifically recall the first couple of questions but they were along the lines of “are you alright” and “how many fingers am I holding up”. Then we came to the slightly more tricky questions of did I remember what happened. As you can probably guess at this point, I wasn’t completely sure. I’d apparently been unconscious for a minute or so as well.

After a few minutes the ambulance “drunk bus” arrived, and the paramedics had a look over me, took my blood pressure, checked my blood sugar and took my details. They said the wound seemed superficial but that someone should keep an eye on me and asked if I’d come with anyone else, I said no and that I’d met up with every body at the bar. Once they’d finished collecting my details the paramedic and I wandered back over to my friends and told them that I should be alright but that someone should keep an eye on me over night as it could be worse than it looked. If I started having blurred vision or started vomiting I should be taken to hospital to be checked out fully.

Everyone came back out of the bar a few minutes later to decide what to do next, I think the party was winding down so most opted to return home. So we said farewell to each other amidst various Harry Potter comments. Claire offered to take me back to her house and keep an eye on me so we headed off in the direction of the tube. On the way back, before we caught the tube, she checked with one of her house mates that was away for the weekend to see if I could sleep in their bed for the night. Fortunately the tube is quite direct though the whole time I was wishing I had a hat to cover up my stupid forehead. When we got off the tube we had a rather pleasant walk to her house, it was good to chat without the noise of the bar.

When we got to her house Claire stuck some bandages to my forehead and chin so that I wouldn’t bleed all over her house mates pillow. We each then went to bed, as I couldn’t quite sleep yet I posted a picture on twitter and then answered a few questions about it before finally putting head down to sleep.

Around eight in the morning I woke up and checked my bandages, it was good having them there as my chin had bled a bit during the night. I left them in place and then lay back down in the bed for a bit to rest and run through the evening. After a little while I checked the internet to find some tips on speeding up the healing process. Vitamin C, Zinc and protein all seemed like best things to get, so I decided that I’d have a fry up with Orange Juice when I got home. The best hangover cure if ever there was one. A tasty multivitamin can follow that up for the Zinc. I also did some “Brain Training” as I figured it might help.

Some time later I got up again and carefully remove the bandages from my face, I managed to loosen the scab on my chin from the bandage with water so I didn’t tear it all off. At the same time I noticed some abrasions on the side of my face, quite shallow though and they’d scabbed up pretty quickly by the looks of things, a bit swollen though.

Around 9.30 Claire came down to check on me, by this time I was feeling reasonably well, as she needed to pack up to go to a ball in the evening I decided I would make my way back home. She gave me some directions back to the tube station and I remembered most of it from the walk the opposite direction the night before.

I got a couple of strange looks from people on the tube but mainly I kept my head down to avoid everyone. When I got back to Mile End I went into the new Tesco got my fry up supplies and some plasters. Then I had a nice big fry up when I got back to my flat.

Later that day I discovered a couple other party injuries, a decent sized but shallow cut on my shoulder and a skinned and bruised knee. None of it, bar the knee, is giving me any real pain and I’ve not had any dizziness or vomiting since the event.

So what actually happened?

Well the best I can piece together is that my head collided with the spiral staircase by the toilets, though its difficult to work out exactly how I got the gash I did from that. The cut on my chin did swell up a little but, but as I’ve no pain in my jaw I don’t think I hit it very hard at all. I’ve also no neck pains so I don’t think I fell down the stairs at all. I suppose its possible that I just stood up into the edge of a stair, that would probably explain the vertical nature of the gash though its not a straight wound.

But I wanna see a picture!!!

Ok fine, Jason took a pretty good one of it the other night. Fortunately its healing up reasonably well, so with any luck it should be faded in a week or two.