Memoirs of a Roadie

Turn The Page

Posted on 10th July 2009

I'm currently preparing myself for YAPC::Europe 2009. My talks are pretty much done, but I still need to fine tune some of the slides, particularly my big talk needs some additional work on its structure, so I don't labour the points. Also for the first time ever, I'm planning to submit a YAPC Lightning Talk. After talking with Mark Keating in Pittsburgh, he asked me to submit a lightning talk to the next NorthWestEngland Perl Mongers' technical meeting. As such I cut down my The Statistics of CPAN talk and gave it a go. The talk was supposed to be 5 minutes. However, 8 minutes later I finished! I guess I still need to cut a few more slides :) Thankfully Mark videoed the night, so I'm hoping I can critique my own performance, and fine tune what I say and the slides I use. Then I can really relax :)

Well not entirely. I've attended every YAPC::Europe conference and I'm now part of the YEF Venue Committee. While I have enjoyed attending YAPC::NA since 2005 in Toronto, I still feel an outsider. In some respects that's good, as it gives me a different perspective. One thing that has struck me over the years is that with every YAPC::NA organising team, there has been usually just one prominent member who is involved with the larger Perl community. In Europe there seems to be at least two (if not more) from each team who are known outside of their local user group. There are many other differences, but it's all helping me to add content for the book I'm writing.

After I helped to organise YAPC::Europe 2006 in Birmingham, I looked at rewriting the YAPC documentation that was available at the time. The previous version had been very US centric and many aspects were no longer applicable to any conference. More importantly, there were many more aspects to organising a conference that were missing. My initial rewrite is still online and hasn't been updated since, which is a shame, as I always hoped that the document would be a living document, with organisers from each year and around the world, helping to add their experiences to the document for future organisers.

As further updates haven't happened, I started to plan a rewrite the document again. Except it quickly became clear that this wasn't going to be a short update. Having witnessed the differences in more recent years between the NA and Europe YAPCs, there were many additions I wanted to make. From compiling and presenting bids, to the actual conference organising and communicating with sponsors, (potential) attendees and the wider IT community. As such, it quickly looked like I was better working on writing a book. I'm hoping that the result will be applicable to anyone organising an open source event, big or small, and will help to improve the conference experience for organisers and attendees alike.

But there was another aspect to writing it as a book that I wanted to help with, and that's promotion. Particularly with YAPCs, promotion to the community has usually been very good. There have been some hiccups along the way, but mostly it works out for the good. But the promotion to the general Open Source communities or the IT media is usually very lacking. Occasionally I come across people who have used Perl, but have no idea that YAPCs would be a good experience for them to learn more. By presenting a guide to organising a YAPC in book form, I'm hoping that it will help to promote how professional the YAPC conferences have become, and maybe inspire more sponsors to get involved too :)

This year, at YAPC::NA in Pittsburgh and I'll be doing likewise in Lisbon too, I've been taking notes. The differences between my casual involvement with YAPC::NA and more involved communications with YAPC::Europe are helping to shape some of my additions for the book. I'd like to get the book finished some time this year, so that I can get feedback from this year's organisers, and possibly other YAPC and workshop organisers from around the world. As yet I still don't have a title for the book, or cover or anything in the way of anything to promote it yet. I do plan to release it Open Source, probably under the Artistic License 2.0. I'm not planning on getting a publisher involved, but rather just release a PDF version. After all the source was originally available to all, and I want everyone organising a big technical event to have the opportunity to benefit from the contents.

If you have any thoughts about the book or what should go into it, feel free to collar me in Lisbon at the beginning of next month. I then plan to post more details about the book sometime in August.

Senses Working Overtime

Posted on 9th July 2009

I haven't written here for a while. and I haven't written that much in my other blogs either. I should have, as I keep having ideas for posts that I figure I should get around to writing up, For various reasons my time has been taken up with other things. I've been very busy with some of my Perl projects, specifically CPAN testers and YAPC Conference Surveys, and relating to the latter, I've recently attended YAPC::NA 2009, have been a judge for the Send-A-Newbie programme and am currently judging the bids, along with the rest of the YEF Venue Committee, for YAPC::Europe 2010 from Keiv.pm and Pisa.pm. On top of that I've also been trying to spend more time with my family.

For the past year or so I've been working 15-20 a week on extra curricular Perl projects. Although I've enjoyed doing the work and I'm really proud of what I've managed to achieve, I do need to take time out for my family. I'm very grateful for the support Nicole has given me, and the fact that Dan and Ethne don't seem to mind me getting up before them at the weekend and waiting to have their breakfast, while I just do this next fix :) But they deserve some of my time now. That's not to say I'm disappearing or anything, just that the balance is slightly shifting in their favour.

Since leaving MessageLabs, I now have a job that is a bus ride away in the centre of Birmingham, which now gives me the freedom to work on the bus for at least an hour a day. It's actually quite amazing how much I can get done, when I have to focus on getting something done with only 30 minites between stops. I've missed travelling to work on the bus, as driving up and down the M5 every work day for the past 6 years has felt more and more of a drain. I used to read a lot, but that all stopped once I started driving to work. I'm now planning to start working through my unread bookshelf again :)

The new job is a bit different, as although they are not a web shop, my role is very much focused on web development. As a consequence I'm really enjoying it. They have a series of sites that have been developed over the years, without any real thought to standards and best practices. As a consequence, they now have me to help clean up their code and processes. I hasten to add that I'm not some kind of saviour, as the guys have been working to improve what they have for some time, but I think they do see me as a bit of welcome relief. It's also been nice to be able to bring a lot of the knowledge I've drawn on for the past 10 years or so, and be able to give their sites a fresh lease of life.

In future posts expect a quick run down of my experiences in Pittsburgh for YAPC|10, some thoughts about YAPC::Europe 2009 in Lisbon and several photos from the various gigs I've been to over the past year. Yes I still have lots of photos to sort though. Soon, I promise :)

The Green Manalishi (With the Two-Prong Crown)

Posted on 9th May 2009

For quite sometime now, I like others I've seen on various forums, have got really irritated with one aspect of the colour scheme used on Linux when doing a directory listing. If you have the colour scheme enabled and haven't changed any settings, then likely as not, if you have any directory that is world writeable, then you probably won't be able to read the text.

When designing websites, it is common practice to use high contrast colours, as you want the site to be readable, and not cause headaches among your readers as they try to understand what you've written. Recently I was investigating some high contrasting colours, and found some good websites that explain what I mean.

With this in mind, why on earth would anyone think that blue text on a dark green background would be easily readable. Unless you get close to the screen and squint!

On Linux systems there is a application called 'dircolors' and if you're lucky enough on RedHat based systems, you even have the '/etc/DIR_COLORS' configuration file, so it is possible to try and figure out what needs to be changed. However, until now I've never manaed to find the offending attribute to change. After a lot of searching this morning, I finally found a site that explains all the short form and long form keys, and additional colours that can be used in the colour scheme, beyond the 8 foreground and 8 background colours. Even better it even explains the effects available. In most places only bold is mentioned.

So in the event anyone else has had the same problem with their directories disappearing before their eyes, take a look at Configuring LS_COLORS by David Newcomb. It turns out the offending key is OTHER_WRITABLE (or ow in LS_COLORS format). I've now set mine to something much more sensible (bold white on a blue background), and it's much much easier on the eyes :)

April Skies

Posted on 1st May 2009

For those that might not be aware, I got made redundant on 31st March (the day after the QA Hackathon had finished). Thankfully, I start a new job next week, so I've managed to land on my feet. However, this has meant that I've ended up having the whole of April off to do stuff. My plan was to work on some of the Open Source projects that I'm involved with to move them further along to where I wanted them to be. As it turned out two specific projects got my attention over the last 4 weeks, and I thought it worth giving a summary of what has been going on.

YAPC Conference Surveys

Since 2006, I've been running the conference surveys for YAPC::Europe. The results have been quite interesting and hopefully have help organisers improve the conferences each year. For 2009 I had already planned to run the survey for YAPC::Europe in Lisbon, but this year will also see YAPC::NA in Pittsburgh having a survey of their own.

The survey site for Copenhagen in 2008 added the ability to give feedback to Master Classes and talks. The Master Classes feedback was a little more involved, as I was able to get the attendee list, but the talks feedback was quite brief. As such, I wanted to try and expand on this aspect and generally improve the process of running the surveys. Part of this involved contacting Eric and BooK to see if ACT had an API I could use to automate some of the information. I was delighted to get an email back from Eric, who very quickly incorporated an API that I could use, to retrieve the necessary data to keep the survey site for a particular conference up to date, even during the conference.

With the API and updates done, it was time to focus on expanding the surveys and skinning the websites to match that of the now live conference sites. The latter was relatively easy, and only required a few minor edits to the CSS to get them to work with the survey site. The survey site now has 3 types of survey available, though only 2 are visible to anyone not taking a Master Class. Those that have taken one of the YAPC::Europe surveys will be aware I don't use logins, but a key code to access the survey. This has been extended so that it can now be used to access your portion of the survey website. This can now be automatically emailed to attendees before the conference, and during if they pay on the door, and will allow everyone to feedback on talks during the conference. On the last day of the conference the main survey will be put live, so you can then answer questions relating to your conference experience.

I'm hoping the slight change won't be too confusing, and that we'll see some ever greater returns for the main survey. Once it does go live, I'd be delighted to receive feedback on the survey site, so I can improve it for the future.

CPAN Testers Reports

Since taking over the CPAN Testers Reports site in June 2008, I have spent a great deal of time improving it's usability for users. However, it's come at a price. By using more and more Javascript to dynamically change the contents of the core pages, it's meant that I have received a number of complaints that the site doesn't work for those with Javascript disabled or who use a browser that doesn't implement Javascript. For this reason I had decided that I should create a dynamic site and static site. The problem with this is that the current system to create all the files takes several hours for each set of updates (currently about 16 hours per day). I needed a way to drive the site without worrying about how long everything was taking, but also add some form of prioritisation so that the more frequently requested pages would get updated more quickly than those rarely seen.

During April, JJ and I went along to the Milton Keynes Perl Mongers technical meeting. One of the talks was about memcached and it got me thinking as to whether I could use it for the Reports site. Discussing this with JJ on the way home, we threw a few ideas around and settled on a queuing system to decide what needed updating, and to better managed the current databases to add indexes to speed up some of the complex lookups. I was still planning to use caching, but as it turned out memcached wasn't really the right way forward.

The problem with caching is that when there is too much stuff in the cache, the older stuff gets dumped. But what if the oldest item to get dumped is extremely costly on the database, and although it might not get hit very often, it's frequent enough to be worth keeping in the cache permanently. It's possible this could be engineered with memcached if this was for a handful of pages, but for the Reports site it's true for quite a few pages. So I hit on a slightly different concept of caching. As the backend builder process is creating all these static files, part of the process involves grabbing the necessary data to display the basic page, with the reports then being read in via the now static Javascript file for that page. Before dropping all the information and going on to the next in the list, the backend can simply write the data to the database. The dynamic site can then simply grab that data and display the page pretty quickly, saving ALOT of database lookups. Add to the fact that the database tables have been made more accessible to each other, the connection overhead has also been reduced considerably.

The queuing system I've implemented is extremely simple. On grabbing the data from the cache, the dynamic site checks quickly to see if there is a more recent report in existence. If there is, then a entry is added to the queue, with a high weighting to indicate that a website user is actually interested in that data. Behind the scenes the regular update system simply adds an entry in the queue to indicate that a new entry is available, but at a low weighting. The backend builder process then looks to build the entries with the most and highest weightings and builds all the static files, both for the dynamic site and the static site, including all the RSS, YAML and JSON files. It seems to work well on the test system, but the live site will be where it really gets put through its paces.

So you could be forgiven in thinking that's it, the new site is ready to go. Well not quite. Another part of the plan had always been to redesign the website. Leon had designed the site based on the YUI layouts, and while it works for the most part, there are some pages which don't fit well in that style. It also has been pretty much the same kind of style since it was first launched, and I had been feeling for a while that it needed a lick of paint. Following Adam's blog post recently about the state of Perl websites, I decided that following the functional changes, the site would get a redesign. It's not perhaps as revolutionary as some would want, judging from some of the ideas for skins I've seen, but then the site just needs to look professional, not state of the art. I think I've managed that.

The work to fit all the pieces together and ensure all the templates are correct is still ongoing, but I'm hopeful that at some point during May, I'll be able to launch the new look websites on the world.

So that's what I've been up to. I had hoped to work on Maisha, my other CPAN distributions, the YAPC Conference Survey data, the videos from the QA Hackathon among several other things, but alas I've not been able to stop time. These two projects perhaps have the highest importance to the Perl community, so I'm glad I've been able to get on with them and get done what I have. It's unlikely I'll have this kind of time again to concentrate solely on Open Source/Perl for several years, which in some respects is a shame, as it would be so nice to be paid to do this as a day job :) So for now, sit tight, it's coming soon...

The Battle of Evermore

Posted on 30th April 2009

Now that The Lord Of The Rings trilogy has finally made it to the big screen in Hollywood style, production is in progress to bring The Hobbit to the big screen for 2011. But that's a long way away for many fans.

It was far too long to wait for one fan, who decided it would be a great idea to do a fan-film, based on the Tolkien's Middle Earth. The writer/director Chris Bouchard enlisted Tolkien fans on the internet, to help with the casting and the pre- and post-production. The film has cost around £3,000 to make, and is a testiment to what can be achieved on a low budget. The film is only 40mins long, but looks to be every bit as enjoyable as the Hollywood counterparts. The film, The Hunt For Gollum, was inspired from Tolkien's writings, adapting the appendices from The Lord Of The Rings books, and fits inbetween The Hobbit and The Lord Of The Rings stories.

The film is to have its first official screening at the Sci-Fi-London film festival on Sunday 3rd May 2009, but thankfully everyone will be able to see it. As The Hunt For Gollum was made as a non-profit venture, part of the goals of the project was to release the film for free. After the official screening on Sunday, the film will be available online for download from the official website, for everyone to watch at their leisure.

In some ways it's a pity that they aren't selling this as a DVD/Blu-Ray release to help fund possible future fan-films, but I suspect that this may cause problems with licensing from the Tolkien Estate, etc. As the sites disclaimer states "[The Hunt For Gollum] is in no way affiliated with, or sponsored or approved, by Tolkien Enterprises, the heirs or estate of J.R.R. Tolkien, Peter Jackson, New Line Cinema, HarperCollins Publishers Ltd. or any of their repective affiliates or licensees." As such getting appropriate approval for a DVD/Blu-Ray release would likely be traumatic to say the least.

Having watched the trailers I'm looking forward to seeing the final film, as it looks to be every bit as magnificent as the big budget films. I also hope that Chris Bouchard gets the chance to work of a big budget feature telling another tale from Tolkien's Middle Earth, as there are plenty more stories that I'm sure other fans would love to see come to life :)

From the Time You Say Goodbye

Posted on 31st March 2009

Today was my last day at MessageLabs/Symantec. After 6 years it was sad to say goodbye to a lot of people, as it has been an absolute pleasure working with such a great group of people. There are some people I'm going to really miss, but thankfully several of them I will be able to catch up with at various Perl events throughout the year.

I've worked on some intresting projects and learnt a lot about SMTP, email and networking during my time with MessageLabs. In fact it's been so interesting it has been worth the morning and evening drive along the M5 for every working day in the last 6 years! But now it's time to move on.

My future challenges currently involve working on all the Open Source and Perl (obviously) projects that I haven't had time for over the past year, taking some time out, relishing NOT having to travel along the M5 every morning and evening, and planning my international speaking tour (okay there are currently only 4 dates throughout the year, but it sounds good ;)) 

I'm not planning to rush into anything, and instead want to take the time to find the right job for me, and to find the right company that can make good use of me. There are a few possibilities that have already been mentioned to me, so I'm hopeful that I won't be looking for too long.

In the meantime, expect several updates to CPAN, CPAN Testers, Maisha, among many other projects I haven't time to work on properly for the past year or so. I shall not be idle :)

New Sensation

Posted on 5th March 2009

In the UK, Walkers Crips are perhaps the strongest brand of crisps on the market. They're also the most inivitive too. Every so often they try a different flavour out, and while most rarely see the light of day again, some do stick around. A more recent one is the Marmite crisps, which having proved so successful are now made by Marmite themselves. Last year Walkers Crisps ran a competition to find some new flavours. Out of the hundreds of entries they picked 6 and have now mass produced them to let the public vote on their favourites.

Walkers have been quite clever about this as they packaged them into two 6-bag bundles, with three flavours in each. It means rather than trying to find the flavours individually in your local corner shop, you can grab a couple of extra 6-bag bundles and try them all out twice.

The six flavours are:

• Fish & Chips
• Crispy Duck & Hoisin
• Cajun Squirrel
• Builder's Breakfast
• Onion Bhaji
• Chilli & Chocolate

The mix could be seen as 2 traditional British dishes, 2 international dishes and 2 just plain weird. Now of the latter, Chilli & Chocolate isn't that unusual, as I've seen and eaten Chilli flavoured chocolate bars before, but Cajun Squirrel! Never having tried squirrel I can't comment on it's accuracy, but I can't really see it being a lasting flavour beyond it's novelty value.

I've now tried them all, and after some consideration I would have to rate them as:

1. Onion Bhaji
2. Crispy Duck & Hoisin
3. Builder's Breakfast

Of the rest I wouldn't be particularly interested in trying again. Cajun Squirrel tastes like Roast Chicken, Fish & Chips tastes like Salt & Vinegar and while Chilli & Chocolate does taste exactly as it says, having first thought the chocolate bars were an interesting idea, I will most definitely not be voting for them.

It'll be interesting to see which flavour wins, and perhaps more importantly how long it lasts as a popular flavour. I think the 2 international dishes are likely to hold out longer than the others, as they both have a distinct flavour of their own, plus we Brits have a strong liking for Indian and Chinese cuisine. Still, whichever wins, I'm sure it'll be overshadowed by the next big Walkers Crisp marketing campaign :)

Open Your Eyes

Posted on 3rd March 2009

Thanks to a twitter post by Simon Phipps this morning, I read with interest a blog post by Patrick Finch, entitled Mozilla and Cybermentors. Mozilla are getting actively involved with the UK charity BeatBullying and their CyberMentors programme.

It's unlikely that anyone growing up hasn't suffered some form of bullying. It comes in all shapes and forms, and while for some of it may be minor or only last a short time, for some it can have devasating effects. A colleague recently took his own life, because the traumatic and abuse he suffered through primary and secondary school, even over ten years later, was still something that affected his life and personality, and was something he felt he could no longer cope with. I personally was first bullied by my 3rd year junior teacher. Yes you read that right a teacher, and I was just 9. It isn't just children that can be cruel and spiteful, adults can too. While I would dearly love to name the teacher in question, I have no proof beyond my word, and even now feel powerless to do anything about it, much as I did back then. Many children who suffer from bullying feel exactly the same way. Even if they told, who would listen and who would even believe them?

As Patrick points out in his post, many young people are growing up never knowing how we used to keep in contact with our friends, without using the internet or mobile phones. We play out so much of our lives online, that it shouldn't be a surprise that a recent Harvard University task force concluded that one of the biggest risks to children on the internet, isn't from sexual predators, but from bullies. The difference between the school yard bulling and cyber-bullying, is that the former is pretty much contain within a small sphere and often there are adults and peers who can deal with it and stop it. On the internet anyone can hide behind their relative anonymity and victimise just about anyone they choose. As it isn't within school grounds, teachers are often unable or ill-equiped to deal with it.

As such, the Cybermentors aims to be a way for youngsters experiencing bullying online to tell someone about it. Mozilla are offering to support 10 members of the Mozilla community to be trained as Cybermentors, who can then spend at least 2 hours a week for 4 months, helping children to cope and deal with any bullying issues. All credit to Mozilla for supporting this, and hopefully other companies will also be willing to help fund training for individuals to act as mentors.

A few years ago the GetSafeOnline campaign was initiated to help make parents and youngsters aware of the potential dangers on the internet. Identifying ways to protect themselves from viruses, phishing scams and spam, as well as unwanted websites, chatrooms and the like. While this programme is different in that it's targeting a very specific danger, it is still all about keeping the internet safe for everyone. I personally value efforts such as this, rather than the sometimes heavy-handed and misguided attempts by governments and self-appointed puritans of the internet to protect children from percieved threats.

I really hope BeatBullying and the Cybermentors programme gets a lot of internet and media exposure, as the more children are aware of it, the more chance they have of coping with it and not suffering mental anguish for the whole of their (possibly short) adult life. If you're a member of the Mozilla community, and think you can spare the time, please read Patrick's blog post and get in touch with him.

Back On Line

Posted on 16th February 2009

After the last few weeks of trying to access Twitter from the command line, I set about writing something that I could expand to micro-blog to any social networking site that supports many of the Twitter API type commands. At the moment it only works with Twitter and Identi.ca, but my plan is to look at creating plugins, or more likely to allow others to create plugins, that can enable the tool to interact with other micro-blogging sites.

After trying to think of a decent name, I finally settled on Maisha. It's a Swahili word meaning "life". You can grab the code from CPAN as App-Maisha.

Currently you'll need to use the standard Perl install toolset to install the application, but ultimately I'd like to have something that you can install just about anywhere without having to go through all the headache of installing dependencies. I'll have a go at doing an .rpm and a .deb package release, and will also try using PAR. It would be nice to have this as a standalone application that just about anyone can use, but for now CPAN will have to do.

My next immediate step is to look at writing something that interfaces to Facebook without requiring a developer key or any such nonsense. It will probably have to involve a bit of screen scraping, unless there is some more official API, but as yet I haven't found it. Everything regards Facebook applications seems to centre around the developer application that can do all sorts of dubious things, but mine is purely for the user to control from their desktop, not a 3rd party website/server. Thus giving them a developer API key assigned to me is wholly inappropriate. It would be nice if they had a restricted User API, which allows you to update your status and look at your friends' statuses, but I think I'll be in the minority wanting it.

You Know My Name

Posted on 9th February 2009

Having mentioned twitter in a recent post, I thought I would mention a project I decided to look at recently. The original project, twittershell, is not my own, but it appeared to be the closest to what I was after, a command line interface to Twitter, with the bonus of it being written in Perl. As a consequence of the latter, I was able to hack on the code and submit a patch to do a lot of what I wanted. I currently run the patched version, and it runs rather nicely for me. However, there is something missing.

Twitter is no longer the only micro-logging service and as such I've also signed up to identi.ca. With the APIs being pretty much the same, it should theorectically be simple to plugin an identi.ca interface to twittershell. Except it isn't. Unfortunately twittershell is written with only the Twitter API in mind. To intergrate identi.ca and other micro-blogging services, it requires a rewrite. So that's where I'm currently at. The original twittershell project hasn't been touched in over a year, so I'm hoping the orignial author won't be offended by me forking the code to a new project.

However, what do I call the new project? I would rather it not be something that identifies itself with any specific blogging service, as I would like it to have a broader appeal, that encourages others to add plugins should a new service come along. I realise this project will likely have limited appeal, as iPhone and GUI apps seem the in thing, but I want something that I can run via ssh/screen on my home box and not have to worry about watching some app running on the desktop.

One idea I had was to call it 'Mazungumzo' (Swahili for talk or conversation ... an idea stolen from Joomla!), then I thought of 'Maisha' (Swahili for life). I did look up some Welsh words, but doubt anyone would be able to pronounce them ;) I also thought of 'Rambler', but that might have too many connections to someone who goes walking across hill and dale of a weekend. So any good ideas for projects names?