Memoirs of a Roadie

Long Time Gone

Posted on 4th May 2010

It has been quite a few months since I last posted here. Quite a few events and projects have happened and held my attention since I last wrote in my blog. And I still have a backlog of photos and videos from last year to get through too!

I did wonder whether anyone might think that after talking about Why The Lucky Stiff in one of my last posts, that I had done the same. Well for those who follow my CPAN Testers work, will know that CPAN Testers 2.0 has been a rather major project that finally got properly underway in December 2009. It's nearing completion, and I'll cover some of the highlights in a future post. Although it's been my most consuming project over the last 6 months or so, it hasn't been my only one. As mentioned in another of my last posts, I'm writing a book about how to host a YAPC. Due to other projects taking a higher priority, this has taken somewhat of a backseat for the time being, but I do plan on getting a second draft together within the next few months. I have looked into self-publishing the book and I'm now planning to have it formerly submitted with an ISBN (the internation book numbers) and supplied via print-on-demand print runs.

Another project that has been ongoing alongside my CPAN Testers work, has been my website management system, Labyrinth. This has been the website application I have been developing since 2002, and although several other Perl web frameworks have now been developed since, to lesser and greater degrees, Labyrinth has had the disadvantage of only having 1 core developer for the past 8 years. It's not an application that will revolutionise web development and deployment, but it has very successfully worked for a number of websites I have developed over the years. After having been relatively stable for the past year or two, I'm now cleaning up the code so I can properly release it as open source. This is mostly so that anyone wishing to contribute to CPAN Testers, or the YAPC Surveys, will then have all the code available to them. If anyone wants to use it and help develop it further, that would be a welcome bonus, but realistically other web frameworks have gained so much mindshare that I'm not expecting Labyrinth to make much of a dent any more. Not that that is a problem, as Labyrinth has made deploying websites so much easier for me, that I'll just be glad to let people help on CPAN Testers and the YAPC Surveys.

Speaking of the YAPC Surveys, YAPC::NA 2010 and YAPC::Europe 2010 are fast approaching. These will be next projects to get up and running. Thankfully the code base just needs a few upgrades to the latest version of Labyrinth, and some work on skinning the CSS to match the respective YAPC sites. All being well this should only take a few days. Then I'll be looking to release this version of the code base for anyone wishing to run similar surveys for themselves. I've already had one interested party contact me regarding a conference in October, so hopefully the code will be suitable, and only the questions need adapting. We shall see.

My other major project this year, also began back in December 2009. As some readers are well aware, I am an ex-roadie. From 1989-1994 I was a drum tech, lighting engineer and driver for Ark, one of the best Black Country bands ever. Not that I'm biased or anything ;) Last year the band got together for some rehearsals and planned a few reunion gigs. With interest gaining, an album was also planned. So this year, the band began recording and booking gigs. As a consequence the Ark Appreciation Pages desperately needed a makeover. I'll write more about what happened next in another post. Ark are back, and Mikey and I are delighted to be able to be involved with the band once again.

That's just a few of the projects that have taken up my time over the last 6-8 months. There are several others that I hope to post about, with family, time and work permitting. Expect to hear a little more from me than you have so far this year.

April Skies

Posted on 1st May 2009

For those that might not be aware, I got made redundant on 31st March (the day after the QA Hackathon had finished). Thankfully, I start a new job next week, so I've managed to land on my feet. However, this has meant that I've ended up having the whole of April off to do stuff. My plan was to work on some of the Open Source projects that I'm involved with to move them further along to where I wanted them to be. As it turned out two specific projects got my attention over the last 4 weeks, and I thought it worth giving a summary of what has been going on.

YAPC Conference Surveys

Since 2006, I've been running the conference surveys for YAPC::Europe. The results have been quite interesting and hopefully have help organisers improve the conferences each year. For 2009 I had already planned to run the survey for YAPC::Europe in Lisbon, but this year will also see YAPC::NA in Pittsburgh having a survey of their own.

The survey site for Copenhagen in 2008 added the ability to give feedback to Master Classes and talks. The Master Classes feedback was a little more involved, as I was able to get the attendee list, but the talks feedback was quite brief. As such, I wanted to try and expand on this aspect and generally improve the process of running the surveys. Part of this involved contacting Eric and BooK to see if ACT had an API I could use to automate some of the information. I was delighted to get an email back from Eric, who very quickly incorporated an API that I could use, to retrieve the necessary data to keep the survey site for a particular conference up to date, even during the conference.

With the API and updates done, it was time to focus on expanding the surveys and skinning the websites to match that of the now live conference sites. The latter was relatively easy, and only required a few minor edits to the CSS to get them to work with the survey site. The survey site now has 3 types of survey available, though only 2 are visible to anyone not taking a Master Class. Those that have taken one of the YAPC::Europe surveys will be aware I don't use logins, but a key code to access the survey. This has been extended so that it can now be used to access your portion of the survey website. This can now be automatically emailed to attendees before the conference, and during if they pay on the door, and will allow everyone to feedback on talks during the conference. On the last day of the conference the main survey will be put live, so you can then answer questions relating to your conference experience.

I'm hoping the slight change won't be too confusing, and that we'll see some ever greater returns for the main survey. Once it does go live, I'd be delighted to receive feedback on the survey site, so I can improve it for the future.

CPAN Testers Reports

Since taking over the CPAN Testers Reports site in June 2008, I have spent a great deal of time improving it's usability for users. However, it's come at a price. By using more and more Javascript to dynamically change the contents of the core pages, it's meant that I have received a number of complaints that the site doesn't work for those with Javascript disabled or who use a browser that doesn't implement Javascript. For this reason I had decided that I should create a dynamic site and static site. The problem with this is that the current system to create all the files takes several hours for each set of updates (currently about 16 hours per day). I needed a way to drive the site without worrying about how long everything was taking, but also add some form of prioritisation so that the more frequently requested pages would get updated more quickly than those rarely seen.

During April, JJ and I went along to the Milton Keynes Perl Mongers technical meeting. One of the talks was about memcached and it got me thinking as to whether I could use it for the Reports site. Discussing this with JJ on the way home, we threw a few ideas around and settled on a queuing system to decide what needed updating, and to better managed the current databases to add indexes to speed up some of the complex lookups. I was still planning to use caching, but as it turned out memcached wasn't really the right way forward.

The problem with caching is that when there is too much stuff in the cache, the older stuff gets dumped. But what if the oldest item to get dumped is extremely costly on the database, and although it might not get hit very often, it's frequent enough to be worth keeping in the cache permanently. It's possible this could be engineered with memcached if this was for a handful of pages, but for the Reports site it's true for quite a few pages. So I hit on a slightly different concept of caching. As the backend builder process is creating all these static files, part of the process involves grabbing the necessary data to display the basic page, with the reports then being read in via the now static Javascript file for that page. Before dropping all the information and going on to the next in the list, the backend can simply write the data to the database. The dynamic site can then simply grab that data and display the page pretty quickly, saving ALOT of database lookups. Add to the fact that the database tables have been made more accessible to each other, the connection overhead has also been reduced considerably.

The queuing system I've implemented is extremely simple. On grabbing the data from the cache, the dynamic site checks quickly to see if there is a more recent report in existence. If there is, then a entry is added to the queue, with a high weighting to indicate that a website user is actually interested in that data. Behind the scenes the regular update system simply adds an entry in the queue to indicate that a new entry is available, but at a low weighting. The backend builder process then looks to build the entries with the most and highest weightings and builds all the static files, both for the dynamic site and the static site, including all the RSS, YAML and JSON files. It seems to work well on the test system, but the live site will be where it really gets put through its paces.

So you could be forgiven in thinking that's it, the new site is ready to go. Well not quite. Another part of the plan had always been to redesign the website. Leon had designed the site based on the YUI layouts, and while it works for the most part, there are some pages which don't fit well in that style. It also has been pretty much the same kind of style since it was first launched, and I had been feeling for a while that it needed a lick of paint. Following Adam's blog post recently about the state of Perl websites, I decided that following the functional changes, the site would get a redesign. It's not perhaps as revolutionary as some would want, judging from some of the ideas for skins I've seen, but then the site just needs to look professional, not state of the art. I think I've managed that.

The work to fit all the pieces together and ensure all the templates are correct is still ongoing, but I'm hopeful that at some point during May, I'll be able to launch the new look websites on the world.

So that's what I've been up to. I had hoped to work on Maisha, my other CPAN distributions, the YAPC Conference Survey data, the videos from the QA Hackathon among several other things, but alas I've not been able to stop time. These two projects perhaps have the highest importance to the Perl community, so I'm glad I've been able to get on with them and get done what I have. It's unlikely I'll have this kind of time again to concentrate solely on Open Source/Perl for several years, which in some respects is a shame, as it would be so nice to be paid to do this as a day job :) So for now, sit tight, it's coming soon...

Rockin' In The Free World

Posted on 26th January 2009

Earlier this month, a good friend of mine, Jono Bacon announced that we was starting to write a book about building communities. It's been a subject that has been discussed at length by many communities, many times over many years, and there is no one right answer to it. Some methods work in one context and don't in another. You see it all depends on the people, and specifically the personalities, who are part of the community and who you want to encourage (or discourage as the case may be) into joining, rather more than the project or common interest element itself.

Jono's book, titled Art Of Community, will be a look at how to build communities from different perspectives. He's getting several notable Open Source community members to help contribute their stories and it looks like it will be a really useful book for those starting a project, or user group to get some ideas of how to make it happen.

The hard part of starting any community, is promotion. Jono himself is taking note of this for the book's promotion too. You see the book itself has started a community of people who are early supporters of the book, and want to help make it a success. Part of making it a success is letting people know it exists. As Jono is already widely well know in technical communities (I've known him for about 8 years thanks to him starting WolvesLUG near me), he does have a head start. But it still needs people to talk about it, discuss it and eventually review it. I thought I'd write this blog post, partly to help promote the website that the book now has, but also make others aware that the book is being written.

I'm looking forward to reading the completed book, as apart from being a great read, I expect it to become a great source of reference for helping new communities promote themselves and florish.

Having started Birmingham Perl Mongers back in 2000, been a Perl community member, a member of the YEF Venue Committee and a major contributor to the CPAN Testers project, I've been very accutely aware how hard it can be to build a community. Though it should be noted that the building part isn't just about getting a project or user group off the ground, it's also about keeping it going, and encourage others to get involve and help the community thrive.

A good case in point is the CPAN Testers project. I first became a CPAN Tester back in 2004, and contributed several thousand reports for the Win32 platform. It was thanks to Leon presenting a BOF at 2003 YAPC::Europe in Paris, that I first became interested enough to join the volunteer effort. Shortly afterwards I started contributing to code for the smoke tools and the websites, creating the CPAN Testers Statistics website in the process. With the help of the Statistics site I was able to promote the project to other Perl programmers at YAPC events, by show how valuable the service the project provides is. Over the last few years the number of testers has grown, and the number of test reports submitted has gone from a about 100 per day to over 5,000s per day. In June 2008, Leon handed over the Reports website to me, as I was eager to improve the websites and make them more useful. Since then, I've had several developers help contribute patches and ideas to the project and it has been very encouraging to see the community driving the site forward. CPAN Testers now have their own server, a whole family of websites and a great tester community. In our case the community has built itself and mostly promoted itself from being a useful set of websites for developers. It'll be interesting to see if Jono pinpoints anything that we actually did do to build the project community and just never realised we were doing it.

I'm also interested in reading the book, as it is likely to have some useful references for a book project I'm currently working on. Although I don't plan on making it a hard copy book, it will be available online, and I hope to encourage contributions and improvements. My book doesn't have a working title as yet, but the subject matter is 'organising Open Source conferences', and will also have thoughts for workshops, hackathons and large technical meetings. The blue print for the project is based largely on my own experiences of organising The 2006 YAPC::Europe Perl Conference, but will hopefully include other thoughts and comments from conference organsiers for other Open Source events, such as the organsiers of LUGRadio Live, which Jono himself was significant instigator of. Like Art of Community, my project will also be available online under a Creative Commons license, and I'll be watching to see how the Art of Community community establishes itself and see whether there are any good ideas I could use too.

I look forward to finally reading the book, but in the meantime I'll just have to keep an eye on the Art of Community website updates.

Harder, Better, Faster, Stronger

Posted on 23rd July 2008

I should really have expected that interest to my site would hit overload last night, but I had thought it would cope. Unfortunately it meant I was sitting on the box, watching when the load average got too high and shutting down processes. As a result I did some quick profiling of the code using the lovely Devel::DProf, and spotted a few calls that were compelely unncessary, both as function calls and database calls. So I've quickly reworked some of the requests, and on my test machine the requests are now being processed in roughly 0.8 seconds rather than 1.6 seconds. Result!

It often takes something like being popular for you to actually take a second look at the performance of your site. Thankfully in my case, the changes are relatively minor, and have made significant improvements. I shall now be taking a better look at a few other sites I run soon, as I'm sure there are similar quick hit improvements I can make.

Killing The Bland

Posted on 3rd July 2008

For those that have suffered with accessing my sites over the past month, I have to apologise. In the first instance it appears that a TLS library wasn't playing well with XEN, the virtual server management system, which runs my server on a physical machine in Amsterdam. Having sorted that one out, with some library upgrades, I noticed that the perl applications running my sites were occasionally going into overload and failing to end, due to what appeared to be hanging I/O requests. I'm still not certain why, but have put some measures in place to reduce the I/O requests.

However, while investigating the server process and open files, etc, I noticed that there were rather a large number of cronjobs running, including several that had nothing to do with anything I'd set up manually. The most resource hogging ones appeared to be PHP related, both for php4 and php5. Seeing as I don't run any PHP on the server, I looked to getting rid of them. It took a while to trace, but it appears that the mod-php4 and mod-php5 had got installed with apache2, possibly following an upgrade I did recently, but the associated packages had all been installed too. I have now removed about 20 packages and several thousand files from the server, which has freed up a bit of disc space, but also stopped several cronjobs, thus reducing the load on the server. I'm hoping this solves the recent overload issues, but if it doesn't at least it might last a bit longer before keeling over!

I've been monitoring the box for the last couple of days and everything looks to be running fine again. If you suffer anything similar, I suggest looking in /etc/cron.d and related directories for things that you really don't expect to be running.