Memoirs of a Roadie

Turn The Page

Posted on 10th July 2009

I'm currently preparing myself for YAPC::Europe 2009. My talks are pretty much done, but I still need to fine tune some of the slides, particularly my big talk needs some additional work on its structure, so I don't labour the points. Also for the first time ever, I'm planning to submit a YAPC Lightning Talk. After talking with Mark Keating in Pittsburgh, he asked me to submit a lightning talk to the next NorthWestEngland Perl Mongers' technical meeting. As such I cut down my The Statistics of CPAN talk and gave it a go. The talk was supposed to be 5 minutes. However, 8 minutes later I finished! I guess I still need to cut a few more slides :) Thankfully Mark videoed the night, so I'm hoping I can critique my own performance, and fine tune what I say and the slides I use. Then I can really relax :)

Well not entirely. I've attended every YAPC::Europe conference and I'm now part of the YEF Venue Committee. While I have enjoyed attending YAPC::NA since 2005 in Toronto, I still feel an outsider. In some respects that's good, as it gives me a different perspective. One thing that has struck me over the years is that with every YAPC::NA organising team, there has been usually just one prominent member who is involved with the larger Perl community. In Europe there seems to be at least two (if not more) from each team who are known outside of their local user group. There are many other differences, but it's all helping me to add content for the book I'm writing.

After I helped to organise YAPC::Europe 2006 in Birmingham, I looked at rewriting the YAPC documentation that was available at the time. The previous version had been very US centric and many aspects were no longer applicable to any conference. More importantly, there were many more aspects to organising a conference that were missing. My initial rewrite is still online and hasn't been updated since, which is a shame, as I always hoped that the document would be a living document, with organisers from each year and around the world, helping to add their experiences to the document for future organisers.

As further updates haven't happened, I started to plan a rewrite the document again. Except it quickly became clear that this wasn't going to be a short update. Having witnessed the differences in more recent years between the NA and Europe YAPCs, there were many additions I wanted to make. From compiling and presenting bids, to the actual conference organising and communicating with sponsors, (potential) attendees and the wider IT community. As such, it quickly looked like I was better working on writing a book. I'm hoping that the result will be applicable to anyone organising an open source event, big or small, and will help to improve the conference experience for organisers and attendees alike.

But there was another aspect to writing it as a book that I wanted to help with, and that's promotion. Particularly with YAPCs, promotion to the community has usually been very good. There have been some hiccups along the way, but mostly it works out for the good. But the promotion to the general Open Source communities or the IT media is usually very lacking. Occasionally I come across people who have used Perl, but have no idea that YAPCs would be a good experience for them to learn more. By presenting a guide to organising a YAPC in book form, I'm hoping that it will help to promote how professional the YAPC conferences have become, and maybe inspire more sponsors to get involved too :)

This year, at YAPC::NA in Pittsburgh and I'll be doing likewise in Lisbon too, I've been taking notes. The differences between my casual involvement with YAPC::NA and more involved communications with YAPC::Europe are helping to shape some of my additions for the book. I'd like to get the book finished some time this year, so that I can get feedback from this year's organisers, and possibly other YAPC and workshop organisers from around the world. As yet I still don't have a title for the book, or cover or anything in the way of anything to promote it yet. I do plan to release it Open Source, probably under the Artistic License 2.0. I'm not planning on getting a publisher involved, but rather just release a PDF version. After all the source was originally available to all, and I want everyone organising a big technical event to have the opportunity to benefit from the contents.

If you have any thoughts about the book or what should go into it, feel free to collar me in Lisbon at the beginning of next month. I then plan to post more details about the book sometime in August.

April Skies

Posted on 1st May 2009

For those that might not be aware, I got made redundant on 31st March (the day after the QA Hackathon had finished). Thankfully, I start a new job next week, so I've managed to land on my feet. However, this has meant that I've ended up having the whole of April off to do stuff. My plan was to work on some of the Open Source projects that I'm involved with to move them further along to where I wanted them to be. As it turned out two specific projects got my attention over the last 4 weeks, and I thought it worth giving a summary of what has been going on.

YAPC Conference Surveys

Since 2006, I've been running the conference surveys for YAPC::Europe. The results have been quite interesting and hopefully have help organisers improve the conferences each year. For 2009 I had already planned to run the survey for YAPC::Europe in Lisbon, but this year will also see YAPC::NA in Pittsburgh having a survey of their own.

The survey site for Copenhagen in 2008 added the ability to give feedback to Master Classes and talks. The Master Classes feedback was a little more involved, as I was able to get the attendee list, but the talks feedback was quite brief. As such, I wanted to try and expand on this aspect and generally improve the process of running the surveys. Part of this involved contacting Eric and BooK to see if ACT had an API I could use to automate some of the information. I was delighted to get an email back from Eric, who very quickly incorporated an API that I could use, to retrieve the necessary data to keep the survey site for a particular conference up to date, even during the conference.

With the API and updates done, it was time to focus on expanding the surveys and skinning the websites to match that of the now live conference sites. The latter was relatively easy, and only required a few minor edits to the CSS to get them to work with the survey site. The survey site now has 3 types of survey available, though only 2 are visible to anyone not taking a Master Class. Those that have taken one of the YAPC::Europe surveys will be aware I don't use logins, but a key code to access the survey. This has been extended so that it can now be used to access your portion of the survey website. This can now be automatically emailed to attendees before the conference, and during if they pay on the door, and will allow everyone to feedback on talks during the conference. On the last day of the conference the main survey will be put live, so you can then answer questions relating to your conference experience.

I'm hoping the slight change won't be too confusing, and that we'll see some ever greater returns for the main survey. Once it does go live, I'd be delighted to receive feedback on the survey site, so I can improve it for the future.

CPAN Testers Reports

Since taking over the CPAN Testers Reports site in June 2008, I have spent a great deal of time improving it's usability for users. However, it's come at a price. By using more and more Javascript to dynamically change the contents of the core pages, it's meant that I have received a number of complaints that the site doesn't work for those with Javascript disabled or who use a browser that doesn't implement Javascript. For this reason I had decided that I should create a dynamic site and static site. The problem with this is that the current system to create all the files takes several hours for each set of updates (currently about 16 hours per day). I needed a way to drive the site without worrying about how long everything was taking, but also add some form of prioritisation so that the more frequently requested pages would get updated more quickly than those rarely seen.

During April, JJ and I went along to the Milton Keynes Perl Mongers technical meeting. One of the talks was about memcached and it got me thinking as to whether I could use it for the Reports site. Discussing this with JJ on the way home, we threw a few ideas around and settled on a queuing system to decide what needed updating, and to better managed the current databases to add indexes to speed up some of the complex lookups. I was still planning to use caching, but as it turned out memcached wasn't really the right way forward.

The problem with caching is that when there is too much stuff in the cache, the older stuff gets dumped. But what if the oldest item to get dumped is extremely costly on the database, and although it might not get hit very often, it's frequent enough to be worth keeping in the cache permanently. It's possible this could be engineered with memcached if this was for a handful of pages, but for the Reports site it's true for quite a few pages. So I hit on a slightly different concept of caching. As the backend builder process is creating all these static files, part of the process involves grabbing the necessary data to display the basic page, with the reports then being read in via the now static Javascript file for that page. Before dropping all the information and going on to the next in the list, the backend can simply write the data to the database. The dynamic site can then simply grab that data and display the page pretty quickly, saving ALOT of database lookups. Add to the fact that the database tables have been made more accessible to each other, the connection overhead has also been reduced considerably.

The queuing system I've implemented is extremely simple. On grabbing the data from the cache, the dynamic site checks quickly to see if there is a more recent report in existence. If there is, then a entry is added to the queue, with a high weighting to indicate that a website user is actually interested in that data. Behind the scenes the regular update system simply adds an entry in the queue to indicate that a new entry is available, but at a low weighting. The backend builder process then looks to build the entries with the most and highest weightings and builds all the static files, both for the dynamic site and the static site, including all the RSS, YAML and JSON files. It seems to work well on the test system, but the live site will be where it really gets put through its paces.

So you could be forgiven in thinking that's it, the new site is ready to go. Well not quite. Another part of the plan had always been to redesign the website. Leon had designed the site based on the YUI layouts, and while it works for the most part, there are some pages which don't fit well in that style. It also has been pretty much the same kind of style since it was first launched, and I had been feeling for a while that it needed a lick of paint. Following Adam's blog post recently about the state of Perl websites, I decided that following the functional changes, the site would get a redesign. It's not perhaps as revolutionary as some would want, judging from some of the ideas for skins I've seen, but then the site just needs to look professional, not state of the art. I think I've managed that.

The work to fit all the pieces together and ensure all the templates are correct is still ongoing, but I'm hopeful that at some point during May, I'll be able to launch the new look websites on the world.

So that's what I've been up to. I had hoped to work on Maisha, my other CPAN distributions, the YAPC Conference Survey data, the videos from the QA Hackathon among several other things, but alas I've not been able to stop time. These two projects perhaps have the highest importance to the Perl community, so I'm glad I've been able to get on with them and get done what I have. It's unlikely I'll have this kind of time again to concentrate solely on Open Source/Perl for several years, which in some respects is a shame, as it would be so nice to be paid to do this as a day job :) So for now, sit tight, it's coming soon...

Guiding Light

Posted on 6th February 2009

In 2006 I, along with 3 others from Birmingham Perl Mongers, organised the 2006 YAPC::Europe Perl Conference. It was thankfully a great success and invigorated several with ideas of things that they could do to join or create communities. Whether that was forming a local Perl Monger user group or starting a code project that would eventually be submitted to the CPAN. However, one person was inspired to go to another YAPC the following year and then submit a talk and speak at the 2008 YAPC::Europe Perl Conference. Had the 2006 conference not been in Birmingham, UK in 2006, Edmund would likely never have gone to a YAPC, and never realised how valuable they are. Not just in terms of the presentations and speakers, but of the communities and projects that are discussed, that he might not otherwise be aware of. And perhaps most importantly, realise just how easy it is to be included into the community and how easy it is for everyone to make a difference.

At the conference dinner in 2008, Edmund was struck by the lack of younger members of the communty in attendance, and started to think about why. For some time I have been trying to understand what we as a community can do to bring new people into the community, and although my perspective has focused on YAPCs, it equally applies to projects and local user groups. However, there is one aspect that I had neglected, that was obvious to Edmund. Funding. Most of those we are trying to encourage to come along to a YAPC are likely to be unwaged or on low wages, and cannot afford the costs of travel and accommodation for 4-6 days.

Last week Edmund launched the Send-A-Newbie website, with the support of the organisers for the 2009 YAPC::Europe Perl Conference to be held in Lisbon, Portugal, together with several members of the Perl community who have voiced approval. It is a great idea, and is a great way to enable students in particular a chance to attend the biggest Perl developer conferences in Europe.

The initaive aims to send at least 6 people, although even if only 1 person is selcted to attend this year, I would consider it a success. As it happens some grant applications have already been received, so it is likely that at least 1 person will attend thanks to the programme. Hopefully more will be approved for grants providing the funding can be obtained.

So how can you help? Well if you have the ability to do so, please consider donating. Mentioned the programme to anyone who you think might be a worthy recipient of a grant, and get them to apply. Mention it at your local user group, and see whether anyone can help with a donation. In order to keep YAPCs and the Perl community healthy we need to encourage potential future stars that attending the conference is a worthwhile oppotunity. If they could benefit from a grant to cover their travel and accommodation costs, then it really is in yours and their interest to do something about it. Applications will be accepted until 1 June 2009, so there is plenty of time yet to promote and apply for grants.

Washing of the Water

Posted on 27th August 2008

A couple of weeks ago I was in Copenhagen for YAPC::Europe, which was a blast. I did my Understanding Malware talk, which seems to have gone down well, and the posters even better!

Before leaving the UK I finally bought a new camera, a Canon EOS 40D. Unfortunately this was my first time using the camera and I was a little disappointed that I wasn't able to get the same quality of photos as my Fuji FinePix 5100. As such don't expect anything too much from these photos. Hopefully over the coming months I'll get used to the camera and improve the picture quality.

For some personal observations of the conference, see my use.perl post about it. I may do a more detailed write-up about the talks I saw and the discussions I had too at some point, but that's it for now. Anyway, enjoy the photos.

• Copenhagen - 11th August 2008
  
• Copenhagen - 12th August 2008
  
• YAPC::Europe 2008 - Copenhagen - 12th August 2008
  
• YAPC::Europe 2008 - Copenhagen - 13th August 2008
  
• YAPC::Europe 2008 - Copenhagen - 14th August 2008
  
• YAPC::Europe 2008 - Copenhagen - 15th August 2008

   

Living After Midnight

Posted on 22nd July 2008

So finally after several hours of trawlling through the 2,044 photos that I took over the weekend, I finally got down to the 744 that I'm going to publish here now. There are a few more from the video finale, that I'll post once the video is online and I can link to it. In the meantime, please enjoy.

• LUGRadio Live UK 2008 - 18th July 2008
• LUGRadio Live UK 2008 - 19th July 2008
• LUGRadio Live UK 2008 - 20th July 2008

Oh and there's a video in there too :)