Memoirs of a Roadie

Young Parisians

Posted on 10th April 2012

Did I mention I went to Paris to take part in the 2012 QA Hackathon? Did I remember to mention all the cool stuff I got done? Well if you've been hiding for the past few weeks, have a look at the last couple of posts :)

As per usual, while there I took my camera along. However, unlike many previous visits to Paris, I didn't do any sight-seeing. And that includes failing to wander around the venue we were in and discovering the real submarine among other things, that others found while taking a breath of fresh air.

Instead I spent my time hacking away, and only occasionly coming up for air for food, drink and some camera action.

With over 40 people in attendance, it was going to be difficult to capture everyone, but I think I managed it. If I did miss you, my apologies. It was great to meet so many friends old and new, and a real pleasure to finally put faces to names that I've known for a while, but not had the opportunity to meet in person.

So many great things happened in Paris, and I'm really looking forward to see what we can achieve in London for the 2013 QA Hackathon. See you there.

• Photos from the 2012 QA Hackathon in Paris
  

Parisienne Walkways

Posted on 3rd April 2012

And so to the final part of my notes from the 2012 QA Hackathon.

CPAN Testers Report Status

After asking several times, Andreas thought he finally understood what the dates mean on the Status page for the CPAN Testers Reports. He started watching and making page requests to see whether his requests were actioned. On Day 3 he pointed out that the date went backwards! Once he'd shown me, I understand now why the first date is confusing. And for anyone else who has been confused by it, you can blame Amazon. SimpleDB sucks. It's why the Metabase is moving to another NoSQL DB.

The date references the update date of the report as it entered the Metabase. The last processed report is the last report that was extracted from the Metabase and entered into the cpanstats DB. Unfortunately, SimpleDB has a broken concept of searching. It will return results before the date requested, and regularly return the sorted results in an unsorted order. As such the dates you see on the Status page may go backwards in time! I'm not going to try and fix this, as it will all work as intended with the new system.

Missing Reports

There have been several questions relating to missing reports over the past few years. Sometimes it just needs me to refresh the indices, but in other cases it may be due to the fact that SimpleDB omits reports from a request. Did I mention SimpleDB sucks? In a request to the Metabase, I will ask for all the reports from a given date. The results are limited to 2500, due to Amazon's own restriction. In the returned list it will often omit entries, due to its ignorance of sorting in the search request. I have gone through the Metabase code on several occasions and can verify it does the right thing. SimpleDB just chooses to ignore the complete search request and returns what it *thinks* you want to know.

Ribasushi questioned me about one of his modules that had been released recently, which still had no Cygwin reports listed, even though he sent a few himself. Further investigation revealed that they are indeed missing from the cpanstats DB. Although they did enter the Metabase, they never came out again.

To resolve this I have been revisiting the Generator code to rework the reparse and regenerate code to enable search requests for missing periods, in the hope that this will retrieve most of the missing results. If it doesn't, then I will be asking David to produce a definitive list for me, and I will make specific requests for any missing reports. The Generator code has been updated in GitHub to include all the performance improvements that have been in live for some time too.

Erronously Parsed Reports

Every so often the parsing mechanism fails and stores the wrong data within the cpanstats DB. These days it seems to only affect the platform, OS version and OS name. I'm not quite sure what is happening, as reparsing the report locally again produces the correct results. This uses the same routine to parse the report, so why they occasional fail remains a mystery. However, to combat this, I  now have a script that can run and search periodicly for this erroneous data and attempt to reparse the results. It can then alert me when it can't fix it and I can investigate manually. The have been occasions where the report can't be parsed due to the output being corrupted on the test machine, which unfortunately we can't always resolve. Sometimes there are enough clues within other parts of the report that point to a particular OS, but sometimes we just have to leave it blank.

It seems in putting some of this code live before leaving the hackthon, I accidentally reintroduced a bug. Slaven was quick to spot it and tell me about it, but unfortunately it was too late for me to fix it, as I needed to leave and catch my flight home. It should be fixed by the time you read this though, so all should be back to your regular viewing pleasure :) With the new script I've written, it should hopefully find and fix these errors in the future, as well as alerting me to fix the bug again!

Thanks Again

So that was the 2012 QA Hackathon. The show ended with a group photo, although a few were missing due to their early departures home, but I think we got most of us in. Including Miyagawa, who was taking the picture. The traditional thanks yous and good byes ensued and then Andreas and I headed off to begin our adventure getting the airport! The next hackathon, the 2013 QA Hackathon, will be in London. We'll have the domain pointed to the right place just as soon as Andy gets the website up and running. I look forward to a lot more involvement for next year, as we have been steadily growing in numbers each year. There has already been some significant output, but the event is much more than that. It's a chance to take to people face to face, discuss ideas and plan for the future. Expect more news for CPAN Testers soon.

Once again I would like to thank ShadowCat Systems for getting me here, and for being a great supporter of the QA Hackthons, as well as many other Perl events over the years. Thanks too to Laurent Boivin (elbeho), Philippe Bruhat (BooK) and the French Perl Mongers for making the 2012 QA Hackathon happen. The Hackathon wouldn't have happened without the generosity of corporations and the communities that donate funds. So thank you to ... The City of Science and Industry, Diabolo.com, Dijkmat, DuckDuckGo, Dyn, Freeside Internet Services, Hedera Technology, Jaguar Network, Mongueurs de Perl, Shadowcat Systems Limited, SPLIO, TECLIB", Weborama, and $foo Magazine. We also have several individuals to thank too, who all made personal contributions to the event, so many thanks to Martin Evans, Mark Keating, Prakash Kailasa, Neil Bowers, 加藤 敦 (Ktat), Karen Pauley, Chad Davis, Franck Cuny, 近藤嘉雪, Tomohiro Hosaka, Syohei Yoshida, 牧 大輔 (lestrrat), and Laurent Boivin

Meanwhile, Dan & Ethne would also like to thank Booking.com for their silly putty ;)

Party In Paris

Posted on 31st March 2012

I'm currently at the 2012 QA Hackathon working on CPAN Testers servers, sites, databases and code. It has already been very productive, and already I have two new module releases.

CPAN::Testers::WWW::Reports::Query::AJAX

This module was originally written in response to a question by Leo Lapworth about how the summary information is produced. As a consequence he wrote CPAN::Testers::WWW::Reports::Query::JSON, which takes the data from the stored JSON file. In most cases this data is sufficient, but the module requires parsing the JSON file which may be slow for distributions with a large number of reports. On the CPAN Testers Reports site, in the side panel on the distribution page, you will see the temperature graphs measuring the percentage of PASS, FAIL, NA and UNKNOWN reports a particular release has. This is glean from an AJAX call to the server.

But what if you don't want an HTML/Javascript styled response? What if you wanted the results in plain test or XML? Enter CPAN::Testers::WWW::Reports::Query::AJAX. Now you can use this to query the live data to for a particular distribution, and optionally a specific version, all the result values and get them pack as a simple hash to do with as you please.

I anticipate this might be most useful to project website who wish to display their latest results from CPAN Testers in some way. They can now get the data, and present it however they wish.

CPAN::Testers::WWW::Reports::Query::Reports

Now we get to perhaps the bigger module, even though its smaller than the one above. This module is perhaps most useful to all those who are trying to maintain a version of the cpanstats metadata from the SQLite database. As mentioned previously the SQLite database has been giving us grief over the past year, and we haven't gotten to the bottom of it. Andreas suspects there is some unusual textual data in some reports that is causing SQLite problems when it tries to store it. I'm not quite convinced by this, but as I'm only inserting records, I'm at a lost as to what else be the cause.

The SQLite file now clocks in at over 1GB compressed and over 8GB uncompressed, and is starting to take a notable amount of disk space (though considerably smaller than the 250GB+ Metabase database ;) ). It is also a significant bandwidth consumer each day, which can slow processing and page displays, as disk access is our limiting factor now.

Enter CPAN::Testers::WWW::Reports::Query::Reports. This module uses the same principles as the AJAX module above, but now accesses an new API on the CPAN Testers Reports site to enable consumers to get either a specific record or a whole range of report metadata records. Currently the maximum number of records that can be return in a single request is 2500, but this may be increased once the system has been proven to work well. Typically we have around 30,000 reports submitted each day, so to allow consumers to make best use of this API, I will look to increasing the limit to maybe 50,000 or 100,000. I want to impose a limit as I don't want accidental requests being sent to consume the full database in one go, as again this would put a strain on disk access.

The aim of the module is to allow those that currently consume the SQLite database, to more regularly request smaller updates and store the results in any database they so choose. Even into a NoSQL style database. It will ultimately reduce the bandwidth, data stored and processing to gzip and bzip2, which then means we can reallocate effort to more useful tasks.

If you currently consume the SQLite database, please take a look at this module and see how you can use it. I plan to include some example scripts that could be drop-in replacements for your current processes, but if you get there first, please feel free to submit them to me too, and I will include them with full credit. If you spot any issues or improvements, please also let me know.

CPAN Testers Platform Metabase Facts

This morning we had a CPAN Testers presentation and discussion hosted by David Golden. As there is plenty of interest from a variety of parties about CPAN Testers, it was a good opportunity to highlight an area that needs work, but which David and myself, as well as other key developers in the CPAN Tester community, just don't have time to do. Breno de Oliveira (garu or IRC) has very kindly stepped forward to look at one particular task, which we have been wanting to write since the QA Hackathon in Birmingham, back in 2009!

Breno has written a CPAN Testers client for cpanminus. At the moment its a stand-alone application, but it may well be included within cpanminus in the future. As part of writing the application, Breno asked David and myself about how the clients for CPAN::Reporter and CPANPLUS::YACSmoke create the report. Due to the legacy system we came from (email and NNTP) we still use an email style presentation of the reports. However, it has always been our intention to produce structured data. A CPAN Testers Report currently has only two facts that are required, a Legacy Report and a Test Summary. However there are other facts that we have already scoped, except they are just not implemented.

Back last year the Birmingham Perl Mongers produced the CPAN::Testers::Fact::PlatformInfo fact, that consumes the data from Devel::Platform::Info (which we'd written the previous year). The problem with the way test reports are currently created, is that we don't always know the definite platform information for the platform the test suite was run on. Reports, particularly in the Perl Config section, can lie. Not big lies necessarily, but enough that it can disguise why a particular OS may have problems with a particular distribution.

Breno is now looking to produce a module that firstly abstracts all the metadata creation parts from CPAN::Reporter, CPANPLUS::YACsmoke, Test::Reporter as well as his own new application, and puts them into a single library that can then create all the appropriate facts before submitting the report to the metabase. Hopefully he can get this done during the Hackathon, but even if he doesn't, we're hopful that he will get enough done to make it easy to complete soon after. Once we then patch the respective clients to use the new library, we will then start to be able to do interesting things with how we present reports.

The CPAN Testers Reports site only displays the legacy style report, which for most is sufficient, but it really would be nice to have some specially styled presentations for particular sections, or even allow user preferences to show/hide sections automatically when a user reads a report.

CPAN Testers Admin site

This is a site that I have been working on, on and off, for about 4 years, before we even had a Metabase. As a consequence it has been promised at various points and I've always failed to deliver. Now I have release the modules above, and there have been several comments already about having such functionality, I think I need to put some focus on it again. I have shown Breno the site running on my laptop and he has given me some more ideas to make it even more useful. It'll still be awhile before its released, but this will likely be down to running with some beta testers first before a major launch, just so it doesn't break the eco-system too badly!

Essentially the site was written to help authors and testers to highlight dubious reports and have them deleted from the system. Although the reports won't actually be deleted, they will be marked to ignore, so that they can be removed from JSON files and summary requests, as well as on the CPAN Testers Report site. This will hopefully enable us to get more accurate data, and bogus reports about running out of memory or disk space can be disregarded.

However, following Breno suggestions, I will look to making the site more public, so that authors can more easily see the reporting patterns without having to log in. The log in aspect will still be needed to flag reports, but the alternate browsing of reports by testers will be much more accessible.

Thanks

I would like to thank a few people who have helped to get me here, and have enabled these QA projects, not just CPAN Testers, to advance further.

Firstly I would like to single out ShadowCat Systems, who have very kindly paid for my flight here. Thanks to BooK and Laurent for organising the event, and to all the sponsors and Perl community who have provided the funding for the venue, accommodation and food for the event. It has already been very much appreciated, and hopefully the significant submissions to GitHub and PAUSE are evidence of just how worthwhile this event is.

Thanks also to all those who are here, and are helping out in all shapes and forms to help Perl QA be even better than it already is.

Things Don't Mean What They Used To

Posted on 26th March 2012

Back in the '90s a young band touted themselves round the Midlands and sometimes further afield. Unfortunately I never got to see them, but did acquire a cassette tape at some point and remember being impressed. I was later told that the keyboard player was my mate Pete Spoz, who I'd got to know as one of the Jellyheads, who used to come along to many of the early Ark gigs. The tape is now somewhere in the loft in one of many boxes og hidden gems.

A few months ago, Pete told me that Giovanni (Pete's brother) was reforming The Sordid Details with Mick Couch, Rick Cox and himself. Here was my chance.

Sadly due to other commitments, of the three Re-Onion dates, Bromsgrove is the only one I could make. So on the 18th March, over to The Hop Pole in Bromsgrove I went for my first Sordid Details gig. Full of friends and family, the night was set for a top gig.

The band began as The Sordid Details, with a great selection of their own classics, along with a couple of covers, the first of which, Wish Away, they were joined by their good friend Ash on vocals. After about an hour they closed the first set with The Stranglers classic No More Heroes.

The second set began with a selection of songs from their days as Stereogram, when they played as a three-piece after Pete left. Pete rejoined them on stage for a run through a selection of covers, and a few more of The Sordid Details classics, before Dave from Jam DRC peformed some guest vocals with the band.

The night ended all too quickly, even though the band had been playing for over two hours! A fantastic night and I'm only sad that I won't be able to see them play The Flapper & Firkin in Birmingham on Saturday 31st March. If you happened to be in town on Saturday and fancy a great night out, you'll not be disappointed with The Sordid Details. Hopefully they'll come out retirement again some day, so I see if my bootleg tape recorder still works!

• The Sordid Details @ The Hop Pole, Bromsgrove
  

The Reasons Why

Posted on 25th March 2012

For those that follow the conference surveys, you'll be pleased to hear that I have now put the results of both the Israeli Perl Workshop and the German Perl Workshop online. These are the first events this year to take advantage of the surveys, although several more are to come.

This marks the second survey for the German Perl Workshop and notes some small differences, while it was the first for the Israeli Perl Workshop. I hope the future organisers can make use of the results and that they allow me to continue the surveys with these workshops next year, and for the years to come.

Although the Israeli Perl Workshop was in English this year, Gabor and I are hoping to be able to provide the survey in Hebrew next year. The German Perl Workshop marked the first survey not in English last year, and it helped to start building up a language pack, which can be used to plugin to the survey software. I plan to formalise this during the year, so that other events, using languages other than English, can still take advantage of the surveys.

Thanks to all the organisers and the survey participants for taking the time to respond to the questions. It is very much appreciated.

Breathe

Posted on 15th February 2012

Cold Air

I am just air
A breeze of coldness
Hold me if you dare
I have travelled far, I am breathless

This a poem written by Ethne, aged 7.

Considering she had no help with the words, it's quite a profound first poem. She wrote it on her MagnaDoodle, so Nicole managed to preserve the effort before it got erased :)

Well done Ethne.

London Calling

Posted on 7th February 2012

London Perl Workshop 2011 Survey - Results Now Online

London Perl Workshop 2011 Survey Results

I'm please to say that the survey results from the London Perl Workshop 2011 are now online. Slightly delayed due to Christmas and my new job, but worth the wait I think. This is the first time we've had a survey for the London Perl Workshop, so I was interested to see how the results differed from YAPCs. The attendees for the workshop differ from YAPCs, as although around 40% of attendees are well know within the Perl community and have attended YAPCs, most of the sttendees, like attendees for many other workshops around the world, don't have the resources or availability to attend a 3-5 day conference event. However, a one-day event, and especially a free event, makes a workshop much more accessible.

It was a shame that we only had 27% of attendees responding, but having said that while my personal aim is always to achieve more than 50% response, 27% is still a great response. As I've said previously, anything more than 10% is a good result. However, now we've done one, hopefully we can encourage more to respond this year :)

The demographic responses interestingly followed what we often see for YAPCs. I guess that may be because most of the respondees are YAPC attendees, but we still had several responses from those who have only attended London Perl Workshops, or for whom this was their first major event. The balance of Perl knowledge, although slightly weighted towards the more experience developeres, was also very pleasing to see with several beginners attending the event. Every year we have been looking to encourage newcomers to these events, as well as into the Perl community. After all, those learning Perl now are the Perl community's future. It was also great to see people being nominated or recommended to attend by colleagues and managers. The promotion of these events is obviously having the right effect.

I was intrigued to see that of all the respondees, 65 of them weren't speakers, with 30 willing to consider being a speaker in the future. Again this is something we should be encouraging, as newer speakers often have a different perspective on a subject, and can bring something new and fresh to the event. It was also encouraging that primary motivations for attending are to get together and meet other Perl developers. Events like The London Perl Workshop are a great way to introduce yourself to other developers you may have spoken to online, or are collaborating with on projects. They are a great way to promote your project, or get to know more about other projects.

In response to the question "What aspects of the conference do you feel gave value for money?", I was actually quite surprised to see comments along the lines of "Since it was free, I do not understand what "gave value for money" means." For those attendees who wondered that, how much did it cost you to attend the event? If you think your answer would be "nothing", consider the question beyond the attendance fee. How did you get there, did you walk, get the train, drive? What about your time, what would you have done that Saturday if you hadn't have gone to the event? Just because the event had no attendance fee, doesn't mean it cost you nothing to attend. Also think about what value it has given you in terms of enhancing your knowledge. Did you see talks or meet people that have inspired you, or given you a better understanding of something you were working on. There are lots of ways "value" can be interpreted beyond any monetary value.

It took a lot of people a lot of time and energy to put event like the London Perl Workshop on, not just Mark and his minions, but also the speakers and the sponsors. From their perspective it is good to know their efforts were appreciated. Thanks you to all those who did respond, and of those that didn't, hopefully we can encourage you to contribute your thoughts this year :)

Sold On You

Posted on 27th November 2011

Two weekends ago, The Paul Menel Band played their first gig with their new line-up. Debbie Saunders joins the band on sax, and adds that soulful sound back into the mix.

The venue, The Marr's Bar in Worcester was a great little club, and I hope I get to see more gigs there. It's only downside was the fact that it is typically an 18s and over type venue, due to licensing, and as a consequence there is a large young audience in the town that aren't able to get in to see gigs. This is a shame as I think we should be encouraging a younger audience to live gigs, as so many venues have closed due to lack of attendance. Hopefully The Marr's Bar doesn't suffer the same fate.

The gig itself was brilliant as always. Due to a late cancellation of the support act, Paul's son Luke Menel stepped in and provided us with a stunning performance of various acoustic indie tracks. Considering his age, Luke is truly a star in the making, and it will be great to see his musical career progress.

Being a small venue, the band's energy was bursting at the seems, and you could tell everyone was enjoying themselves. Several times during the set, Steve Harris looked lost in his only little world with licks flying along the fret board. Steve Swift, Bill Devey and Ian Diment all deftly locked the band into the groove and allowed Debbie, Dr Steve and Paul to soar, as they did often.

During Under Your Wing, Paul brought Luke back on stage to add backing vocals as he does on the forthcoming album, Three Sides to Every Story. The song itself is a very personal song for both Paul and Luke, so it was great to see them both taking centre stage. Tonight was a great showcase for the band, and hopefully some videos from tonight will eventually surface on YouTube as we had several different camera angles on the go.

In the meantime, collating my photos of the night we have the soundcheck, Luke's performance as well as the headliners.

Date: 12th November 2011
Venue: The Marr's Bar, Worcester

• The Soundcheck
• Luke Menel
• The Paul Menel Band

How Soon Is Now?

Posted on 27th November 2011

The YAPC Conference Surveys site has now been updated with the results of the Pittsburgh Perl Workshop and the German Perl Workshop.

The site has also been update to provide a tabbed display of the different types of event, to make it a little easier to find results. Over the next month or so I am looking to get more of the past data online, as well as the feedback that I normally send to just the organisers. I have lots of data waiting in the wings, and its only been my lack of free time that has prevented me from finishing off the sanity checks.

There are also plans for the future surveys, and as previously mentioned, the German Perl Workshop has given me the push to work with other languages. There is still some work to be done, but the first non-english language survey did seem to go very well. Perhaps understandably there are translations that I missed, so my next step is monitor (particularly for the results pages) London Perl Workshop what was missing, and provide Max (if he doesn't mind of course ;)) with the additional text for translation. I will then use this as a basis for all future workshops, which I will then provide via a git repo for anyone wishing to use the surveys in other languages. Note that for the short term the survey results will be presented in the same language the survey was presented, although in the longer term I would like to be able to allow switching the text (at least the questions) to english or other available languages.

The London Perl Workshop is still running, and has another 2 weeks to run. If you attended the LPW this year, and haven't completed the main survey or the talk evaluations, please take the time, as it really does help the organisers and speakers to make the events better and better.

If you're interested in running a survey for your event next year, please get in touch (barbie@cpan.org) and let me know in plenty of time, particularly if you'd like to run the survey in a non-english language.

Die Mensch-Maschine

Posted on 10th November 2011

German Perl Workshop 2011 - Speaker Evaluations

I have now sent out all the talk evaluations from this year's German Perl Workshop or more correctly Der 13. Deutsche Perl-Workshop. If you were a speaker and haven't received an email, please check your spam folders first, and let me know (barbie at cpan . org) if you don't find it. The mail will have come from barbie at birmingham . pm . org.

My thanks to all the organisers of GPW2011 and everyone who took the time to respond to the evaluations. From previous experience the speakers have very much appreciated your feedback. I would also like to extend extra special thanks to Max Maischein aka "Corion", who took the time to translate all the questions, templates and emails into German for me.

The results of the main survey will be published soon on the YAPC Conference Surveys site.

This is the first survey that I have undertaken in a non-English language, and for the most part it has been very successful. While there have been some slight problems due to byte vs character lengths (I'll save my 'why-oh-why did we ever start with ASCII and not UTF-8' rant for another day), the work Max has done to provide all the translations has started me on a path to be able to accommodate other languages.

At the moment the plan is to create a GitHub repository of all the necessary files, with language branches containing the appropriate translations. Then should anyone wish to request a survey instance in the future in a non-English language, their first step will then be to provide the necessary translations for me. It currently takes roughly a day to set-up an instance, so drop-in replacements for these files will ease the set-up process. It will also mean that as time goes on and questions get added, refined or deleted, we can replicate these changes across all languages.

I'd like to see the survey site get more use in the future, and although I'm happy to run the survey sites, with the support of Birmingham Perl Mongers, the longer term goal has always been to allow others to create their own instances. With the official release of Labyrinth this year, much of the tool set is now Open Source. I still need to release the Survey Plugin for Labyrinth and the additional command-line tools used, but getting the language translations moving will be a big step forward. Hopefully I'll have more news in the new year.