Memoirs of a Roadie

Highway Star

Posted on 15th September 2013

A few years ago I posted about whether the M6 viaduct over the Gravelly Hill area of Birmingham, was the longest bridge in the UK. In February 2012, someone added a page to Wikipedia for the bridge, which appears to be called the Bromford Viaduct. The Wikipedia page references the same page I found from the Motorway Archive, but nowhere on that page did it reference the name. So now I had two questions:

1. Is the Bromford Viaduct its official name?
2. Is this still the longest bridge in the UK?

Regards the name, it would seem that the proposals for HS2 refer to it as the M6 Bromford Viaduct. After some further searching, it would seem that The Highways Agency calls it the Bromford Viaduct too, so I guess that is its official name, seeing as they are the government department in charge of UK roads. I still think the Spaghetti Viaduct sounds better :)

However, the second question still remained unanswered. On the Motorway Archive page it states the bridge "was then the longest continuous viaduct in Great Britain", which implies it no longer is. Having said that, in November 2010, a few months after my initial post, it seems someone else had thought the same thought, and added the bridge to the Wikipedia page for the longest bridges. Seeing as there is no other mention of a longer UK bridge and the Second Severn Crossing is measured as being 472m (1576ft) shorter, I think, until proven otherwise, Brummies can be proud to have the longest bridge in the UK.

Of All The Things We've Made

Posted on 26th August 2013

Several years ago, we frequently updated the Birmingham.pm website with book reviews. To begin with, updating all the book information was rather labourious. Thankfully, on CPAN there was a set of modules that had been written by Andrew Schamp, that provided the framework to search online resources. I then wrote drivers for Amazon, O'Reilly & Associates, Pearson Education and Yahoo!. As the books we were reviewing were technical books, these four sources were able to cover all the books we reviewed.

A few years ago, I started working for a book company. In one project, we needed to evaluate book data, particularly for books where we had no data or very little. Often these were imports or out of stock titles that we could still order, but we were lacking information about. As such I created a number of further drivers, particularly for non-UK online catalogues, to help retrieve this information. I managed to create a collection of 17 drivers, and 1 bundle, all available on CPAN.

Via my CPAN Testers work, I've been promoting the CPAN::Changes Kwalitee Service website. Neil Bowers read one of the posts, and thought it would be good to improve the Changes files in his distributions, by way of QuestHub. I'd not heard of this site before, but after reading Neil's post I joined up, as I had been looking for a suitable way to keep a TODO list of my Perl work for a while. Neil had created a stencil to standardise the Changes file in 5 distributions, but unfortunately, I only had a few distributions of my own to complete. Another stencil emerged to add License and Repository information to 5 CPAN distributions. Again, I'd completed this for most of my distributions, apart from my 18 ISBN distributions, which I'd never got around to creating repositories for.

Then Neil had the idea to look at some of the quality aspects of all the CPAN distributions, and highlight those that might need adoption. As part of his reviews of similar modules over the past few years, he's adopted several modules, and was looking at what others he could help with. The results included 2 of the modules written by Andrew Schamp, which formed part of the ISBN searching framework I used for my ISBN distributions. Seeing as they hadn't been touched in eight years, I suspected that Andrew had moved on to other languages or work. So I contacted him to see whether he was interested in letting me take the modules on and update them.

It turns out that Andrew had written the modules for a college project, and since moving to C and with his programming interests now nothing to do with books, he was happy to hand over the keys to the modules. Over the past week, I have now taken ownership of Andrew's 5 modules, added these and my own 18 ISBN distributions to my local git repository, added all 23 to GitHub, updated the Changes file, and License & Repository info to the 5 new modules and released them all to CPAN. My next task is to update the Repository info in my 18 ISBN distributions and release these to CPAN.

Although I don't work in the book industry anymore, writing these search drivers has been fun. The distributions are perhaps my most frequently releases to CPAN, due to the various websites updating their sites. Now that I have access to the core modules in the framework, I plan to move some of the repeated code across many of the drivers into the core modules. I also plan to merge the three main modules into one distribution. When Andrew originally wrote the modules, it wasn't uncommon to have 1 module per distribution. However, as all three are tightly bound together, it doesn't make much sense to keep them separate. The two drivers Andrew wrote have not worked for several years, as unsurprisingly the websites have changed in the last 8 years. I've already updated one, and will be working on the other soon.

It's nice to realise that a few of my CPAN Testers summary posts inspired Neil, who in turn has inspired me, and has ended up with me helping to keep a small corner of CPAN relevant and up to date again.

If you're a new Perl developer, who wants to take a more active role in CPAN and the Perl community, a great way to start is to look at the stencils on QuestHub, and help to patch and submit pull/RT requests to update distributions. If you feel adventurous, take a look at the possible adoption list, and see whether anything there is something you'd like to fix and bring up to date. You can also look at the failing distributions lists, and see whether the authors would like help with the test suites in their distributions. You can then create your tasks as quests in QuestHub and earn points for your endeavours. Be warned though, it can become addictive :)

There is one more ISBN distribution on the adoption list, and I have now emailed the author. Depending on the response, I may be going through the adoption process all over again :) [Late update, the author came back to me and he's happy for me to take on his distribution too]

Who Knows Where The Time Goes?

Posted on 24th July 2013

YAPC::NA 2013 - The Results Are Out

The YAPC::NA 2013 Conference Survey results are now online.

The number of responses was much lower than in previous years, which is a shame, but may in part be due to one comment I received, saying it was too long. Reviewing the survey, I'd have to agree, and I'll be removing some of the questions for future surveys. Some of the questions had good intentions originally, and did provide an insight to what people got out of the conference. However, there is now a degree of predictability about them, that doesn't warrant their inclusion. Such questions about holidays and speakers you missed really don't add anything any more. The latter has generated some interesting comments over the years, but typically the same names appear each year.

This year was also slightly different, as the organisers asked for a lot of additional questions. Particularly related to the Code of Conduct. I will be forwarding the results of these questions to the TPF in the next day or two. They may choose to make the results public, but for now they won't appear on the YAPC Survey site. Of the other questions they asked, most related specifically to YAPC::NA, and wouldn't be applicable to other conferences and workshops. These too will be reviewed for next year.

Interestingly, VM Brasseur has done some analysis of the survey data, particularly around the age of attendees, and the length of time people have been a Perl programmer. Although the survey includes the former, it doesn't really include the latter. We do ask what level people feel they are at, but it'll be an area I'll be reviewing for future surveys.

As both the surveys and VM's analysis shows, the Perl community (at least those answering the survey) is getting older. I've noticed this too when attending. There are new and younger people attending, but generally the audience has been getting older. In the UK, this was identified in an technical article I read a few years ago (sadly I don't have a link to the source), which highlighted a shift in the late 80s/early 90s away from writing computer games on Spectrums, Dragons and Beebs to just playing consoles. I suspect the age of attendees at other technical conferences are also seeing a shift.

As noted in a previous post, I'm going to be looking at the Conference Survey software over the summer, and hopefully integrate it more with the Act software. I'm hoping this may encourage more to respond. I'll also be reviewing the survey itself, and looking at better and more relevant questions to include. If you have ideas of how to improve the survey, please feel free to drop me an email.

Enjoy :)

One Way Or Another

Posted on 1st July 2013

YAPC Conference Surveys Update

Last night I managed to get all the talk and tutorial feedback emails out to the speakers from YAPC::NA 2013. If you were a speaker at the event, and haven't received an email from me, first check your spam filters, and then contact me, and I can resend yours.

This year we had 351 talk evaluation responses, 29 course evaluation responses and 121 conference survey responses. Quite a few talks had no responses, which is a shame, as I'm sure the speakers had several attendees for all their talks. Talk and tutorial feedback is exceptionally useful to speakers, especially first time speakers, as it gives them the opportunity to see what the audience thinks they need to improve on. Often a first time speaker might think they didn't go over well, only to find the feedback very positive. Most feedback is rarely negative, and where areas of the talk can be improved, the critique is given in way to help the speaker for the future. If you didn't give any talk feedback this year, please consider doing so in the future.

The responses for the conference survey was quite low compared to previous years. I had one private email simply say it was too much, which in part I think is due to the extra section this year. There has been a lot of controversy about the Code of Conduct this year, and the organisers wanted to find a way to get thoughts about anonymously from the attendees, the survey being the ideal way to do this. I shall be sending the results and comments to TPF this coming week, for them to consider what improvements can be made. If you didn't complete the section in the survey, and want to air your views, please send your thoughts to admin<@>yapcna.org.

The remaining parts of the conference survey will be online in the next week or so.

In other news both survey results for the last London and German Perl Workshops are now online. Apologies for taking so long to sort these out. I've had rather more distracts this year than normal, and getting them finished took much longer than anticipated. However, there has been some work in the survey software, particularly with regards to formatting the results, to speed up the process. It also helped that I had someone with an interest in survey software that wanted to use the code base for their workplace. The fruits of the changes will finally be making it to GitHub and CPAN over the summer.

While making the improvements, and getting the YAPC::NA 2013 survey online, some problems came to light that could be improved somewhat by allowing the Act conference software to talk to the conference survey software, and not just the other way around. Currently, the survey software can talk to an Act instance and retrieve the list of paid attendees and talks. However, it currently doesn't get the list of registered attendees (applicable to free conferences such as London Perl Workshop), or the list of courses and course attendees for large events (such as YAPCs). The latter is a little tricky to do, due to the way courses are stored, but I believe it can be done. As such, over the next year, I aim to provide some patches to allow the survey software to expose a little more of this data.

For several years I have also been asked if there is a way to integrate the survey into Act, as people forget their links and lose emails. In addition, I always receive a regular complaint from one attendee that I send spam, with the 3 emails I (usually) send for the survey. As such, I'm looking at writing further patches to firstly provide the survey link to the Act instance the attendee has signed up for, and secondly to provide an unsubscribe link, so those not wishing to take part in the survey at all, can decline further email reminders. Again I'll be aiming to work on this for next year's round of YAPCs.

If you have suggestions for further improvements to the survey, please let me know. Once I get the source code packaged and released, I'll organise a more formal method for submit ideas and questions.

The Time of the Turning

Posted on 7th May 2013

A few weeks ago I had the pleasure of attending the 6th annual QA Hackathon. The event has become THE event for developers of test modules, projects and toolchain applications to come together to discuss ideas and plan for the future, as well as release some great work while they are there too.

This year Shadowcat, the primary sponsors, took on the organisational duties. The event was originally to be in London, but due to personal circumstances the decision was made to move the location to Lancaster in the North West of England. Personally they made the right choice. The venue itself was the new InfoLab building at Lancaster University. The attendees came from far and wide once again, and it was great to catch-up with friends old and new, and even be introduced to some newer friends.

My plan for the weekend was mainly to look at CPAN Testers. With the servers for the Metabase coming soon, David Golden and myself had hoped to be able to set them up, and start looking at changing the backend code to work with the new Metabase database. Unfortunately, the servers weren't ready for us just yet, so I started looking at other things. For myself, one area of CPAN Testers, particularly the cpanstats database side of things, needed attention. Speed of processing reports.

My first task once settled in, was to look at the way that the reports are consumed from the Metabase. Due to the way SimpleDB has become very unreliable with the results it sends, in order to avoid missing reports the criteria for the date search has been altered slightly to be a little more thorough, and a smaller range is now used to retrieve a set of GUIDs. The results now appear to be a little more complete, although we still appear to be missing some every so often. There is also a tail of log.txt which also helps to catch up with the reports. This work saw a new release of CPAN-Testers-Data-Generator.

A big factor with the slowness of the CPAN Testers server is that it requires a lot of disk I/O, with the database updates being a key factor. The most intensive updates are surrounding the SQLite database that could be downloaded. This also includes creating the Gzip and Bzip2 archives. As only web crawlers seem to be downloading the files, I've suspended the update. This has now freed up a lot of resources and consequently some of the other tasks, particularly the builder has improved.

Next, the builder was the focus of my attention. Previously the builder has been building pages for both authors and distros all at once. Although the author pages are viewed slightly less, they were getting built more frequently, due to the way the requests are pushed into the queue for each report. Initially the logic for building pages was altered, which improved some of the higher requested pages, but the more optimal solution was to split the builder into two, one for authors and one for distros. With the reduction in processing elsewhere, this improved the builder performance considerably. Monitoring the way the author pages are built since the hackathon, has also allowed me to alter when the builder for authors runs. This has then allowed the builder for distros to take a higher priority. With more distro pages than authors, this now gives distro pages more opportunity to be built quicker. Currently reports are being built in less than 24 hours of being submitted. These updates saw a new release of CPAN-Testers-WWW-Reports.

Another release while at the event, related to the QA Hackathon itself, was the main QA Hackathon website. Before the event, BooK had asked if the files that make up the website that the main QA Hackathon uses could be added to GitHub. As such, I packaged up the site into a git repository and released it. If you wish to help contribute to the site, please do.

Although there was a lot of coding work involved in the weekend, one of the bigger uses of time was the Lancaster Consensus organised by David Golden. For a few hours each afternoon, a large group of key toolchain developers, secondary project developers and various interested parties, gathered to discussed various aspects associated with configuration, installation, testing and specification of Perl and CPAN. With so many developers in one room, it wasn't too surprising to have a few opposing views, but with a guiding hand from David, we did achieve a consensus. If you wish to read the outcome, please read David's write-ups of the discussion points. The Consensus meetings were perhaps the greatest achievement of the event. While there might not have been too much immediate coding output from them, the potential to improve Perl and CPAN is considerable. From a CPAN Testers perspective, Post-installation testing, Case insensitive package permissions and Rules for distribution naming were perhaps of most interest. Although it may be some time before Post-installation testing could be hooked into a CPAN Testers smoker, it will be a valuable addition to the testing reports against pre-installed environments.

During the event, I had several discussions with Garu regarding his work on the cpanminus smoker client, and the common smoker client. In the last minutes of the hackathon we were able to push through a very notable report submission. It is exactly this sort of collaborative effort that makes these hackathons worthwhile. I look forward to see everyone again in Lyon.

The QA Hackathons could not be the success they are with the support of all the sponsors. My personal thanks to them for helping to providing accommodation, food and a venue for us all to hack. A big thank you to cPanel, Dijkmat, Dyn, Eligo, Evozon, $foo, Shadowcat Systems Limited, Enlightened Perl Organisation and Mongueurs de Perl.

To Wish Impossible Things

Posted on 4th May 2013

The QA Hackathon website has had a bit of an update today. Primarily a new page and new photos have been added, but plenty of other updates have been included too.

The new page is a review page, to collect various blog and news posts relating to each year's event. Originally I listed all the reviews from previous years in the side panel, but now that we've just had the 6th annual event, the list was looking a little bit too cramped.

With the extra space, I've also been able to include the group shots that were taken at some of the events. Unfortunately there was no group shot taken in Birmingham, and I've not seen any during the 2010 and 2011 events, so if there are any, please let me know. Also if there is one of the Tokyo Satellite event this year I would love to include it on the site.

I've added some write-ups to the last few events in the About page. The biggest change though is likely only visible to those with screen readers, as I've made many changes to links and images to provide more accessibility. Several fixes to layout, spelling and wording have also been included too.

The site, particularly the list of reviews, is still incomplete. If a blog entry is missing that you think should be there, or you spot other items that could do with an update, feel free to email me with details, or fork the repo on GitHub and send me a pull request.

Enjoy The Silence

Posted on 4th May 2013

Unfortunately I've been having problems with my outbound mail from my personal mail server. I recently updated the server, and although the mail software was also updated, the problem seems to be regarding resolving IP addresses to the Virgin Media nameservers. I can resolve DNS lookups via my laptop, but the server gets stuck and times out :(

If I don't get it fixed soon, I'll have to look at using Gmail on a more permanent basis, so that I can at least reply to messages again and know they're going to get through. Although I don't mind using Gmail on occasions, storing all my mail with them does make me a little uncomfortable.

So if you haven't heard from me, and have been expecting a response, it's likely that I replied, but it never got through. By all means send me a reminder so I know to send a reply through Gmail to make sure you get the reply.

I've also been quiet on my blog for quite sometime. I have been rather busy with work, family, work, Perl, open source and work, and finding time to write here has been low on my list of things to do. However, I have a few posts coming soon to help redress the balance.

Lost In The Echo

Posted on 26th August 2012

I've just released new versions of my use.perl distributions, WWW-UsePerl-Journal and WWW-UsePerl-Journal-Thread. As use.perl became decommisioned at the end of 2010, the distrubutions had been getting a lot of failure reports, as they used screen-scraping to get the content. As such, I had planned to put them out to pasture in BackPAN. That was until recently I discovered that Léon Brocard had not only released WWW-UsePerl-Server, but also provided a complete SQL archive of the use.perl database (see the POD for a link). Then combining the two, he put up a read-only version of the website.

While at YAPC::Europe this last week, I started tinkering, and fixing the URLs, regexes, logic and tests in my two distributions. Both distributions have had functionality removed, as the read-only site doesn't provide all the same features as the old dynamic site. The most obvious is that posting new journal entries is now disabled, but other lesser features not available are searching for comments based on thread id or users based on the user id. The majority of the main features are still there, and those that aren't I've used alternative methods to retrieve them where possible.

Although the distributions and modules are now working again, they're not perhaps as useful as they once were. As such, I will be looking to merge both distributions for a future release, and also providing support to a local database of the full archive from Léon.

Seeing as no-one else seems to have stepped forward and written similar modules for blogs.perl, I'm now thinking it might also be useful to take my use.perl modules and adapt them for blogs.perl. It might be a while before I finish them, but it'll be nice to have the ability to have many of the same features. I also note that blogs.perl.org also now has paging. Yeah \o/ :) This has been a feature that I have been wanting to see on the site since it started, so thanks to the guys for finding some tuits. There was a call at YAPC::Europe for people to help add even more functionality, so I look forward to seeing what delights we have in store next.

Every Day Is Exactly The Same

Posted on 15th July 2012

Recently Mark Keating of the Enlightened Perl Organisation created a new Google Calendar for Perl community events, particularly for Perl Monger group meetings. As I haven't been updating the other calendars I have access to for some time, it gave me the push needed to clean-up my script, and post all the forthcoming events to the calendars.

I've now updated the Birmingham.pm events page, to display the new calendar, as well as the West Midlands Tech Events calendar.

If you have access to any similar calendars, you can now update them with Perl (if you weren't already), with the aid of my helpful script. Feel free to use and abuse as you wish. Note that you will need to have a login to Google Calendars, and have access to the calendars you are submitting to.

Nomzamo

Posted on 19th May 2012

Earlier this month, on the 5th May 2012, The Paul Menel Band, stripped down to Paul and Steve, together with special guest Martin Orford, provide support to Pendragon at The Assembly in Leamington Spa. They may have been the supporting band, with an early time slot, but word had got out that doors were opening early, as nearly half the venue was full by the time the guys took to the stage. A great turn out, as even the Assembly staff noted.

Paul and Martin came on stage first for a rousing rendition of the Piano/Vocal Version of No Love Lost. A great start to the set, and got everybody eagerly awaiting the other delights from the Nomzamo 25 Years Anniversary Tour. This tour was a celebration of the release of Nomzamo, the album Paul and Martin first joined forces on, when they were in IQ back in 1987. 25 years later, the songs are as classic as they were back in 1987. Paul then welcomed Steve to join the two on stage. The Nomzamo set then continued with one of the best live performances of Promises (a personal favourite) and Nomzamo, the title track of the album.

To break the set up, the band added a few surprises throughout the set. The first surprise was a track from Martin's latest solo album, The Old Road, with Paul and Steve doing a fantastic job of Ray Of Hope. Common Ground began with Steve sitting on the floor with the acoustic guitar, having forgotten to arrange to have a stool on stage, and ended with some great electric guitar fret work, much to the appreciation of the audience. The next surprise, of sorts, was the Paul Menel Band song She's Up On The Chair Again, taken from the forthcoming album. Then on to the final song from the Nomzamo part of the set, Still Life, with yet more amazing guitar solos from Steve.

As so to the final song of the night. Although I knew ahead of the set what it would be, I hadn't seen the rehearsals, so wasn't quite expecting the result. As Steve stepped forward to the mic I was just expecting some backing vocals, and was just as surprised and impressed as everyone else when he took on the Roger Waters vocals for Comfortably Numb. Paul and Martin handled Dave Gilmour's vocals to great effect too. It was an unusual song for the set, but an absolute delight too.

Seeing Martin on stage again after so many years, was great, to see him playing and singing was fantastic, to see that he and Paul sounded as good as they did 25 years ago, if not better, was a joy. The following weekend they repeated the performance at De Boerderij, Zoetermeer, Holland, to just as much applause. Hopefully, this won't be the last time we see Paul and Martin on stage together, as there is already talk of an Are You Sitting Comfortably 25 Years Anniversary Tour in 2014 :)

If you missed the gig, fear not we recorded the whole event, and you can now watch the full set on YouTube. You can either watch via the playlist, or by clicking the links below to view the individual song performances:

• 01 - No Love Lost (Piano Version)
• 02 - Promises
• 03 - Nomzamo
• 04 - Ray of Hope
• 05 - Common Ground
• 06 - She's Up On The Chair Again
• 07 - Still Life
• 08 - Comfortably Numb
  

Photos coming soon.