As the statutory consumer watchdog for postal services, it is the responsibility of Consumer Focus to scrutinise this data and help the public hold the Royal Mail to account.

Recently, we’ve been working with our policy team in Consumer Focus Wales to see how we can improve the transparency and visibility of the information we receive from the Royal Mail. Today we are pleased to launch the Wales Mail Report – a tool to make this data available to the public in an easily digestible form.

The Wales Mail Report takes the quality of service and complaints data from numerous large PDF documents and presents it in HTML tables with interactive charts. We’ve broken the data down by postcode area so you can see how well the Royal Mail is performing in your area. We’ll also be updating the data every time new figures are published.

Best of all – we’ve made everything available in open formats. You can download all the information in CSV and JSON formats under the Open Government Licence.

Although the tool only covers the postcode areas in Wales at the moment, we hope to be able to do the same for other parts of the UK over the next few months.

Let us know what you think and if you re-use the data we’d love to see what you’ve been able to do.

Our supplier has worked to replace the faulty equipment and bring our services back online.  This work was completed at around 2.15pm this afternoon.  There are however a few outstanding issues with some of our websites:

• We have taken the decision to temporarily suspend new registrations on StayPrivate.org for a few hours to ensure that the server is stable and no problems occur.
• Recalled Product alert emails that should have been sent this morning have not been sent.  We will schedule these to be sent later today.

This blog and the CF Labs twitter account will be updated when the items above are resolved. We will be monitoring situation for the next few hours to ensure everything is running normally and will publish a more detailed post later in the week once we’ve had a chance to discuss the cause with our supplier.

We apologise for any inconvenience this outage may have caused.

Update (18:27): We are now accepting new registrations on StayPrivate.org.

Update 2 (21:26): This morning’s recall alerts have now been sent.

Since the first report was published in March 2010, a new rating scheme has been developed by the Food Standards Agency (FSA). This scheme will give all food premises in England and Wales a rating between 5 (very good) and 0 (urgent improvement necessary) dependent on the outcome of hygiene inspections.

The second of these annual reports was launched at the Senedd (the home of the National Assembly for Wales) earlier today. While compiling this report, staff identified more than 60 premises serving food to vulnerable people that achieved a substandard rating (2 or below). These premises included schools, colleges, nurseries, care premises and hospitals.

We were asked if we could plot these premises on a map to show where they were located to accompany the report.

Luckily, the Food Standards Agency publishes all of their content under the Open Government Licence. This meant that we could build a scraper to work through the food hygiene website to pull out the qualifying ratings to use on the map.

We also used open data from mySociety’s MaPit service to pinpoint the location of each premises so we could plot it on a Google Map. The MaPit service uses data published under open licences by Royal Mail and Ordnance Survey.

If you want to know more about this work, take a look at the press release, published report and map. Welsh language versions of the press release, report and map are also available.

The biggest change you’ll notice is the design of the site. It’s more of an evolution of the design than a brand new look – we’ve kept the same structure, but given it a bit more of a “CF Labs” style (similar to StayPrivate.org and Community Post Offices).

We’ve increased the width of the site which has allowed us to improve the layout of the product recall notices. We’ll be making better use of this extra space on other parts of the site over the coming months.

We’ve also spent some time working on the recall statistics we generate – in some cases, we felt they weren’t particularly clear. For example, the chart showing the top dangers in recall notices included “chemical” near the top, but further down it also included “chemical and damage to sight”, “chemical and drowning” and “injuries, choking and chemical”.

This meant that some of the figures weren’t fully representative of the real numbers. As a result, we’ve completely rebuilt the statistics engine to take account of this kind of thing. This has also enabled us to output the data in CSV and JSON formats for those that want it. As always, these statistics are available under the Open Government Licence.

The other big change we’ve made is to the scraper we use to import recall notices from the European Union RAPEX system. Occasionally odd characters crept onto the notices where certain special characters hadn’t been encoded properly by the source. We’ve updated the scraper so it handles this encoding on their behalf prior to importing to reduce the likelyhood of this happening in future.  To improve the existing data on the site, we’ve gone back and re-imported the latest 300 recalls (everything from January 2011 and most of December 2010).

Occasionally RAPEX make changes to notices that have been published such as fixing errors, adding additional information or removing notices that have been cancelled. While we were picking up and replicating these changes before, our updated scraper allows us to detect these in much greater detail.

To ensure users can see an accurate history of each recall notice, we’ve now included a column on the notice’s page that will list any changes made. Where a text change has taken place, we’ll include the original text for comparison. You can see an example of this on one of the recalls we re-imported yesterday.

We’ve got more improvements planned over the next few months, so keep an eye out for updates on this blog.

Data encryption

When you sign up to StayPrivate.org you provide us with a significant amount of personal data. At a minimum you provide us with your name, email address and postal address. If you are a business, you also provide us with details about your organisation.

You then have to provide varying amounts of data depending on which of the marketing opt-out services you wish to use. This includes things like telephone numbers (for the TPS, CTPS & FPS) and names of other people at your address (for the MPS).

All of this information is encrypted when it is stored in our database. For example, my name ‘Dafydd’ becomes something like ‘A0JQNABiWi5SawY2’ or ‘BEUEYFQ2WCwHPgAw’. Without the encryption key, we can’t decode it to see what it means and neither can anyone else.

Data storage

This is another key element in our service – we don’t store data for any longer than is absolutely necessary and you can ask us to destroy it at any time. The options open to you depend on what you select when you register.

If you don’t sign up for an account, we will destroy the data once we’ve registered you with the opt-out services. This is done automatically & there is nothing for you to do.

If you signed up for an account but didn’t tick the “store data” box, we’ll automatically destroy everything except your name and email address once we’ve registered you with the services. You can ask us to destroy the rest by clicking the “Close Account” button in the dashboard.

If you signed up the account and ticked the “store data” box, we’ll keep it until you ask us to destroy it. We’ll send you a reminder every year and ask you to confirm you still want us to keep it – if you don’t, we’ll destroy it automatically. You don’t need to wait for the reminder however – at any point you can ask us to destroy it by clicking the “Close Account” or “Delete Data” buttons in the dashboard.

Secured Connection

On Monday this week we launched full SSL encryption on StayPrivate.org. If you’ve ever used online banking or shopped online, you might be familiar with the padlock that appears in your web browser to show that the webpage is secure.

This means that your web browser is using a secure connection (HTTPS) to communicate with the website to ensure the data you send and receive remains private.

From this week, StayPrivate.org is only available using a secure connection, so you will see that padlock on every page of the website. Whenever you view data on the site or send data to us, it will be secured and will remain private.

This is an important step forward for StayPrivate and ensures that security and privacy of your data is being taken care of at each step in the process.

There are many different reasons for wanting to stop automated scripts from submitting forms. For example, scripts are regularly used to submit spam comments to blogs or to attempt to guess a user’s password by trying many different combinations very quickly.

Unfortunately CAPTCHAs introduce a big accessibility problem as they rely on visual perception (being able to see and understand the text). Any users with visual difficulties (for example, users with a disability) will potentially have trouble completing the test correctly. Anyone who uses a screen reader to browse websites will not be able to complete the form because it cannot read the image – after all, if it could, it would defeat the whole object of the test. Some CAPTCHA tests also include an audio version alongside the image, but this does not resolve the issues completely.

Because of these accessibility issues, we try and avoid using CAPTCHAs on all CF Labs applications. We try and make use of other methods such as email verification (where we send you a link in an email which you need to click) and only use CAPTCHAs as a last resort.

In the case of our StayPrivate.org service, we’ve built a CAPTCHA into our ‘log in’ and ‘forgotten password’ process – however we hope that the vast majority of users will never see it. While we want to reduce the possibility of an automated script trying to break into a user account, we didn’t want to impose a CAPTCHA every time you try to log in. Instead, we only show it if you enter the wrong email address and password combination three times in 15 minutes. This method means that the vast majority of users never see the image (and so do not face the accessibility issues), while we can reduce the threat of automated scripts.

To assist the minority that do activate this test, we’ve chosen to use the reCAPTCHA service which includes an audio version as well as the standard image.

This is not a perfect solution but hopefully it is a good compromise between security, accessibility and ease of use.

Last year, Consumer Focus appointed a new supplier of the data required to run the service, which necessitated a complete rebuild of the system. Due to limited time and resources, we opted to undertake the rebuild in two phases:

1. Build new back end and database with basic front end
2. Build new front end

In January we launched the first phase of the project. From a visitor perspective, they would have noticed very few changes – we essentially copied the existing interface design and hooked it up to the new database. The biggest change was behind the scenes – we pretty much started from a blank sheet of paper.

Previously we received new data every week or so with updated prices, now the new supplier (Energylinx) provide us with an XML feed every time a price change occurs – this ensures we have much more accurate data. We also changed platforms – moving from Windows/ASP.Net to Linux/PHP.

A few weeks ago, we completed the second phase and launched a brand new front end to the site. Previously the EPC was stuck in an iframe on the Consumer Focus website, now it carries the full template and appears as if it is part of the website.

We’ve included various improvements such as bookmarkable URLs for price searches (previously you would have to fill out the search form every time you wanted to check prices), clearer highlighting of cheapest prices, and details of when the prices were last updated.

We’ve reintroduced the PDF Fact Sheets and improved the monthly emails which are sent out. We’ve also borrowed some of the elements of the StayPrivate.org “account” system, to improve the way users can sign up for the monthly alerts and manage their settings.

As always, we’re not done and will continue to make improvements over the next few months. If you have any suggestions or feedback let us know.

The team has been compiling quarterly data on the number of complaints about energy companies received through various sources including Consumer Direct, Consumer Focus’ own Extra Help Unit and the Energy Ombudsman.

We’ve worked closely with the team to design and develop a service for publishing this data in a meaningful format. CF Labs produced a number of prototypes designs which were refined through various stages before eventually reducing to 3 possible options. These options then underwent independent testing by TNS-BMRB in September which helped us to pick the final option.

Yesterday we launched the Energy Supplier Performance ranking system on the Consumer Focus website. The system ranks each of the ‘Big Six’ energy suppliers using stars – 0 for the worst performing companies, 5 for the best. You can read the methodology behind the rankings on the website and in the accompanying press release. The energy team will be publishing updated rankings on a quarterly basis.

As well as providing a ‘snapshot’ of the current rankings, we wanted to display the data in a bit more detail so we’ve included a second page to do this. This page contains a table and interactive graph showing the performance changes over time using a three month rolling average.

We’ve also published this data in JSON and CSV format so it can be easily reused. You can access these formats from the ‘Historical Data’ page. As usual, we’re publishing this information using the Open Government Licence for public sector information.

At the moment, the ranking system only covers the ‘Big Six’ – Scottish and Southern Energy (Atlantic, Scottish Hydro Electric, Southern Electric & Swalec), British Gas (and Scottish Gas), E.ON, Scottish Power, EDF Energy and npower – as these cover 99% of the domestic energy market. Consumer Focus is looking at publishing similar information for smaller suppliers next year.

We appreciate any feedback you may have on the system.

CF Labs was asked to find a way of presenting some of this information and making it available as an online resource. On Sunday we launched Community Post Offices, a simple directory to publish this data.

The website allows you to search for community post offices in your area, find out where they are located and see what services they provide. If the post office has a website, we’ve provided a link so you can find out more details. We’ve also built a form to allow other community post offices to register to be listed on the website.

We’ve also used the data to produce a few pages showing statistics such as the core activities, legal structures and services provided.

You can read more about the Community Post Offices project from the Consumer Focus press release.

Third Party Tools

We’ve made extensive use of MaPit from mySociety and UK Postcodes from Pezholio to provide location information from the organisation’s postcodes. This has allowed us to power the postcode search and pinpoint the location of each post office on the embedded Google Maps.

We’ve also used Google’s Javascript Charts API to produce the graphs on the statistics pages and Scribd to embed the accompanying report.

Data Licence

All the non-commercial data on the website is being released using the Open Government Licence for public sector information. This means that you are free to re-use the information as long as you acknowledge us as the source. More information about the licence is available on the National Archives website.

To make it easier to reuse the information, we’ve published most of it using machine readable formats. Initially we’ve done this using the JSON format and CSV for the statistics. We’ve published instructions on how to access these formats on the website.

Feedback

As always, we appreciate any feedback you have. Either leave a comment here, or email communitypostoffices [at] consumerfocus.org.uk.

We will be upgrading the Consumer Focus website throughout the day on Wednesday 6th October Thursday 7th October.  During this work, there may be holding pages in place and the website may be unavailable for short periods of time.

We apologise in advance for any inconvenience this may cause.

More Detail

The Consumer Focus website is currently powered by the WordPress MU content management system.  During the day we will be upgrading the website to use the latest version of WordPress (3.0.1) which provides more stability and enhanced features.

We are also taking this opportunity to restructure content and introduce a refreshed template to improve the presentation and make the site easier to use.

Updates

We’ll be posting updates during the work through the @cflabs twitter account.

EDIT: Due to unforeseen circumstances we had to delay the maintenance by 24 hours.