Paying for postcodes

,

David Howarth MP
House of Commons
London
SW1A 0AA

Dear David Howarth,

I would like to encourage you to sign Early Day Motion 2000, proposed by Tom Watson:

That this House notes with concern the attempts of Royal Mail to restrict access to the postcode database for not-for-profit web services; further notes with alarm that this heavy-handed approach has led to not-for-profit websites which seek to provide essential services to the public being left unable to function; and calls on the Government to ensure that the database is made freely available to anyone for not-for-profit use, thus enabling citizen-focused projects to flourish and innovate.

The Postcode Address File is an important piece of our national information infrastructure: it links postcodes with street addresses and geographical position. This data is critical for research, websites, and information services concerned with geographical locations within the UK. The Royal Mail owns this database, and charges users £3,750 per year1. This makes it wholly unaffordable for ordinary members of the public, community organizations, small charities, and so on.

Here are four examples of not-for-profit websites which would benefit from the Postcode Address File but cannot afford the current licensing fees:

The ongoing expense of using the Postcode Address File means that there are many community websites and other applications that people would like to build, but cannot afford. We would all benefit if there were some way for individuals and non-profit organizations to freely license the Postcode Address File.

Yours sincerely,

Gareth Rees

  1. ^ I’m not 100% certain about this figure: I’m presuming that an online service would need to have a “system licence” (covering a single computer, presumably a server responding to postcode lookup queries) or an “end user per click licence” (using the Royal Mail’s own online lookup API). But maybe there’s a cheaper option that I didn’t spot?

Background

From 2004 to 2006, the Postal Services Commission (Postcomm) held a public consultation into the future mananagement of the PAF. This was contemporary with the emergence of not-for-profit web-based public services (for example, the first MySociety web site, WriteToThem, was launched in 2005), so it’s not surprising that the public responses don’t make the kind of public benefit argument that I made above. Generally the concerns are about licensing terms rather than price, from the point of view of business users of the PAF, and especially from resellers and competitors.

Nonetheless, several of the public responses to the consultation argued for cheaper or free licensing of the PAF on public policy grounds. David Dorricott of AFD Software (a reseller of the PAF, and involved in a legal dispute over the terms of their license):

AFD’s view is that postcodes are an integral part of UK address lists and are ubiquitous in areas that are nothing to do with delivery—such as insurance underwriting or public administration. Database owners must be feed from Royal Mail’s continued claims to own property in their databases.

Christopher Roper of PointX (also involved in a dispute over PAF licensing) presented more detailed examples of public disbenefits arising from the high cost of PAF licences:

Is it reasonable, for example, for Royal Mail to maximise the revenue it derives from value added resellers and end users, even if such a pricing regime excludes legitimate use by, say, parts of the public and voluntary sectors and low volume commercial users? A major source of inaccuracy in the public registers maintained by the Environment Agency, for example, is the failure to use software that would ensure the transcription of PAF-compliant addresses. This may be a product of poor clerical procedures within the Agency, but it might equally stem from the requirement to pay for multiple PAF licences across all the Agency’s regional and area offices.

It is also the case that very few government agencies update postcodes on addresses they already hold as they are changed in PAF. For example the NHS code service was, relatively recently, distributing addresses for all doctors in the country, at least 10% of which did not hold a currently valid postcode. This is a serious situation for suppliers of facilities data such as lists of doctors or dentists, but is even more serious where postcodes are used to re-code administrative data, such as the receipt of benefits, into statistical data sets for the ONS Neighbourhood Statistics Service. Insufficient research has been carried out to estimate the extent to which such statistics are compromised by inaccurate, or out of date postcoding; however it appears likely that the level of error in such data is high and that the expense and awkwardness of the present distribution mechanism for PAF is to blame.

Roper also noted some adverse consequences for his own business of the Royal Mail’s maximalist approach to intellectual property:

If, for example, one has a dataset containing a great deal of address data, it makes sense to check it against PAF to make sure the addresses are PAF compliant. However, if you then want to license your dataset to third parties, Royal Mail now claims that the third party has to have a valid PAF licence before they can use the dataset. A specific case where this arose concerned PointX [... which] built a Points of Interest database, designed for use by mobile telephone operators and others offering location based services over the internet. It provides exact map references for over two million addressable and non-addressable objects. For the addressable objects (business premises, churches, pubs etc) it provides a postal address.

The primary purpose of the data is not to provide postal addresses, but it seemed proper to check the addresses we were collecting for PAF compliance and to that end PointX had a PAF Licence. A small proportion of the addresses had missing or incorrect elements that were rectified from PAF. Because the PointX data is licensed on to third parties, we asked Royal Mail if our PAF license covered this use of PAF. We were told by Royal Mail that it did not, and that we would have to pay a royalty on our onward sales where our licensee did not hold a full PAF license, even though the proportion of PAF data in the PointX dataset might be less than 5%.

Keith Dugmore of the Demographics User Group suggested that the PAF should be freely licensed for the public good, though without detailed argument:

Members of the Demographics User Group view PAF as a vital data source for a range of purposes. The growth in its use throughout both commercial and government organisations in the last three decades has resulted in its becoming a de facto standard. We believe that this should continue, and be formalised with the PAF being recognised as part of the national data infrastructure. PAF should be maintained for the public good, and its use encouraged – to the benefit of Royal Mail’s operations – by being made free at the point of use, using OPSI’s Click-Use License.

Postcomm’s April 2007 decision noted the concerns but, as far as I can tell, offered no analysis of them and essentially reaffirmed the status quo, that the Royal Mail could set its own licensing terms and prices and if anyone didn’t like it, they could complain to Postcomm (see sections 6.48 and 6.49).

Charles Arthur criticized the Postcomm decision in the Guardian:

The Free Our Data campaign has a simpler solution: make PAF free online and remove the copyright restrictions on reselling it. The revenue impact on Royal Mail would be minimal; yet as well as engendering a lot of goodwill inside the public sector, doing so could encourage new private sector businesses presently put off by the cost. It’s government-owned data; we should all get its benefit.

Skip a couple of years to 2009, and Harry Metcalfe was sufficiently annoyed with the Royal Mail that he created the website “Ernest Marples” (named after the former Postmaster General and fraudster). This site offered an API for converting postcode to latitude and longitude, which it allegedly accomplished by looking up the postcode in turn on one of the many websites that have paid a licence fee to Royal Mail and offer this service on their own website. (Google Maps is one well-known example of such a service, but I have no knowledge if Ernest Marples used Google Maps as one of its sources. Possibly the description of Ernest Marples’ algorithm was misleading and in fact it used a leaked copy of the PAF?)

Francis Davey pointed out that this novel attempt to (effectively) use the PAF without licensing it was likely to fail if legally challenged, because of the Royal Mail’s database right in the data.

Francis also noted that one of the points of Ernest Marples was to be sued:

As Harry [Metcalfe] explained at Open Tech 2009, part of the purpose of the site is political. There is a strong body of opinion that the Royal Mail should not have a monoploy on this extremely important database. If Ernest Marples is sued that will generate terrible publicity for the Royal Mail (as it should).

The Royal Mail duly obliged with legal threats against the site on 2009-10-02. This led to some publicity and Tom Watson’s Early Day Motion (though Julian Todd criticizes the lack of a more robust response, such as waiting to actually be sued).

Things you could easily do:

I don’t want to pretend that free licensing for non-profits should be the ultimate goal here. It’s a change that would enable sites like PlanningAlerts to get up and running again, but it doesn’t really take on the general problem of part-public, part-private, datasets like the PAF. Just as it sucks to be a community website without the resources to license the PAF in the first place, it sucks to be a business completely dependent on the PAF for your own operations, but without the clout to negotiate sensible licensing terms.

Some kind of radical reform is needed, but we have to keep in mind that someone is going to have to pay for the maintenance of the PAF (a somewhat unbelievable £16 million a year, according to the Postcomm report). It wouldn’t be a victory for open source if competitors to the Royal Mail decided to develop their own post-coding systems. (Compare with the situation for mapping: many competitors to the Ordnance Survey prefer re-surveying the country to paying Ordnance Survey licensing fees.) I think it’s obviously a public good, so we’ll get the best value for money if taxpayers pay for it directly and license it freely to everyone, but good luck making that kind of case in the current economic climate.


Update . The government responded to an earlier petition on the same subject, saying, basically, “go complain to Postcomm”:

PAF users (or any stakeholder) can complain to Postcomm if it [sic] feels that Royal Mail is not complying with the terms of section 116 of the PSA 2000 or Condition 22 of its licence. Postcomm would then consider the merits of any such complaints in the light of its statutory duties.

The cited section of the Postal Services Act 2000 says:

116. The Postcode Address File. —

  1. The owner for the time being of the Postcode Address File shall—
    1. maintain the File, and
    2. make the File available to any person who wishes to use it on such terms as are reasonable.
  2. Compliance with subsection (1) shall be enforceable by civil proceedings brought by the Commission for an injunction or for interdict or for any other appropriate relief or remedy.

So, who’s going to make the argument to Postcomm that the current terms are not “reasonable”?

Update . David Howarth replied.

Dear Mr Rees,

Thank you for your e-mail of 13 October about the availability of postcode data.

I have signed Early Day Motion 2000 as you asked.

I thought that you might be interested to see a copy of the reply I received last year after I raised the issue with the Minister responsible.

Yours sincerely,

David Howarth
Member of Parliament for Cambridge

The reply he refers to is dated 2008-07-30, from Pat McFadden, then Minister of State for Employment Relations and Postal Affairs. McFadden basically says, “go talk to Postcomm”:

Your constituent’s concerns are fully appreciated. I should explain that the Postal Services Act 2000 acknowledges that Royal Mail is the owner of the Postcode Address File and proves for that ownership to continue. The management and maintenance of the PAF is the responsibility [of] the company. The PSA 2000, via section 116 [see above], deals specifically with PAF and requires Royal Mail as its owner to maintain and make the PAF available to any person who wishes to use it on such terms as are reasonable. Section 116 is broadly replicated in Condition 22 of Royal Mail’s licence. Postcomm, the independent postal regulator, is responsible for licensing postal operators, including Royal Mail, and if anyone feels that a condition of licence is not being met, they should be advised to contact the Regular for a view. This includes any concerns over the reasonableness of access to the PAF.

Update . The government proposes to provide free access to a subset of the postcode data. According to the “Frontline first” report:

Commitments: government’s public data principles

‘Public data’ are ‘government-held non-personal data that are collected or generated in the course of public service delivery’.

Our public data principles state that:

Within these guiding principles we will take the following actions to open up data and promote transparency:

Actions: opening up data and promoting transparency

We will release valuable public datasets and make them free for reuse. This will include:

It’s not clear from the report exactly which postcode datasets will be made freely available, but this BBC report suggests that it will be the PostZon file, which links postcodes to geographical locations (but not addresses). This is the file that appeared on WikiLeaks in September.

The PostZon file would be adequate for many location-aware services. Nearly everyone knows their own postcode; most addresses are given out with postcodes; and it’s possible for people to look up a small number of addresses via the Royal Mail’s postcode finder (though you can imagine cases in which this would be sufficiently onerous as to make the service useless).

I think PlanningAlerts, The Straight Choice, and Jobcentre ProPlus could manage with the PostZon file. CycleStreets would benefit from the PostZon file to a small extent, but for planning routes to individual addresses you really need the PAF: many postcodes are too big geographically, or cover too many addresses, to be suitable as endpoints for a route.

Update . Another petition gets an identical “go complain to PostComm” response. Someone should bite the bullet and make a complaint. CycleStreets has, I think, the best case.