Open Data

We like data, and we like to share ...

Our research work commonly involves the analysis, interpretation and presentation of large quantities of non-sensitive data generated by the government, energy companies and others.

This data is useful, and in the spirit of openness and transparency – and with support from the Esmée Fairbairn Foundation – we're pleased to make some of it available here. This is our Open Data Collaboration Initiative, and we hope it will lead to:

  • better evidence to underpin policy to tackle climate change and target fuel poverty;
  • stronger innovation and enterprise in developing and sustainable energy solutions;
  • a new basis for collaboration; and
  • empowerment of communities and local organisations to use data to develop appropriate responses to local circumstances.

Click on the links to take you to the different datasets:

1) Display Energy Certificate data
2) Lower Super Output Area data
3) Fuel poverty data (including 'hard-to-treat' data)
4) Energy Company Obligation data
5) Energy consumption data (domestic)
6) Heating and housing census data (at smallest output area level)
7) GB household emissions dataset
8) GB postcodes off the mains gas grid
9) Energy Performance Certificate data (at parliamentary constituency level)

Please email suggestions of other sources of data to add to this collection to opendata@cse.org.uk.


1) Display Energy Certificate data

CSE has obtained the full Display Energy Certificate Register dataset for all UK public buildings from the Department of Communities and Local Government (CLG), under the Environmental Information Regulations. Read about how we did it here.

It contains data on energy use, floorspace, emissions and efficiency ratings for 40,000 buildings of over 1,000m2 in England and Wales that are visited by the public. The full dataset contains data from 2008 (the first year of the register), 2009, and 2010.

We have also made a ‘latest data’ spreadsheet containing only the most recent energy performance data for any one address. However, around 146 addresses have more than one record for a given year, and without looking at the actual DECs it is impossible to say which of these is the most recent. For this reason the ‘latest data’ spreadsheet includes a supplementary table of  'duplicates' listing all the buildings for which this is the case.

Download the combined 2008, 2009 and 2010 data here (12MB)
Download just the 2010 data here (7MB)
Both files are in the newer Excel format (.xlsx) which keeps file sizes down; if you would like a .xls version, please email vicki.white@cse.org.uk

DECs for individual buildings can be downloaded from www.ndepcregister.com using the Certificate Reference Number (variable RRN in the datasets). The government's methodology for collecting the data can be found here.

CLG have also put in place a procedure to request access to Energy Performance Certificate data. Click here for details. The department has also indicated that it intends to publish approximately 723,000 records relating to the energy performance of non-domestic buildings in response to a request for information.

Data health warning | Errors in the data collection or data entry can result in a building being given the wrong energy rating value. For example, we have found cases of buildings for which there is no kWh or CO2 data but which nonetheless have an A rating. This is beyond our control, and CSE cannot accept responsibility for errors in this data which was supplied by CLG.

Licensing update, May 2014 | CSE has received a few queries from potential users about the licensing of this data, and including requests for written guarentees that it can be re-used by third parties (which we can't give). This is our response:

In reality, the licensing of this data is a slightly grey area. It is public sector information which was obtained from CLG under the Environmental Information Regulations. Paragraph 97 of this guide from the Information Commissioner's Office (ICO) says that "Public authorities should make the environmental dataset available in a re-usable form, so far as reasonably practicable. Although there is no duty to license re-use under the EIR, we would also encourage public authorities to make the dataset available under a licence that permits re-use."

We didn't receive any information about licensing when CLG gave us this dataset. Our view is that CLG are highly unlikely to object to its re-use as the ICO recommends that this should be allowed with this type of data - which is now several years old and has been available on our website for several years, and used by others (including the Guardian in 2011), without objection. 

Some time ago we submitted a FOI request for updated DEC data but this was rejected on the basis that the data is now 'publicly' available albeit for a large but apparently not unreasonable admin fee. We are looking into whether it would be permissible for us to buy the data and then make it available to others for free.

In addition, Honest Buildings has submitted a request to data.gov.uk for the latest DEC data to be published on that site (see http://data.gov.uk/data-requests/display-energy-certificates), but the request was made in November 2012 and it is still under review.

[Back to top]


2)  Lower Super Output Area data

The Community Energy Saving Programme (CESP) was designed to target the lowest income households living in homes that need significant investment in energy saving measures and low carbon technologies.

CSE mined the CESP data stores to develop and publish a database of useful information to assist with this targeting. The dataset is available to download here (NB: this file is zipped).

[Back to top]


3)  Fuel poverty data (including 'hard-to-treat' data)

CSE has recently published a free tool called HEAT that enables you to model opportunities in the English Housing Stock and also predict fuel poverty. Read more about this here.

As part of our commitment to open-data we published projections for fuel poverty in 2012. The file also contains breakdowns for 2009, 2010, 2011 and 2012. It can be downloaded here.

Our latest 'Nowcast' of fuel poverty, 2013 shows the breakdown of predicted fuel poverty in England in 2013 by EPC band, according to both the old definition of fuel poverty and the new Hills definition. Click here to download the tables (Excel).

CSE's rural fuel poverty website (www.ruralfuelpoverty.org.uk) is no longer being updated but still contains useful information and data. It focuses on rural fuel poverty particularly in relation to 'hard-to-treat' homes and includes downloadable maps and data. The full dataset used to produce the website can be downloaded here (NB this file is zipped). A useful key and glossery can be downloaded here.

[Back to top]


4) Energy Company Obligation data

We're often asked for data which can help with targeting measures to households and stats about progress to date. These are available but can be hard to find, so below we've given you some links to the published datasets and an official tool.

The list of low income and rural areas eligible for CSCO can be found on the government website here (NB this is a .xlsx file). 

Ofgem provides an online tool which identifies CSCO-eligible areas at postcode, LSOA, and local authority level. It also shows Adjoining Areas, which are not given in the above spreadsheet. The tool can be found here.

The government publishes monthly and quarterly statistics on ECO and the Green Deal, which can be found here.

There is also some data on off-gas areas available. For this, see Section 8 below.

[Back to top]


5) Energy consumption data (domestic)

CSE has developed an analytical model known as 'Dimpsa' (Distributional Impacts Model for Policy Scenario Analysis) which is now used under license by the Department of Energy and Climate Change for the Government’s own assessments of the distributional impacts of policies.

You can read more about Dimpsa here.

Underlying Dimpsa is a comprehensive dataset developed of UK household energy consumption. This dataset is derived from the socio-demographically representative sample of UK households surveyed in the Office of National Statistics Living Costs and Food Survey. Data from six of these surveys (2004/5 to 2009) has been combined, generating a sample size of over 36,000 cases.

We've published some key statistics on average household energy consumption derived from the Dimpsa dataset. The information provided includes estimates of average annual household electricity and gas consumption for all households in Great Britain, and broken down by income bands and consumption bands (representing ‘quintiles’ and ‘deciles’ of the dataset).

Ian Preston, CSE's Senior Analyst, said: "You could, for example, gather information about different energy tariffs, and then use these profiles of consumption by income to estimate how much different households pay for their energy."

The number of households in each consumption and income quintile and decile is also provided. This information is fundamental in understanding the distribution of energy consumption patterns across households in Great Britain.

The spreadsheet of summary data is available to download here.

A guidance document providing key information about the data, including the method behind developing the dataset and examples of how the data can be used is available here. We highly recommend you read this document before using the data provided in the spreadsheet.

If you use this data please reference it as ‘CSE Dimpsa data’. We are very interested to find out what people are doing with this data, and we also welcome questions about it. Please send any comments or queries to opendata@cse.org.uk.

For more energy consumption statistics please see our work on identifying consumer archetypes for Ofgem.

[Back to top]


6) Heating and housing census data (at smallest output area level)

The Office for National Statistics publishes summary census data over a range of geographical levels after a national census has been performed. As the 2011 census data has become available, CSE has downloaded and compiled selected datasets at the lowest available geographical area. Output levels typically contain 70 to 150 households and so cover a relatively small geographical area, dependent upon rurality.

CSE has downloaded and joined key data sets from the Office for National Statistics census 2011 website. There are many potentially interesting topics that could be investigated from the data available, and CSE has chosen data that could be relevant to characterising housing and their energy usage.

Variables include:

  • The main central heating type of households
  • Tenure
  • Number of bedrooms
  • Household type (detached, flat etc)
  • Household composition
  • Rurality

We believe that this data could be useful for targeting schemes such as the ECO, as areas likely to be off gas and containing more vulnerable customers, or areas containing ‘large’ households could be identified. Here’s an example of how we used the data to create a map of the percentage of gas centrally-heated households across the South West:

Data is available by Government Office Region (click below for .xls files):

East Midlands
East of England
London
North East
North West
South East
South West
Wales
West Midlands
Yorkshire and Humber

Or for the whole of England & Wales (NB Excel 2007 as 181,000 rows).

No processing has been performed on this data, and so for any questions and information about the data users should refer to the Office for National Statistics website. Users should be particularly aware of the statistical disclosure control for 2011 census information.

[Back to top]


7) GB household emissions dataset

This household-level dataset covers the annual CO2 emissions from domestic energy consumption and personal travel by car, public transport and non-domestic flights for all households in Great Britain and is derived from threenationally representative surveys:

  • Expenditure and Food Survey (EFS, 2004-07): used to derive estimates of household emissions from the consumption of energy in the home (electricity, gas and all non-metered fuels);
  • National Travel Survey (NTS, 2002-2006): used to derive estimates of household emissions from all personal (defined as leisure and commuting) travel by private road vehicle and public transport;
  • Civil Aviation Authority Air Passenger Survey (CAA APS, 1999 to 2008): used to derive estimates of household emissions from international air travel.

The dataset was initially developed as part of a Joseph Rowntree Foundation-funded research project, ‘Distribution of Carbon Emissions in the UK: Implications for Domestic Energy Policy’, where is was used to explore the distribution of emissions across households in Great Britain by a range of socio-demographic variables.

As far as we are aware, this is the first integrated analysis of emissions from both the consumption of energy in the home and personal travel based entirely and directly on nationally representative survey data, and the results are summarised in section 4 of the main project report and in more detail in a supplementary project paper.

However, a great deal more analysis of this dataset could still be done - e.g. the distribution of emissions from the different sources across different socio-demographic groups - and we hope that providing the data others could take up the challenge.

The dataset is designed to be analysed using SPSS (though it's also available in Excel) and the instructions provided reflect this. We also assume users have a good level of understanding of the Expenditure and Food Survey, which provides the core structure for the emissions distribution dataset. For more information about the EFS please refer to ONS documentation.

 To request the data, please email opendata@cse.org.uk stating your name, your organisation (if applicable), the format you require (SPSS or Excel) and the reason you are interested in this data (this helps us understand its utility and how it could be improved).

We strongly recommend that users read the accompanying ‘How-to’ guide.

[Back to top]


8) GB postcodes off the mains gas grid

CSE has published a complete list of Great Britain postcodes that are not connected to the mains gas grid, up-to-date for 2013. We've also added the census area and the co-ordinates of the centre of each postcode on the list.

The households in these areas do not have access to mains gas for heating or cooking, and are likely to be reliant upon more expensive forms of heating such as oil or bottled gas.

Download the list here (Excel 2007, 22MB)

NB There are 412,696 postcodes in this dataset. Please be aware that conversion to other file formats such as .xls may result in loss of data, because not all spreadsheets support a sufficient number of rows. After conversion, please check the number of records before you use the data.

This data was produced using Xoserve's list of off-gas postcodes, which was released late last year with little fanfare. Before this, off-gas postcode data had not been available for around 10 years. Xoserve's list is available to download here.

You may find this article from DECC's Energy Trends 2013 report useful. It contains more information about the Xoserve data, plus estimates of off-gas households at LSOA level and an analysis of characteristics of properties without gas meters.

[Back to top]


9) Energy Performance Certificate data (at parliamentary constituency level)

This work uses data contained in the National Energy Efficiency Data-framework (NEED) administered by the Department for Energy and Climate Change (DECC). The data shows the numbers of dwellings in each EPC band by Lower Super Output Area (LSOA) – an area that represents around 650 houses on average – and is now available on the government website.

We have used this data to calculate the numbers of dwellings in other types of areas including at Middle Layer Super Output Area (MLSOA) and parliamentary constituency areas in England and Wales. As a result, data showing the number and proportions of dwellings in each energy efficiency (EPC) band by LSOA, MLSOA and parliamentary constituency can be downloaded here.

In addition, we have also produced a series of maps of the parliamentary constituencies. These show where the highest proportions of the most efficient and least efficient dwellings are located.

Percent of homes EPC band C or above
Percent of homes EPC band F or G

This formed part of research for Citizens Advice that looked at level of energy efficiency of housing in different areas of the country.

For further information contact:

Dr Toby Bridgeman | 0117 934 1435