How can we make open data work for our users?

For a long time, there’s been an established view that publishing more open data should encourage the proliferation of new applications and services across the public and private sector based on the availability of common datasets across local authority boundaries. 

Historically in Hackney, we’ve only ever dipped our toes in the world of open data and to be honest, we don’t know whether there’s enough evidence to prove that simply publishing as much as possible will realise these benefits – we just don’t know enough about the user need to justify a big bang open data offering.

That being said, we’ve been inspired by the ODI, Nesta and others who are keen to encourage greater transparency and common standards in data publication and we have reviewed some of the good practice of other local authorities who’ve experimented with open data portals in recent years. But our HackIT manifesto tells us to think big and act small – we don’t want to dive into the deep end of an expensive open data portal for publishing every non-personal data set we can find. Instead, we want to understand the kind of data users want to see, and work out the most simple and cost effective way of providing that data in a useful way.

Currently, we don’t proactively publish the data that is most commonly requested via Freedom of Information (FOI) requests, which results in a significant burden of resource spent re-actively responding to FOI requests. In his 2018 manifesto, the Mayor Glanville committed us to more transparency and this has helped shape a set of basic principles to help us experiment with user focused open data publication. We’re keen to open ourselves up to some challenge on this and share what we learn as we experiment:

  • Focus on releasing data commonly requested via FOI – We believe that concentrating our open data offering on the most frequently requested topics will release significant amounts of officer time and provide a better service for residents. Therefore, we will focus on these opportunity areas first.
  • Use common standards wherever possible – to be valuable for developers and organisations, we need to publish our data in a format that is easy to join across local authority areas. Where a standard doesn’t exist, we will try to match other Councils who are already publishing this data to maximise reusability. We will openly publicise our data structures to encourage reuse by other local authorities.
  • Automated with a focus on currency of data – to be of maximum value in reducing the FOI burden we face, the data included in our open data products should be as current as possible. To ensure we aren’t just moving the resource problem from FOI administration to open data publication, data updates should be automated and require a minimum amount of officer time to publish.
  • Adopt a flexible approach to the technology we use to publish our data – we don’t believe in being constrained by an online data portal for publishing open data. We will aim to use the best modern technology and data analytics techniques depending on the data involved to provide a data dashboard, visualisation and full download of the dataset. We are motivated by providing a great user experience that’s easily discoverable first and foremost.
  • Aim to meet GDS design standards – all of our products will be designed in line with modern, accessible design standards and we will always test our products with our users.
  • Understand our impact – we will always begin by understanding the baseline volume of FOI requests for the data set in question and monitor over time, the impact of publishing the dataset. We expect to see more exemptions (where an FOI is refused under the grounds that the data is already published) and over time, fewer FOI requests overall. If the data set isn’t reducing FOI demand, we will look for other areas where we can add more value by publishing new data.

Our first experiment is with Penalty Charge Notice (PCN) data (that’s parking tickets to you and me…) – it’s one of the most commonly requested datasets and we think publishing this openly will help residents answer their questions in a more timely way and reduce the time we spend responding to FOIs on the topic. We’re experimenting with providing a simple  app on the parking pages on our website, which will allow users to extract the type of information from the data that is often asked for in FOI requests. Our great User Research team are helping to keep us focused on delivering something simple and intuitive for residents. We’ll also be trialing a light touch open data portal which will allow us to curate our open data sets in one place on the website. We’ll share more as we develop our MVP over the coming weeks. 

Hackney’s drive for a digital ‘Submit My Planning Application’ service

Doesn’t time fly!

It seems like only yesterday (Nov ‘17) when we were holding a Planning design sprint with @euanmills and his Connected Places Catapult team. Since then we’ve been fortunate enough to win an MHCLG funding bid to take forward our ambitious idea for the ground breaking Submit My Planning Application digital service (SMPA).

For the past year Hackney ICT have been working tirelessly with Snook and their technology partners Hactar to deliver a really high quality prototype for our SMPA digital service. Thanks to some great help from the Hackney Planning team, local businesses and residents the service has been fully designed around user needs. working collaboratively with the Open Systems Lab and the Southwark Planning team has saved us reinventing the wheel by utilising their digital Local Planning Policy engine approach.

Going back to the original brief, our goal was to either build a new service or stimulate the market into providing excellent digital Planning services.  We still want to meet the real needs of the Planning Authorities, businesses and residents that are suffering from a stagnant software market -our digital service will be the first step in that direction. However, I really didn’t see Northgate’s acquisition of Snook coming, a really interesting partnership and something to keep an eye on!

So after several sprints, retrospectives, all the iterative user research, many weeknotes and of course the obligatory show & tells we now have a working prototype and plan to deliver a Live minimal viable product later this summer.

So what are we doing now?

Basically, lots of building, testing and deploying! We have six sprints scheduled to take us to a live working product. See our previous show & tells, even better see all the whole project, here’s our project site alternative you can follow us on Pipeline.

Our aim is to have a Live working digital service by the end of the summer ‘A service so good people prefer to us it’. This is a bold statement, but we know demand failure is hitting a high of over 50%, therefore plenty of room for improvement. Imagine Amazon getting 50% of deliveries wrong! Also in this case the people do actually have an alternative i.e. the Planning Portal and iApply hence we intend to deliver an excellent alternative.

Normally when developing a new system from scratch, a lot of the work done is behind the scenes. Nowadays it’s not so much behind the scenes anymore, Hackney being part of the digital declaration, projects posted on Pipeline and working in the open you can find the code is stored on GitHub. You’ll see we’ve also been utilising existing capabilities such as gov.notify and developing excellent simple to follow documentation such as our API walkthrough. Currently we have 90% of the MVP under test and using real life applications to make sure the software works as expected. 

Our LLPG and GIS systems are now integrated, not to forget the most important integration i.e. with Tascomi our soon to be back office Planning Management system. We’re also keeping a close eye on the GLA’s London Development Database. Our service will need to capture all the right data up front in a standard format that can be seamlessly used right through the Planning process and not locked away in those closed and difficult to use PDFs.

What’s coming up and worth looking out for

The next couple of weeks are all about finishing the build and testing, testing and more testing!

As well as building the MVP we’re focussing on the not so exciting logistics of making the service Live within our new DevOps ways of working and if timings work out well, we’ll be integrating with Tascomi!

Key dates to look out for:

Come and join us at our offices or remotely, we’ll be around afterwards for deeper discussions on how we can shape the service and how others can adopt it 

  • 17th Sep Connected Places Catapult (Plantech Week)

We’ll be presenting our new digital Service ‘Submit My Planning Application’, Book your tickets through Eventbright

  • 19th Sep GLA – Event (Plantech Week)

14:00 GLA Plannig Event , Plantech for Local Government

(Venue City Hall)

  • 19th Sep SMPA Product Launch! (Plantech Week)

18:30 Connected Places Catapult Third Thursday

(Venue Urban Innovation Centre)

7 reasons I love the Summer in Hackney map

We launched the Summer in Hackney map this week. Here’s an ode to why it might already be my summer highlight.

1. It came from nowhere

I first heard of the Summer in Hackney project in its first weeknote. I was surprised and delighted: it’s not often we do something that I haven’t heard of beforehand. So it was liberating to read a weeknote and then see a Show & Tell. That’s why we ask the team to ‘fail in a fortnight’. 

2. There was more vision than definition

The project began with a vision for what we could achieve for users, rather than a clear definition of how we’d do this. We need to start more projects with a goal in mind rather than a thing we want to do. 

3. It was a team

The GIS team worked with user research, our designer and front-end developer to prototype and build a product. The team worked together, testing and re-using components (including our front-end patterns as well as things like the research pop-up banner). 

4. We did our user research

It would be easy to have done the Summer in Hackney map because we thought it was a good idea. We didn’t. The team did user research, together, to find out what users need.  

5. It blends tactical and strategic

The team described Summer in Hackney as a side-hustle, which sounds cool but is a bit unfair. It helped us learn about new ways of working and experiment with a different approach to delivering GIS data to users. 

6. The team delivered

We shouldn’t take for granted that a piece of work that began in July, delivered whilst it was still summer – and before the peak of summer. But we’ve also delivered a tangible improvement to residents. 

7. We’ve created components for re-use

The team did the work in a way that means we’ve learn about a pattern for how to provide maps and developed re-usable components for future maps – reinforcing how this project is a nice tactical intervention providing a strategic opportunity for change. 

From this:

Summer in Hackney - old

To this:

New summer in Hackney map

Building a frontend pattern library

At Hackney we’re currently in the process of rolling out a new design for our website. This has involved designing a UI Library of components that can be shared across other projects. Now we want to take those components and build them into a frontend pattern library so that code can also be shared across projects. We want this to take the form of an npm package that could be included in all projects so that styles, scripts and potentially templates could all just be imported individually from that package when required.

The starting point for our UI Library has been the GOV.UK Design System and many of our new Hackney components build on existing GOV.UK components with Hackney branding applied, often through changes to colour, font and/or spacing. Because of this I am keen to utilise the GOV.UK Frontend components in our frontend pattern library to avoid unnecessary work, duplication of code and to make use of all of the research and development that has already gone into GOV.UK Frontend. But if we are going to do that it also makes sense to use the GOV.UK Frontend wider setup (the Node.js app, Nunjucks templates, etc) as a basis for our Hackney Frontend; again this minimises duplication of code and effort and ensures maximum compatibility.

So the plan is to create an LBH Frontend repository that is very similar to the GOV.UK Frontend repository but also consumes the GOV.UK Frontend package so that its components can be used and built upon. The LBH components will all be prefixed “lbh” and these will be what are used by developers working on Hackney projects. Some components will be completely original and unique to Hackney, others will build on the GOV.UK components but with changes to markup (in which case we would rewrite the markup but utilise some GOV.UK classes) or styling. In cases where the Hackney component is very similar to the GOV.UK component, a component template could be as simple as including the GOV.UK component using Nunjucks and including an override class. There is a question about whether or not this last scenario provides value since that is something we could do without a LBH Frontend, but I think that establishing a set of LBH Frontend standard components that don’t require any modification feels like it is worth doing.

As Hackney is also doing a number of React projects it will also make sense to make a React version of the pattern library, similar to the GOV.UK React repository.

As both of these projects will be utilising GOV.UK Frontend we will need to have a plan for updating GOV.UK Frontend. Updates could include important accessibility changes, desirable new components, or crucial fixes so will be important to include. I think largely the plan will involve making sure that we are aware of GOV.UK Frontend changes and understand what has changed so that we can anticipate how this will impact LBH Frontend and build it into resourcing plans. I also think a Visual Regression tool such as BackstopJS would be useful in this instance so that we can automatically see any unexpected visual changes caused by updating GOV.UK Frontend.

Do you have any thoughts about this plan? I’d love to hear what you think. 

Love Summer map now live!

Breaking News! Now live at

Explore summer in Hackney with our map of playgrounds, pools, picnic areas, drinking fountains, walking routes and m
Explore summer in Hackney with our map of playgrounds, pools, picnic areas, drinking fountains, walking routes and more.

We’ve just finished a four week side-gig to get the best out of the fantastic resource that is Map.Hackney. While it’s a valuable tool for urban planning, it also holds hidden gems such as locations of drinking fountains, wildflower meadows, paddling pools and quietways. Which we want to shout about.

With an assembled team of a mapping expert, user researcher, developer, designer and deliverer, we kicked off our sandals and took the idea to the people. Our first stop was London Fields, where we interrupted unsuspecting bystanders to ask what they like to do in summer in the city.

Out of this research came four personas that we hoped would resonate with people. The one who likes to hang out with friends in the park and play ping-pong (‘Chilling’); the one who wants to keep the kids occupied for an hour (‘Playing’); the one who wants headspace while meandering through the streets (‘Wandering’); and the one who wants to create their own summer (‘Everything’).

With a wireframe best described as no-fi, the team was able to spin out a prototype faster than ice melts in a mojito. More research yielded great feedback: are the blue lines rivers? [cycle paths]; is that a bus route? [Hackney boundary]; can you see cafes in parks [heck, yes!]. All this was iterated back into the prototype. If you click on the slides, you can try it out for yourself via the QR code.

We all learnt something on this project. For some, it was the first experience of Agile working; for others, GIS was a whole new world; still more were introduced to GitHub. But the learning curve never felt too steep.

We’ve still got some end-to-end testing and colour contrasts to work out but we’re ready for our Monday launch. Let the summer begin!