GIS, supporting the delivery of more digital projects in Hackney

Hackney’s corporate GIS team sits within our Data and Insight team within ICT. Our mission is to support projects, colleagues and residents to make the most of Hackney’s spatial information. However, we have a blocker: most of the work we do comes to us via colleagues who already use GIS, or know someone who uses GIS, or at least have an idea about how spatial information can help them in their jobs. This represents a fair amount of people, and their number has increased a lot since we launched our new Intranet GIS (Statmap Earthlight) 12 months ago. However we think we could do better and we want to find new ways of supporting our digital delivery teams with spatial analysis, particularly those who don’t yet know the power of it as a tool.

So last week we gathered our colleagues from across our Digital and Delivery teams, and explained to them how we are delivering our mission across Hackney service areas. A possibility for our colleagues to mentally match our offer with the various situations they encounter when they are out. A chance for us to discover use cases and ‘GIS gaps’ we wouldn’t have suspected otherwise.

We detailed the 3 work streams we are following to realise our mission, as follows (every local authority’s GIS team will probably recognise itself).

Infrastructure and data governance: we administer and develop the corporate spatial data warehouse, which has been providing a single view of Hackney’s spatial information since 2008 and integrates with other council systems through web services and APIs. Most importantly, we’re supporting our community of GIS users, and working with our data owners to introduce new ways of managing data quality. The first step of this journey was the creation of our metadata model and data catalogue.

Data Analysis and Insight: we are using spatial analysis to answer questions and support decision making. We use different supports to develop and share results, like Qlik Business Intelligence dashboards and Jupyter notebooks.

Data sharing and web mapping: we are providing technical support for service areas to share their data when they have to. We developed and maintain Hackney’s open mapping portal. We are, however, trying to move away from the monolithic portal approach and are practising our ‘Agile delivery’ of web maps focusing on a specific sharing need, using Leaflet.

Our presentation had two immediate benefits: firstly, our user research team requested access to the Intranet GIS with the view that it could help in discovery work that they do with services to understand user needs. Secondly, a colleague from Delivery asked how non GIS-users can realise the richness of our spatial data repository if the only way to browse this repository is… to be a GIS user. Do we need a channel for non-GIS users to discover GIS? What should it look like? This will guide our thinking for the preparation of the ‘Annual GIS day’ in November. Any suggestions welcome.

You can see a subset of the examples we showcased in this slide deck. We are keen to hear about how our approach compares to how other GIS teams are operating and think of ways we can learn from others to improve our offer in Hackney.

Working with Pipeline to share our digital efforts

Pipeline is a digital service that is designed to help us all work together to deliver digital services, so good that people prefer to use them:  https://pipeline.localgov.digital/.

Adopting Agile at Hackney

Over the last two years we have been working hard to change our approach here at Hackney, adopting Agile delivery and user-centred service design. This has meant lots of big changes in our team, how we work and the way that we manage our portfolio of projects.

With the move to Agile delivery comes a dramatic acceleration of pace and our shift from large projects to quicker, sprint based delivery has meant that our portfolio has become much more complex than before. As Lead Delivery Manager responsible for coordinating that I’ve been working with colleagues to look hard at how we can make that as easy as possible.

Hackney has devised its own Agile lifecycle, based on the Agile delivery principles set out in the GOV.UK service manual, and although we know that we need effective governance in place, it’s essential that this doesn’t slow down delivery. One of the key pieces of governance we need for delivering projects is an easy way to see what projects we are delivering and keep track of status and updates.

We are working as part of the wider local digital community

Hackney have committed our support to the Local Digital Declaration and want to help support collaboration in digital local public services. You can read more about why we think that’s important on our blog: https://blogs.hackney.gov.uk/hackit/in-support-of-the-local-digital-declaration. As well as managing our own portfolio of delivery, we also wanted to make sure that we could be open about the work we’re doing, share learning that others can use and identify opportunities to work collaboratively with other councils who are looking at similar challenges.

We started with user needs

We identified the following user needs:

  • As a delivery manager I need to report once on the status of my project so that it’s easy to update stakeholders
  • As a portfolio manager I need to see an overview of all my projects so that I can communicate our projects
  • As a manager I want to see other authorities’ pipelines so that I can identify opportunities for collaboration

We looked to see what tools might meet those needs

We did an initial pre-discovery exercise to see what tools might best meet our needs. Agile isn’t new and there are a wealth of tools that are being used by Agile teams across the globe. But we didn’t find anything that felt like it quite fitted the bill.

We felt that the best match of the tools we looked at was Pipeline, which had been built as an alpha in 2014 by LocalGov Digital. This was a good fit with our aim of working collaboratively, but would need some tweaking to give us the portfolio tracking capabilities that we needed.

We decided that rather than buying a thing or making a thing ourselves, it would be best to contribute to a community tool and see how we might ‘stand on the shoulders of giants’. We got in touch with the LocalGov Digital team and were delighted that they were up for that.

So, what did we do?

Acknowledging that we needed something quickly and with our internal developers all at capacity we devised a brief and posted it onto the Digital Marketplace. We got strong bids and chose Rainmaker Solutions to help us to build on and iterate the great work that had already been started in 2014.

We held a workshop with key users to tease out what else was required to make the Pipeline service a tool that would add real value to Hackney and other public organisations. The previous iteration of Pipeline had a promising start but its use had declined, so we needed to create a tool that would add enough value that people feel compelled to use it. A quick discovery exercise delivered lots of feedback which was captured as user stories and prioritised in a backlog for an intense 3 sprint project.

As the Product Owner for the project I ran a session at the recent LocalGovCamp event in Birmingham to demonstrate the updated service and get feedback from peers. I presented these slides and ran two quick workshops. The first was to gather ideas of the functionality that people would like to see in Pipeline and the second was to discuss why people hadn’t used the first iteration of Pipeline and consider why they might not use the updated version. The functionality discussion threw up a few ‘Robocop’ items but the main thing we found was that Pipeline would meet the needs of the users in the workshop (you can see the feedback we captured here). The barriers gave us more to consider. We asked ‘why won’t people use Pipeline’ and found a range of concerns in response (the feedback from the discussion is here):

  • Fear of working in the open
  • Scared of failing in the open
  • Duplication of effort
  • Didn’t see what value it would add to their organisation
  • A gimmick that won’t be used
  • One size fits all won’t necessarily work for their organisation

Fear of working and failing in the open’ was a really interesting barrier. We were incredibly lucky to have Mike Bracken (one of the founders of GDS) visit Hackney this week. He spoke passionately about working in the open and failing fast and learning. I believe his comment was ‘it’s easier to deal with failing if you’ve been open about what you’re doing’. Fear of failure is a cultural issue in many organisations, including local government – my view is that working in the open enables us to focus on being clear about what we’re trying to deliver.  

Duplication of effort’ is a tough one and is something we’re mindful of. Currently our project updates are posted in a public Google+ community, HackIT Delivers. Our Delivery Managers will continue to post their updates in the G+ community and will link status updates in Pipeline to the G+ posts. It’s not ideal, but is minimum duplication to share our work with a much wider audience. Further iterations of Pipeline will hopefully include open APIs that we can connect other project status update tools to.

Adding value’ and ‘it’s a gimmick that won’t be used’ aren’t technical blockers that can be fixed. We’ll show through doing. It’s a case of hitting critical mass: when the product is used widely there will be value derived from the tool and a compelling reason to use it. We can make that happen if we all use Pipeline.

One size fits all’; this was a challenge for this next iteration of Pipeline. At Hackney we follow the GDS approach for many things, but our project life cycle (HAL) has project phases that use different naming, which fitted better with how our organisation manages projects. To stimulate collaboration it was decided that Pipeline would use the standards that have been developed by GDS as that’s a model that we all recognise. So, it is a one size fits all approach but that one size is award winning and widely recognised as a beacon of best practice. We think we can adapt to it without much difficulty.

What Happens Next?

We are starting to use Pipeline and we’re keen that you benefit from it, too.

We’re meeting LocalGovDigital colleagues and the MHCLG Local Digital Collaboration Unit shortly to discuss how Pipeline might be used to help publish all of the collaborative projects they’re looking to fund and deliver – including applications for the Local Digital Fund that has been launched to support the Local Digital Declaration.

This latest iteration of Pipeline was delivered in 3 sprints. There were lots of user stories and additional functionality delivered but there is still a big backlog of additional user stories that we identified through the discovery work (you can see these on this Trello board). We will be looking at how we might fund an additional development sprint in the next couple of months, maybe using Pipeline we might find another organisation that wants to work with us to deliver the next release!

Governance so good, people prefer to use it

Governance as a service

At HackIT we’ve been thinking about how we run ourselves, and our work. I’ve been looking at what we need to do next to iterate our approach to governance. Our HackIT manifesto already sets out our key principles — and there’s been lots of work done to remove some tortuous processes that weren’t working for us.

We’ve already opened up our work, use the local gov digital standards as a benchmark, have adopted the GDS tech code of practice to guide us, introduced pair programming and test driven development, and we’re using agile principles and rhythms to deliver value early, and increase pace of delivery.

But the team is changing and developing — new people are joining us from all sorts of different organisations (and we have 21 new apprentices starting). We need to be able to scale, develop and embed our approach effectively — recognising that we’ll learn along the way and we’ll want to adapt it as we go.

Why is governance important to us?

Governance helps us maximise the flow of valuable work. That’s basically its purpose — with three main functions:

  • Coordinate what we’re doing and stop doing stuff, so we can go faster
  • Focus our people and money, so we can deliver what matters
  • Answer the question “How’s it going?”

My hypothesis is that we don’t need more governance. But because we are scaling a new approach to working using agile we do need to be really clear about what we’re doing and why, communicate it well, and keep checking in with ourselves to make sure it’s effective.

Principles

We’ve got some governance principles to help us get this right:

  • Work in the open by default — because that enables us to reduce formal governance
  • Most decisions should be made at team level — that’s where the best information is
  • When a decision impacts more than one team — teams are responsible for discussing and agreeing what to do between them
  • Where a decision impacts us all — we need to discuss that more formally at a senior level
  • Clear protocols and guidance help us so we avoid overwriting each other’s decisions.

We’re still working on some of our protocols and guidance — for instance around our data strategy and our API strategy — and some, such as the GDS tech code of practice, and the local government service standard we’ve already adopted because we know they work.

What are we doing next?

We’re going to clearly delegate responsibility and decision making to team level wherever possible. To support our teams we’ll focus on growing key skills and behaviours around leadership, decision making, working in the open and use of evidence. As a senior team we’re committing to regularly and clearly communicating our approach including how we feel about risk.

These are big commitments and we know we can’t do everything at once. So over the next three months we’ve decided the focus will be on:

  • Using the updated Pipeline tool that went live this week to openly show the flow of our work
  • Running 5 service assessments, learning from doing these so that we know what our change process (production into live support) might look like in the future
  • Carrying out a discovery phase on a next iteration of our Hackney Agile Lifecycle to support our understanding of and narrative about our governance approach
  • Building a strategic procurement plan using data and insight from our contracts register

Standing on the shoulders of giants

Some really clever and thoughtful people have done great work on agile, governance and working at pace. Here’s my curation of some of the best blog posts/articles I’ve read, along with my thanks to all of them for sharing their work so openly:

HackIT Apprenticeship Programme – a managers view

Planning a successful induction for our first group of apprentices

I recently attended the manager’s induction session for the Hackney apprentice programme. This was a session run by the Hackney Works team for all managers that either are,  or will be, managing apprentices under the scheme.

At HackIT we’re investing in long term digital skills by hiring 21 digital apprentices. To be successful we know that we need to make sure our managers are equipped to support our new recruits. I’m one of those managers – and thought I’d share what I’ve learnt so far. I’m looking forward to welcoming our team’s three apprentices later this month, and working with Arch, their learning provider.

What is an apprentice?

Firstly – it’s important to know  what an apprenticeship is, and indeed what it isn’t.

It is all about learning and development, this may well be the apprentice’s first experience of the workplace. Where they will gain hands on experience, mentoring and nurturing to become, in our case, the future of ICT.

What it is not,  is a lesser option than going to uni. Although society seems to put more emphasis on academic success, we should not see an apprenticeship as any less important or valid than further education.

This may be the first time they have entered the workplace formally so during induction apprentices may need  more structured first days, weeks, or even months than other staff would. We need to explain where they fit in and what we hope they will be achieving. We’ll be asking ourselves, who are the key people that they need to meet on those first few days? We have to be careful not to overload or scare them by introducing everyone in a large office like a conveyor belt. Apart from this being a little overwhelming, it’s also very difficult to remember so many people all at once.

Learning on the job

During their apprenticeship 20% of their time will be spent on learning. This must be structured and regular. We must provide a quiet place for this in the office. This will be backed  up by creating a month by month work plan. Assigning a “buddy” from the team also helps as they may be more open with someone who isn’t a manager. The buddy will also help the apprentice with work on a day to day basis.

First day at work

We’re not assuming our apprentices will know about the workplace environment. Do you remember your first ever day at work? We had an interesting discussion around this at the training, where we shared our experiences and memories. Although it was a long time ago for some of us we could all still remember the details “It was raining” “the building was massive” “I didn’t know who anyone was, or what they did for the organisation”

I’m also mindful that that the apprentices will possibly be much younger than other members of our team. Or have different cultural barriers. So we’ll need to find ways to make them feel comfortable and welcome.

What happens at the end of the apprenticeship?

We want apprentices to move on to role in their chosen profession – this might be with us, but it might equally be in the wider industry -and that’s very much still a success story for the organisation. There isn’t a guaranteed job at the end – but we will help with next steps like updating CVs. Three months before the end of apprenticeship we’ll start having conversations to discuss their next steps. 

HackIT digital apprentices programme – delivering long term change

Our programme is taking shape

Since Rob Miller last blogged about our apprenticeship programme we’ve been working hard to get it off the ground. It’s a key part of our workforce strategy – we know that in a market where digital skills are at a premium we will need to work hard to attract the right candidates, and that growing our own talent is vital.

A successful recruitment campaign over the summer means that we have 21 new people joining us in September on a variety of level 3 and 4 apprenticeships. They come from a diverse range of backgrounds but they are all either Hackney residents or attended a Hackney school – part of the borough’s commitment to providing opportunities to residents.

In fact the standard of applicants we got was so good, and our managers are so engaged in wanting to develop the opportunity, that we’re hiring 3 more than we originally planned to. We’re confident we can support them all to develop over the next 2 years, in partnership with our 3 learning providers Ada, Arch and WKCIC.

Building a pipeline of talent is a team sport

We couldn’t have done this without the Hackney Works team who have given us fantastic support throughout. Their skills and experience have been invaluable in helping us through the process.

We’re also working with the wider digital community in Hackney to help us grow digital skills in the borough. Hackney has a thriving tech sector with world leading business (large and small) based here. In June we hosted an event for local employers and asked them for ideas.

They said:

  • Make sure apprentices are given specific projects to deliver so that they can build their own personal portfolios of experience
  • Support your managers with the skills they need to manage people who are at an early stage of their careers and have limited experience of the workplace
  • Work with us to share ideas and create opportunities to work together

And that’s what we’re doing.

We’re working with local employers to help the new recruits build their professional networks.  This includes working with MadeTech organising meetups with their trainee developers and our apprentices, Diva Apprentices to connect our apprentices to others in Hackney media companies and Amazon Web Services, developing mentoring relationships between Amazon’s graduate trainees and our apprentices.

We’d love to work with other local employers as well – if you’re interested please get in touch with us.

Looking ahead – women in tech

We’re also thinking now about the next time we recruit to our programme – in 2020. We will be working with people from different backgrounds, life experiences and heritages – an important enabler of building empathic digital services people prefer to use. We have 5 women and 16 men joining us in this first cohort. As a reflection of the IT industry as a whole this isn’t bad, but we are ambitious about improving it in the future and we think there is a lot we can do over the next two years to make that happen.

Sharing our work: Data Awareness Training Content

Over 1,000 of our staff have completed our mandatory Data Awareness training so far, and we are aiming to reach 100% compliance by the end of the roll-out. One of our aims of the project was to share our work, so here are our reflections and the content itself.

In preparation for GDPR, we asked:

How might we:

  • … equip our staff to fulfill their responsibilities to keep our resident and staff data safe, and to handle it lawfully.
  • …equip our decision makers to make decisions about data with an understanding of the law.
  • …support people working on behalf of Hackney who don’t have access to training budgets eg. foster carers.
  • …support people in our communities with data protection responsibilities that don’t have access to training eg. scout leaders.
  • …give other organisations the chance to use our content instead of spending their own time and effort on creating something similar.

We talked to several specialists agencies about how they could help, and selected Helpful Digital to work with us on development of the content. We decided to use their Digital Action Plan – their personalised, digital skills programme to help groups develop the confidence and skills to use digital tools at work.

How did we develop the content?

We kicked off with a workshop with Helpful Digital and staff from our Information Management and Information Security teams. We decided to develop one plan for people handling data, and one plan for people making decisions about data. We started by producing Hackney-specific content that referenced our policies and procedures.

The content took some time to refine – we wanted to make sure that it contained useful practical advice rather than regurgitating the law. The Council deliver such a vast range of services that it was a challenge to find examples that would be applicable to everyone. After our first draft we were lucky enough to have willing volunteers from across the Council to test out the content. Getting some extras eyes and perspectives on it helped us to identify the changes needed in the next iteration.

What next?

  • We will remove the references to Hackney Council policies and procedures, to produce equivalent plans for non-Hackney users.
  • We will adapt the decision-maker’s plan to to include scenarios specific to our Councillors.
  • We will plan for our annual refresher, thinking ahead about how we adapt content to support this.

How can you use this content?

You are free to use this content however you will find it useful. You might keep it in document form, but could also choose to move the content into your own Learning Management System. Another option is to use Helpful Digital’s Digital Action Plan platform with our content at a small cost – you can read more about this option here.

The Content:

We have published the content on GitHub here.

  • Content for people handling data is called ‘Beginner’ and marked 1-5.
  • Content for people making decisions about data is called ‘Intermediate’ and is marked 1-5.
  • There is a final quiz for all levels of plan.

Some of our post-it note ideas in the early stages of creating the content:

Developing a minimum viable Business Index

Hackney has  successfully completed our MVP for the Business Index (BX) a couple weeks ago and now we’ve had our project retro, I thought I’d share some thoughts after reflection. This is my first experience of working in an agile way, along with other team members in Data & Insight. There was much to learn along the way and valuable experience gained. To that extent, the BX MVP was much more than the creation of an automated ETL process and REST API.

Background

At Hackney, there has historically been an issue with duplicated business data within the systems as the business name has usually been a free text field with no validation. There’s a famous case where a supplier who was on our trusted supplier/contractor list had had numerous enforcement actions taken out against it but because there was no validation on the business name, upon investigation it was seen that this particular business had been created differently 18 times – therefore avoiding detection and being able to remain on the ‘trusted suppliers’ list. This was just one of the examples of how free text recording of businesses in the council has been allowed to proliferate bad data and therefore making it much harder to join up data and services.

Set up

We appointed two agencies to work with us – Unboxed and MastodonC. Unboxed were to carry out our user research and MastodonC for the technical build. I was appointed Product Owner and so the project started in earnest. We had 6 x 2 week sprints to complete our MVP. After extensive user research and many times defining and re-defining our scope for the MVP, the user that we thought we could benefit the most in this iteration would be the developers working with the Public Realm Digital Transformation team – namely working on a new application for businesses wishing to apply to for a Shop Front License. This would be a new application for a license which would then serve to hold all other license details in the one portal. They had a need for an applicant to enter their postcode, select their address and select their business name associated to that address – thus keeping the data consistent.

https://hackney.gov.uk/markets-shop-front-traders

 

The build

The technical discovery and build came next. We needed to derive the business name and the key datasets we would be using would be LLPG (Local Land and Property Gazetteer), Non Domestic Rates and Civica APP (where currently all license data was sitting).  The LLPG is the councils core address dataset and each address has a unique property reference number (UPRN) – most systems in the council are integrated with the LLPG and hold the UPRN against their addresses. Civica APP has the UPRN attached to it’s licensing data. Non domestic rates doesn’t have the UPRN but the reference number for Non Domestic Rates is held in the LLPG which creates a link. By using these 3 datasets, we would determine which was the correct business name for property. Our logic was determined as – Business Name would come from LLPG – if no business name was present in the LLPG then it would come from Civica – if multiple names existed in Civica then the last seen date would be used to determine the name put forward for Shop Front Licenses in the API. We had decided that only 1 business name would be returned in the API and there would be an option to add a new business name if it wasn’t returned. This will be sent through to the Data & Insight team for investigation/verification and then added to the LLPG. A link to our API is below.

https://api.hackney.mastodonc.net/index.html#/api

 

Looking back…

As mentioned at the start, this was the first time I and other members of the Hackney team had been working on an Agile project and I don’t think we could have achieved our MVP without doing so. The project team stand-ups were an essential part of this – even if you had nothing to update, just keep in the loop and give a smile and a wave to the other members of the team made a difference. Same format each day – What did we do yesterday, what are we working on today and any blockers – jointly being able to update our trello board. This kept us on track for each sprint. However, after our fortnightly Show & Tells where we shared what we had done in the previous sprint – we would hold a Sprint Retrospective and planning session for the next one. It was these that were, I would say, the most important part of the process. Even though we spoke to each other everyday, it was during the retros (mostly) that concerns were raised – honestly and with respect.  

After successfully completing our MVP we had our final Project Retrospective. Whilst working on the MVP there were a number of things that went well – some that didn’t… by having a Project Retrospective it was a great opportunity to capture all the learning – by everyone. There were a few things which were specific to this project alone but most things can be applied to future projects.

Committing to working in the open

Every three months we provide an update to our board which sets out what we’ve been working on, the progress we’ve made and areas that we need to focus on.

As part of our commitment to working in the open (a key part of the Local Digital Declaration) we’ll be publishing these on our blog so that others can see what we’re up to. We’re particularly keen to hear from other councils who’d like to compare notes, share ideas and suggest ways that we might be able to go even further with the work we’re doing at Hackney. And we’re very open to following up opportunities for closer working together too.

In this quarter’s update we’ve got plenty of great progress, including:

  • Over 4,000 residents checked their registration to vote in the May 2018 election using our One Account service
  • The design led approach we’ve taken for the implementation of the Homelessness Reduction Act means we are able to complete interviews with people presenting as homeless much more quickly and easily
  • We’re starting to link in with the ‘Government as a Platform’ services provided by GDS through our work to improve income collection
  • 50% of repairs contact is now entirely online (with no need for a call back from the contact centre)
  • Our new digital service for fostering is live, making it easier for people to find out how they can provide a caring home for vulnerable children
  • We’ve successfully moved c 3,500 people over to modern productivity tools
  • Our Local Land & Property Gazetteer has achieved the gold standard for data quality this month
  • Our commitment to data is continuing through a broad programme of work supporting the requirements of the General Data Protection Regulation, including new Digital Action Plan training for all our staff – designed to make it easy for our people to understand their data responsibilities
  • We’re continuing to do the hard yards to make sure our systems and applications are reliable and secure (thankless, never-ending, but vital)
  • We’ve had brilliant new colleagues joining our team following our recent recruitment, and we’re well underway with the recruitment to our new apprentice programme
  • And our work with colleagues in other support services is helping to make it easier for our users to get help and advice when they need it so that they can focus on delivering excellent services for our residents

But it’s also essential that we’re frank about where we need to give more attention so that we can do even better. Key areas we’re focusing on at the moment include:

  • Making sure that we can demonstrate and realise the benefits of the work we’re doing
  • Supporting our team so that they can provide high quality support for the new services we’re rolling out
  • Learning more about how we can understand data and data quality
  • Making sure that we are focusing on users and user needs in everything we do
  • And making sure that we take the time to bring new colleagues joining other Council services up to speed, which is well worth the time and effort

You can read the full update here: http://bit.ly/2KdM6Wb

In support of the Local Digital Declaration

At Hackney we are working hard to provide our residents with digital services that are so good, people prefer to use them. By harnessing the power of digital technology, data and service design it is now easier than ever before for Hackney residents to check their rent accounts quickly from their mobile phone, get support to find opportunities for employment and skills development, access the Council’s help to reduce the risk of homelessness, and find opportunities to provide a foster home for a child.

This is just the start and there is much more that we want to do. We believe firmly in working with our residents to co-design our new digital services and we also believe in the importance of working collaboratively with our colleagues in other councils so that we can share the benefits of each other’s work and deliver progress together more quickly and at lower cost.

Much has been written and said about how digital collaboration in local government should be done. Suggested solutions range from formal shared services, where councils come together through joint structures, through to less formal joint working across councils based on identifying shared opportunities. The challenge is to make sure that we can continue to respond to local needs, minimise bureaucracy and set a clear and purposeful direction that will deliver at pace over the medium and long term.

We welcome the new Local Digital Declaration (http://www.localdigital.gov.uk/declaration) and we believe that this sets out principles that are consistent with our values in Hackney.  We are pleased that this is being developed in partnership with councils across the country, not imposed from Whitehall; we support the focus on service design, digital skills and the underlying technology foundations that need to be in place; and we share the commitment to open and collaborative working across the sector so that we can learn from one another.

We also calling on our current and future suppliers to commit to the principles in the Declaration. From our experience in Hackney we have seen how working with innovative digital suppliers can help us deliver exceptional results and we know that their expertise can also help us develop our in-house skills for the longer term. We need to work with suppliers who will adopt the best of modern digital technology, commit to open standards and can demonstrate their ability to deliver services of the standard that our residents rightly expect. You can read our open letter to suppliers here: http://hackit.org.uk/work-with-us/suppliers/an-open-letter-to-suppliers.

We are looking forward to playing a full role as part of the local government digital community to help realise the vision set out in the Declaration and we are confident that this will benefit our residents in Hackney as well as citizens in other areas of the country.

The Mayor of Hackney sets out our commitment in this video:

Developing our API strategy

TL;DR

One of the decisions at last week’s ICT management meeting was to confirm our commitment to an API strategy:

  • Ensuring that all of our key datasets are available via a REST API so good, that people prefer to use it
  • Replacing existing connects with REST APIs that can be used independently of each other
  • Developing our skills to ensure we can use the modern programming languages, development techniques, tools and services necessary to deliver high quality services
  • Developing our cloud-first approach, ensuring we’ve got a sustainable digital architecture that supports continuous deployment and continuous integration
  • Ensuring our code remains secure but is open source

Background

Last year, we were considering options for renewing our CRM. We reviewed the CRM market and also experimented with lowcode and, in particular, Outsystems which seemed to be the best on the market. We built Pay My Rent in just 12 weeks using Outsystems, connected to data in Dynamics, and it’s been working well ever since.

However, when we began developing the housing repairs service, we found that lowcode was a barrier to working with suppliers. They would have to spend time learning lowcode. Most of our suppliers were expert in Ruby on Rails, a modern, lightweight programming language – but not one we knew. We also learnt that whilst lowcode helped us develop apps quickly, we were still reliant on data in legacy systems for managing the end-to-end journey.

We also learnt that if we connected new services to legacy systems using the integrations we used to write (SOAP webservices and direct database connections) then we would have to re-write them when we were ready to migrate from Universal Housing. So we experimented with developing a REST API to support online repairs. Previously we would have built an integration that put data directly into the database, which:

  • Would be hosted on-premise (for security) so unusable to anyone outside Hackney
  • Was impossible to monitor availability in realtime – so we’d have found out it was broken when part of the service failed
  • Could not be re-used, and would have had to have been re-written if either the application or the database changed

We called this work a ‘prototype’ because there were a number of questions we couldn’t yet answer:

  • Did we have sufficient skills in the team to write REST APIs?
  • What were the benefits of writing REST APIs?
  • Could our existing technology support an API-driven approach?
  • Could we get the best from existing software (lowcode, CRM) whilst increasing our capacity by working with local agencies?

Over the last six months we have been trying to answer these questions.

Work to date

We deployed the ‘Hackney API’ that enabled repairs, and started adding more data to it. It was underpinned by a draft set of standards so that we could develop future APIs to a consistent format.

The API was hosted on-premise as a short-term workaround, and the code was on a private repository of GitHub initially. We then re-wrote (‘refactored’ in the jargon) the API to individual APIs that work independently of each other. That means changes to code won’t create problems in other parts of the API and enable us to continuously develop the services.  

The development team has made a strong start in developing their skills. We began by co-locating our developers with partners in agencies. We got some great feedback from our agencies about the speed with which our devs learnt.

The NHO team adopted the API-based approach and began developing a more flexible approach so that they could better meet user needs. The team, and the income collection team were both able to benefit from the API built for Pay My Rent, because they’re using the same data. So they developed the idea of a suite of APIs that could be used by any team.

Our emerging standards were adopted by the Business Index team.

We’ve also shown that we can build APIs on our own. Sachin, Sandrine and Tapan, working with Tejus developed a suite of APIs to support the LLPG.

What we’ve learnt

Writing great APIs demands we use a range of techniques to improve the way we write code. Pair programming, test-driven development and coding in the open are all important attributes of writing great code and therefore making our APIs reusable outside Hackney.

Our developers are learning a lot, but need help, support and time to increase their skills. There are too many demands on their time, so we need to get better at prioritising what they do.

You can’t just deploy cloud-based applications which seamlessly connected to on-premise, legacy systems. And you can’t publish open source code that’s reusable if you haven’t got the security sorted out.

There are lots of tools that can help us – from Platform as a Service such as Heroku to software tools such as Swagger, for documenting APIs, and TeamCity/Circle CI for continuous integration. However, none of these are shortcuts to writing good code.

We need to be able to continuously deploy code, ensuring it works and connects seamlessly to our systems and infrastructure. Our change control process enables this, but we’ve got more to do to support reliable, secure access between cloud software and our legacy applications.

 

What we will do now

We’re making a clear commitment to an API-based approach. We believe it will:

  • Enable us to re-use data and connections as we develop more digital services
  • Provide the potential for third party applications to connect to our data and (potentially) us to use more third party apps and services
  • Reduce our dependency on large systems and reduce the risks of switching between applications
  • Increase the visibility of integrations between systems, reducing the costs of support and maintenance
  • Improve the security of our systems by having a secure OAUTH system in place

However, we recognise that this won’t be straightforward. We’ll need to:

  • Work closer together, ensuring our digital design team, developers, architects, infrastructure and data teams can all contribute to the approach
  • Support colleagues in their continued learning and development
  • Learn from experts to ensure that our API products are high quality
  • Track our capacity to ensure that new services are improved continuously but that our legacy applications continue to be well-supported
  • Track the cost and benefits of this approach to ensure it delivers a better IT service for residents, businesses and colleagues

We’re already working with MadeTech to provide support and challenge for our API strategy. We’ll now be drawing on a range of experts to support the whole team in understanding best practice. We want to increase the number of opportunities available for people to learn coding – whether through our new learning platform, secondments into the team or secondments into projects to develop new services. And we need to work harder to open up our work, so that we:

  • gain from other people’s wisdom (their input can help improve the quality of our code) either indirectly, or through recruitment
  • make a valuable contribution across local government by helping other authorities who are starting out on a similar journey / don’t have the resources that we do play a part in resetting the local government digital landscape, helping the sector to unlock itself from the traditional oligopoly

There’s a lot to do – but the opportunity has never been greater. We’re in a unique position in Hackney to make this work.