Developing our API strategy

TL;DR

One of the decisions at last week’s ICT management meeting was to confirm our commitment to an API strategy:

  • Ensuring that all of our key datasets are available via a REST API so good, that people prefer to use it
  • Replacing existing connects with REST APIs that can be used independently of each other
  • Developing our skills to ensure we can use the modern programming languages, development techniques, tools and services necessary to deliver high quality services
  • Developing our cloud-first approach, ensuring we’ve got a sustainable digital architecture that supports continuous deployment and continuous integration
  • Ensuring our code remains secure but is open source

Background

Last year, we were considering options for renewing our CRM. We reviewed the CRM market and also experimented with lowcode and, in particular, Outsystems which seemed to be the best on the market. We built Pay My Rent in just 12 weeks using Outsystems, connected to data in Dynamics, and it’s been working well ever since.

However, when we began developing the housing repairs service, we found that lowcode was a barrier to working with suppliers. They would have to spend time learning lowcode. Most of our suppliers were expert in Ruby on Rails, a modern, lightweight programming language – but not one we knew. We also learnt that whilst lowcode helped us develop apps quickly, we were still reliant on data in legacy systems for managing the end-to-end journey.

We also learnt that if we connected new services to legacy systems using the integrations we used to write (SOAP webservices and direct database connections) then we would have to re-write them when we were ready to migrate from Universal Housing. So we experimented with developing a REST API to support online repairs. Previously we would have built an integration that put data directly into the database, which:

  • Would be hosted on-premise (for security) so unusable to anyone outside Hackney
  • Was impossible to monitor availability in realtime – so we’d have found out it was broken when part of the service failed
  • Could not be re-used, and would have had to have been re-written if either the application or the database changed

We called this work a ‘prototype’ because there were a number of questions we couldn’t yet answer:

  • Did we have sufficient skills in the team to write REST APIs?
  • What were the benefits of writing REST APIs?
  • Could our existing technology support an API-driven approach?
  • Could we get the best from existing software (lowcode, CRM) whilst increasing our capacity by working with local agencies?

Over the last six months we have been trying to answer these questions.

Work to date

We deployed the ‘Hackney API’ that enabled repairs, and started adding more data to it. It was underpinned by a draft set of standards so that we could develop future APIs to a consistent format.

The API was hosted on-premise as a short-term workaround, and the code was on a private repository of GitHub initially. We then re-wrote (‘refactored’ in the jargon) the API to individual APIs that work independently of each other. That means changes to code won’t create problems in other parts of the API and enable us to continuously develop the services.  

The development team has made a strong start in developing their skills. We began by co-locating our developers with partners in agencies. We got some great feedback from our agencies about the speed with which our devs learnt.

The NHO team adopted the API-based approach and began developing a more flexible approach so that they could better meet user needs. The team, and the income collection team were both able to benefit from the API built for Pay My Rent, because they’re using the same data. So they developed the idea of a suite of APIs that could be used by any team.

Our emerging standards were adopted by the Business Index team.

We’ve also shown that we can build APIs on our own. Sachin, Sandrine and Tapan, working with Tejus developed a suite of APIs to support the LLPG.

What we’ve learnt

Writing great APIs demands we use a range of techniques to improve the way we write code. Pair programming, test-driven development and coding in the open are all important attributes of writing great code and therefore making our APIs reusable outside Hackney.

Our developers are learning a lot, but need help, support and time to increase their skills. There are too many demands on their time, so we need to get better at prioritising what they do.

You can’t just deploy cloud-based applications which seamlessly connected to on-premise, legacy systems. And you can’t publish open source code that’s reusable if you haven’t got the security sorted out.

There are lots of tools that can help us – from Platform as a Service such as Heroku to software tools such as Swagger, for documenting APIs, and TeamCity/Circle CI for continuous integration. However, none of these are shortcuts to writing good code.

We need to be able to continuously deploy code, ensuring it works and connects seamlessly to our systems and infrastructure. Our change control process enables this, but we’ve got more to do to support reliable, secure access between cloud software and our legacy applications.

 

What we will do now

We’re making a clear commitment to an API-based approach. We believe it will:

  • Enable us to re-use data and connections as we develop more digital services
  • Provide the potential for third party applications to connect to our data and (potentially) us to use more third party apps and services
  • Reduce our dependency on large systems and reduce the risks of switching between applications
  • Increase the visibility of integrations between systems, reducing the costs of support and maintenance
  • Improve the security of our systems by having a secure OAUTH system in place

However, we recognise that this won’t be straightforward. We’ll need to:

  • Work closer together, ensuring our digital design team, developers, architects, infrastructure and data teams can all contribute to the approach
  • Support colleagues in their continued learning and development
  • Learn from experts to ensure that our API products are high quality
  • Track our capacity to ensure that new services are improved continuously but that our legacy applications continue to be well-supported
  • Track the cost and benefits of this approach to ensure it delivers a better IT service for residents, businesses and colleagues

We’re already working with MadeTech to provide support and challenge for our API strategy. We’ll now be drawing on a range of experts to support the whole team in understanding best practice. We want to increase the number of opportunities available for people to learn coding – whether through our new learning platform, secondments into the team or secondments into projects to develop new services. And we need to work harder to open up our work, so that we:

  • gain from other people’s wisdom (their input can help improve the quality of our code) either indirectly, or through recruitment
  • make a valuable contribution across local government by helping other authorities who are starting out on a similar journey / don’t have the resources that we do play a part in resetting the local government digital landscape, helping the sector to unlock itself from the traditional oligopoly

There’s a lot to do – but the opportunity has never been greater. We’re in a unique position in Hackney to make this work.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.