When is an API Gateway not an API Gateway? – When it’s an API!

HackIT’s development team have some big ambitions with our API Hub and API Factory (our project which is refactoring of our Hackney APIs into a microservices architecture and delivering future APIs in the most effective and consistent way). The team has been steaming ahead with ideas to make our APIs so good people prefer to use them. But we know that this drive for innovation can’t be done without regular checks against industry standards and best practice. So with that in mind, members of the dev and security team sat with a team from Amazon as the first part of a “Well Architected Review” to sanity check what we were doing and ensure that our approach meets industry standards. Our HackIT manifesto states that we learn more “standing on the shoulders of giants” and Amazon are one of those giants we want to learn from.

We as Hackney team would like to thank AWS team for giving us valuable insights on the tech platform. We are really looking forward to the next session on 17th July.

Once everyone had settled in – with some of the Amazon team (comprised of Katie, Alice, Abir and Mani) very much looking forward to taking part in their first workshop from bean bags, Rashmi set about giving an overview of  why are we doing this, where we are currently and where we want to be.  

Well Architected Review

We then set about diving into the Well Architected Form which is built in to AWS’ cloud platform – albeit we had to switch our region to Ireland as it’s not available in the London region at the moment – but who doesn’t love Ireland? We began answering a series of questions over various areas such as what our priorities are, how we can understand the state of our application and how we can reduce defects. 

It became apparent as we were completing the form that at this point in the process it might be better to have a less structured discussion about what our questions and worries are. We were advised to complete the form at some point in the future but continued the meeting by talking more specifically about what we have in AWS and what we’re trying to achieve.

Amazon estate options

This led to “story time” starting with Alice (solutions architect at Amazon) telling us from her bean-bag that she felt like a teacher at story time. We broke out the white-board and Alice began explaining how many organisations set up their AWS estate (although she was worried as she’d been told that apparently drawing with red pens makes people angry subliminally).

Thankfully the red pen only seemed to engender rather a lot of excited faces amongst the dev team as we thought about the possibilities of building a more secure and easy to manage AWS setup.

We also discussed segregating our current landscape into master account, shared services (such as Active directory federation for example), workspace and multiple developer account model. The multiple developer account model would allow templated “scratchpad” areas for developers to effectively “mess around” and try things out. This would all use a product called Landing zone that would make managing this setup far simpler – providing advice on security, cost optimisation and performance.

Below is the sketch out of how the multiple account model along with Landing zone might look.

The above seems complicated – however this would potentially give us far greater security and a more resilient cloud setup. Specific accounts for specific workloads provides a modularised approach to the cloud infrastructure meaning components can be added and removed with less opportunity for error.  

Hackney specific question time

We then moved on to more specific discussions about our platform setup including our use of the API Gateway and how we would manage API Keys, as well as our strategy for migrating and synchronizing data in the cloud.

A lot of time was spent around the current setup within AWS (which consists of a single API Gateway and a single API inside it). Inside that single API all of the rest of our APIs are individual paths. A suggestion for the future was to split each API in to its own API Gateway. This discussion went to and fro for some time with much pointing at screens and hand gestures. Eventually we came to the realisation that we need 1 Gateway and multiple APIs (because this would make access management much easier) but we aren’t sure what that looks like until we’ve had enough time to think it through and design it accordingly.

We also discussed our current ecosystem with the team. One thing we specifically discussed was our current mechanism for logging. They recommended their tool ‘CloudTrail’ however they implied we should be more clear about what we expect from the logs. 

We also found out about a tool provided by AWS – ‘Config’. The tool tracks configuration changes and can be customized to include triggers which notify us when a change has occurred. For example, anytime someone changes our root user password, we could set a trigger to get a notification. We will look into the tool as we believe it could increase the security of our platform and applications. 

Another key take-away from this session related to a current concern we have around managing access to our APIs and that our current implementation would not scale very well.  After much discussion and consultation with our AWS colleagues we came across some really useful ideas that the team were itching to explore where we could really expand on the use of the API Gateway resource.

We discussed how we could we master our current housing dataset on cloud. We spoke about work previously done as a spike. We spoke about the Database Migration Service (DMS) approach we’d taken to extract the schema and data out from our legacy database and described how it had failed due to incompatibility of our legacy system with the tools on offer. The AWS team suggested we send them the details which they could review and provide further advice on it.

Our next steps

The meeting has left us with several areas for research, discussion and food for thought prior to our second conclusive meeting we have scheduled on the 17th of July. To conclude, we have identified several areas for further discussion and review:

  • Two new proposals for API access management that we have come up with and that will be reviewed by AWS team.
  • The feasibility of replacing our EC2 services (running Docker containers on hosted servers) with Fargate (running Docker containers in a serverless environment) and what would the benefits of that be in terms of performance and costs.
  • Possible data migration services provided by AWS to help us get our legacy data into the cloud.

Service assessments: treasure, dragons and a place to dry your socks

An unsuspecting vanguard 

The Hackney Spacebank team are looking at ways we can improve access to low cost or no cost meeting space in the borough for voluntary and community groups, micro businesses and residents. 

We’ve been squirrelling away for the last six months – give or take a couple of fire breaks – and we’re now ready to build our MVP. It feels like we’re at the bottom of a mountain, blinking into the sun, trying to get a glimpse of a hazy summit. We’ve got some navigation, our wits and a great team ethos. Like the best kind of adventures, there will be mysteries to unravel, treasure to be found and dragons to be tackled along the way. 

We’ve completed two service assessments on our journey thus far. These are like a bit Scottish Bothies – stone dwellings for individuals or small groups hiking in the mountains. A place of shelter to pause, unpack our bags, refuel and regroup. At Hackney, service assessments are an assurance, not a gateway. They are an opportunity to ask: are we doing the right thing, in the right way, at the right time? 

Our Spacebank team have been accidental pioneers of Hackney service assessments. We volunteered to do the first ever discovery phase service assessment back in March where we experimented with a self assessment approach and a couple of weeks ago we completed the first prototype (Alpha) assessment. 

A reluctant start

I didn’t feel popular when I broached the subject of prepping for a service assessment mid way through a series of five sprints. I understand why. The team were knee deep in the detail of prototyping a new service. I was asking them to step back and reflect. All the information we needed was in their heads and I had to find a way to extract it with minimal disruption. 

Over the next couple of sprints, I ran two 1-hour workshops. We took three or four service assessment standards and answered two questions: what’s the story we want to tell and how can we evidence this?* We looked at the good stuff and the not so good stuff, treating everything as valuable learning and identifying areas where the assessors could help us improve. 

For some of the team, completing a service assessment was new territory. I kicked the first workshop off with a short slide deck so that everyone knew enough** to join in. After we had bottomed out each service standard and assigned them to a team member, we agreed how to structure the assessment itself. We wanted the prototype to do the heavy lifting. We used a demo to walk through our proposed minimal viable product (MVP) and highlight some of our key messages. For example: our growing understanding of users’ needs, iterations as a result of testing, and tech choices.

On the day we were ready. There was a healthy dose of nerves, supplemented by caffeine and chocolate. After the demo, we broke into small groups each led by an assessor. This was the opportunity for an in-depth conversation on a smaller number of service standards. To wrap up we came back together and assessors fed back their thoughts and recommendations to the whole group. 

And at the end of it all, what did the team say: 

“The assessors were excellent and gave really useful, actionable feedback.” Emma, Developer

“I liked how we prepared by giving the assessors the information they needed for each service standard on Trello before the assessment. This allowed us to use the time with the assessors to get their feedback on things we could consider during the next phase of the project.” Sam, User Researcher

“I think the format in which we did an intro, a demo and breaking up into groups and playing back the highlights was a really good way of making the most of our time in our service assessment.” Joy, Service Designer

And our assessors said:

“Super interesting alpha assessment today at #HackIT. It’s always a privilege to meet talented committed teams doing the hard work to make things better for users.” Kate McCaul, Head of Digital at Acas

What are we learning?

Here’s what we are learning about participating in service assessments:

  • Preparation in bite sized chunks – little and often but involve the whole team
  • Take the lead (delivery managers/product owners) but share the responsibility
  • Give ownership of each standard to the relevant team member
  • Show “the thing” and practise showing “the thing”
  • Give assessors early sight of preparations
  • Structure the assessment to support conversations rather than questions and answers
  • Talk about the not so good – your best learning comes from this

Feel free to explore our Trello board and read our service assessment report. A big thank you to our assessors: Cate MacLaurin, Head of Delivery at Hackney, Kate McCaul, Head of Digital at Acas and Giulia Merlo, Service Design Lead at Cancer Research UK. Any tips on dragon handling are welcome. 

*The second question became redundant quite quickly as describing the story surfaced the evidence simultaneously

**What’s enough? A judgement call based on the personalities in the team and confidence levels. 

The hiking analogy of this blog post has been inspired by my long-time friend Shona MacPherson who lives in the Scottish Highlands. Today, she starts a solo hike of the Pacific Crest Trail, an epic 2650 mile walk from the U.S. border with Canada to the U.S. border with Mexico. She’s raising money for Mikeysline – a suicide prevention charity. 

User stories not requirements will give us better outcomes – discuss

I pitched a session at #procurementunconference19 boldly titled ‘User stories not requirements will give us better outcomes’

I wanted to find out if other people had experiences to share about how we move away from long lists of requirements, developed in silos, then given to suppliers – to a more collaborative, outcome focused way of doing things. 

At HackIT we use the digital marketplace where we can, and make wide use of CCS frameworks. I used a recent example of a procurement for ‘printing as a service’ where we’ve tried hard to give the suppliers space to tell us what they think the solution might be for us, rather than specifying every detail in advance. 

The discussion around the table was incredibly useful – a host of examples of where people are trying new things, facing challenges and being bold with their approaches.

Here’s the top 5 things I wrote down during the discussion:

Too often we’re trying to buy something where we’ve got uncertainty– and instead of trying to make it more certain (by locking down requirements) we need a way of embracing the uncertainty and using it to get better outcomes overall. Otherwise we risk buying the wrong thing.

Procurement works best when you have the right people working together at the right time, who understand the problem you’re trying to solve. This means having product groups and multi-disciplinary teams in place, and break down professional silos where documents are passed from team to team.

Can we use experiments to understand the problem – and if so, how do we make experiments as small as possible? The smaller they are the less risk you have, which means you can fail faster. For example – we’ve added in a 6 month pilot of our new ‘printing as a service’ solution, but could we have made this smaller and more iterative? How can we involve suppliers and users during our experiments?

Can we use KPIs that measure delivery against culture and approachwhere we might usually only use outputs (eg availability), and if we could do this what would our measures be and would it drive innovation and engagement?

There was an example of teams using story points or dev pointsas an agreed deliverable in a statement of work – and of teams planning in advance to create iterative statements of work in collaboration with suppliers, rather than fixing an overall statement of work at the start.

Notes from #procurement unconference19, Session 1 – Bear

Originally published at madebycate.mclaur.in

Extending Repairs Hub to RCC Agents

The Repairs Hub has now been actively used for over 2 weeks in the Repairs Contact Centre and the agents have successfully used it to raise 500 repairs so far. During the last week, we’ve implemented some changes based on what we’ve learned from users during this time.

Improvements made in the latest release:

Added link from the NCC (CRM) app directly to the property page in Repairs Hub – there’s no more need to search twice for the same address

Added support for further SOR codes when raising repairs

Description for new repairs is now limited to 250 characters (and we’ve added a handy character counter)

Resolved issue with assigning ‘gas breakdown’ jobs to ‘gas servicing’ – they should come through correctly now

Added alert for agents to not book an appointment for immediate and emergency jobs

Added ‘raised by’ on works order page to display the name of agent who raised the repair

We’re now going into our last week and planning to make a few more eagerly awaited improvements. We will hold our last Show & Tell on Tuesday 9th July at 10:30 in the ground floor meeting room at the Florfield depot. Come join us there!

Link to Sprint 6 Show and Tell Slides

LOTI – working together to extend digital apprenticeships across London

This week saw the launch of the London Office of Technology and Innovation (LOTI), and at Hackney we’re really proud to be one of the founding members.

One of LOTIs first strands of work is a shared endeavour to scale digital apprenticeships across the core LOTI boroughs, with a working goal of 100 apprenticeships by 2020. We want to lay the foundations for developing digital skills across London’s diverse communities.

We’re also clear that we want to use this opportunity to improve diversity in the tech sector – challenging the status quo, and using our collective energy to make a difference.

Hackney’s ambitious apprenticeship programme is well underway so we’ve been asked to lead the work on this strand. Apprenticeships are a key part of our workforce strategy – we know that in a market where digital skills are at a premium we need to work hard to attract the right candidates, and that growing our own talent is vital. It’s also a great way of bringing new ideas, energy and diverse experiences into our team.

The day after LOTI’s launch we held a kick-off workshop with 9 other London local authorities, the GLA and Government Digital Service, and the new Director of LOTI, Eddie Copeland.

Hackney’s approach

At Hackney we’re open about what we think has helped us make a success of our apprentice programme so far and also about the challenges we’ve faced. We shared that story with the room, starting with our apprentices’ experience of joining our team and beginning their studies. We then asked people to share with each other the things they are working on, details of any barriers they’re facing, and the ambitions they have to create apprenticeship opportunities for people in their boroughs.

That’s given us a really good insight into where we should focus our efforts next. The priorities we’ll be working on include:

  • Sharing our knowledge and skills
  • Collaborating on specific issues, such as navigating the apprenticeship standards and procuring training providers
  • Building momentum by helping and supporting each other when we’re faced with a ‘you can’t’

We will also need to decide how we’ll collaborate when we’re back in our home councils. We’re a large-ish and diverse groups of people, and our various organisations almost certainly use different collaboration tools. So we’ll be giving some thought to how we can best grow this network, share our knowledge, and work with the core LOTI team as they get started and see how we can foster easy collaboration between us.