Housing data in the cloud: Weeknote, w/c 02.12.2019

Housing data in the cloud. It does what it says on the tin. 

We’ve got data, lots of data. 

This data lives in an old house which has had many occupants. Each occupant has added data, moved that data around, put it in different rooms, called it different things and used it to prop up the fabric of the building. The old house is weighed down with data, nobody can find what they are looking for and removing data risks a structural collapse. 

We are changing this. 

That’s a bold statement, let me put some context around it. We are taking our first experimental steps to see if we can move a tiny piece of housing data into a cloud platform. 

Back in October we did a week long discovery to identify a data candidate for prototype and to think about what success might look like. Thankfully, we’re not starting from scratch. Other fine minds have looked at our old, overstuffed house – we’ve valiantly attempted renovation and even extension. Colleagues from MadeTech and pulled all this learning together and made a set of recommendations, which we are testing in the prototype we are building over the next few weeks.  

Introducing the dream team

The avengers getting ready for battle.
This is what we look like in my imagination. I’m Natasha Romanoff obviously 🙂

We are working with support from AWS and MadeTech along with our award winning team of Hackney developers. With got expertise from our data and insight team and three technical architects (at my last count). I’m terrified, the team is absolutely buzzing. We’re finally staring into the eyes of our nemesis – let the battle commence. 

We are working in 5 day sprints. I love the drive of the team. They want to work hard and fast. We’ve covering new ground everyday. The team are absorbing new ideas, skills and ways of working like a sponge. We don’t all see things in the same way, but the team are embracing this too. 

This week, we’ve identified our use case. Right from the outset we want to demonstrate how our work can bring tangible value to the Council services that rely on this data. We’ve got to keep this grounded in business need and ultimately the needs of Hackney residents. We’ve also set our cloud platform in AWS this week. Next up: is deciding what database we need for this prototype. There is A LOT of debate about this in the team. I’m expecting a few fireworks. We’ve got a spike early next week to try and crack this.

Manage Arrears weeknote: w.c. 2/12/2019

*Phase 3, Sprint 6*

The Manage Arrears project is about creating a simple service for residents and caseworkers that is easy to use and improves arrears management and collection. Phase 3 particularly focus’ on enhancing the system for caseworkers. Our goals for this phase include;

  1. Onboarding the Rents team onto Manage Arrears (using Universal Housing to carry out actions unavailable in Manage Arrears)
  2. Automating Letter 1 and Letter 2 for Rents
  3. Adding the Service Charge LBA template for Leasehold Services
  4. Enabling the Rents team to work in patches

This week, we cleared our diaries to focus on preparing Manage arrears for Rents. We have slightly de-scoped what we originally planned to release on Monday 9th December to allow caseworkers an opportunity to feedback on ‘real’ cases appearing in their worktray. Also because earlier in the week we thought removing certain patches from UH would affect the agreement process (note: we are now confident that this is not the case).  

*Highlights of this week*

1. +Joe Bramwall shadowed a caseworker earlier in the week to time their current workflows/processes. He gained some useful insights. One thing he observed was the personal interactions between the officer and tenant as he watched them contacting a resident before sending a letter 2. This will be lost through automation. The current proposal is to automate this stage from next week but his findings have reinforced the usefulness of remapping the new processes using Manage Arrears. 

2.  Training caseworkers and team leaders from patches W2 and W3 on Thursday. This session was useful as it helped me think about key aspects of the system which will require ongoing support after Made Tech conclude this phase. The types of things which popped up included new action diary types and text amendments to correspondence. I have started to compile a list to make sure our in-house resources learn and/or have documented guidance on how to make simple adjustments like these.  

*Not so cool thing*

During retro, our proxy-product owner pointed out that the service area thought they would be getting more automated SMS nudges sent to residents. This has been missed 🙁 It is not impossible for us to add but it may not be possible for us to include in this phase.

*Next week*

We plan to: 

  1. Work closely to support newly on-boarded caseworkers from W2 and W3
  2. Switch the schedule for the early SMS message back on
  3. Have caseworkers cross-check a few cases due for an automated Letter 2
  4. Switch on automation for letter 2 (Letter 1 will be switched on after the Christmas break, this will involve us removing cases from UH)
  5. Focus on finishing the remaining priorities in the worktray
  6. Continue time tasks so we can make a comparison between the old practices and new

Hackney Spacebank: Weeknote, w/c 02.12.19

Trees being blown in a storm.
It’s been one of those sprints

The weeknote of doom.

I dread sitting down to write this sort of weeknote.

We didn’t meet our sprint goal. We got 90 per cent of the way there but we missed “done”. It’s profoundly frustrating for the team. 10 days’ hard work and apparently nothing to show for it. 

Facing the unknown

The sprint goal felt right, the number stories felt achievable, our output was clearly defined. So, what happened?

The unknown hit us like a stonking great wind. A invisible force that you can hear and feel, but you can’t see it or catch it.  

What do we know now that we didn’t know two weeks’ ago:

  • What we can – and can’t do – with content on the Hackney website*
  • New team members needing a more clarity on our user groups 
  • The working cadence of other teams we are collaborating with 
  • The untapped enthusiasm of our colleagues at Dalston library**

What’s yoga got to do with anything?

Pokemon character Stitch trying to do a  yoga headstand.
Stitch grappling with a headstand

I’m borrowing this analogy from Matthew Cain. Agile – like yoga – a daily practice. After a while the rhythm of its ways, its principles and values starts to shape a team’s mindset. Yoga sculpts the body inside and out. In the same way agile moulds how a team works together and how it behaves. 

Our habit of sprint review and retro with a #nojudgement culture turned our failure into actionable nuggets – golden pieces of learning forged in the heat of honest conversation. 

I knew that I had a team that trusted each other to say hard things. I now know that we can weather storms together. 

Failure feels good

I’m proud that the team took hold of the agile principle to deliver a small piece of value to our end users in two weeks. There’s nowhere to hide when you make a commitment like this. This takes courage. 

We’re not going to retreat into safe incremental steps. We are going to keep working iteratively, even if it means we fail more often. 

*Our website is an MVP.

**This is great! 

Manage a Tenancy : Week ending: 2019-12-06

Weeknotes are a way for us to keep people informed about progress on the project. Given the technical nature of the re-platforming work we will use them to explain technical choices that we are making, including the benefit and impact of these choices. 

Project goals

  • Enable housing officers to use MaT offline when there is poor mobile coverage
  • Enable Hackney to decommission Outsystems, saving £80,000 per year
  • Enable Hackney to support, develop and deploy future improvements to MaT more quickly, at lower cost

Sprint goals:

The goals for Sprint 3 are to:

  1. Have the cloud infrastructure in place to deploy the front end and the API layer
  2. Complete the front end for ID & Residency Check and Review & Submit steps in the Tenancy and Household Check process (in React)
  3. Be able to start testing the rebuilt service with HackIT team, working from live infrastructure
  4. Have an updated pipeline so that we can deploy code
  5. Fully understand what is required to rebuild the remaining two processes (Home Check and ITV)
  6. Have a clear handover plan between dxw and HackIT (including named individuals and timeline)

Good things

We have built the Terraform cloud infrastructure. Mirela and Liudvikas have done a great job setting this up (with the help of Matt Keyworth); the final step is to test backend deployments and then we will have completed this sprint goal. There’s still some work to do to make it reusable across Hackney, which we may have time to look at in a future sprint. 

We have mapped out the journey for when housing officers are unable to enter a property. This is an addition to Tenancy and Household Check, and we are now testing this with Housing Officers.  

We had a great show and tell. We ran a show and tell (video here) with the Housing team on Tuesday, and we had great attendance and good feedback from people. We were able to demonstrate how easy it is to assemble or modify processes, once the React components have been built. 

Learned things

Collecting diversity information. There was a step in the original Tenancy and Household Check process to collect diversity information from residents, but this had been removed before the service went live. We may want to reinstate this step provided we can do so in a way that makes it clear to residents that their responses are anonymous.  

The front end handover. This has been helped by Emma being able to set aside time in early January to get to grips with how to build a process using the React components we’ve been building. However we’re still not clear how the underlying code libraries (including the task of building React components) will be handed over, and who to – the team are putting together some options to address this. 

Understanding the best ways for HackIT to use Terraform. Terraform is a DevOps product for managing digital infrastructure in the cloud. As part of Manage a Tenancy we’ve been building our knowledge of this product to understand who we can best use it as part of our CI/CD pipeline. We’ve been learning from and sharing knowledge with other technical folks in HackIT and also several of vendor partners. We’ve gained a lot of confidence and are happy that we’ll be able to gain the benefits of this not just for MaT but for many of other new services in the future.


We experienced an issue with the live service this week. A configuration in AWS was wrongly changed which impacted multiple services. Mirela was able to investigate and address this quickly, but this prevented some of the work we wanted to do on Thursday. The Terraform infrastructure that we have set up should prevent issues like this from occurring in future, once the hub is replatformed. That Terraform work might eventually be applicable to other projects in Hackney, making their infrastructure more resilient.

Building components in React is still time consuming. The decision we took last week to simplify this task has had a positive impact, but it remains the most challenging aspect of the front end work, and continues to slow our progress on rebuilding processes. We’re still exploring how we share the load with other teams working with React at Hackney.

We have a few blockers. There are a handful of activities in our backlog which have morphed from being low-priority to being blockers. These include finalising the privacy impact assessment, defining acceptance criteria for Tenancy and Household Check and finalising the HackIT approach for end-to-end testing. We discussed these as a team and have a plan for removing these blockers.  


We had a smaller team this week than we normally do. Thanks to everyone who worked on the project this week – Chris, David, F, Gill, Mirela, Richard and Tuomo.  

What’s next

Next week we will:

  • Running our third show and tell, which will take place in Stamford Hill on Tuesday 10th December at 3pm
  • Continue building Tenancy and Household Check – we’ll be focusing on the ID & Residency and Review & Submit steps, along with the start and end pages for offline syncing.
  • Scope out in more detail what is required for Homecheck and Initial/Introductory Tenancy Visit
  • Test the pipeline set up to automatically deploy code

The following people will be working on the project:

  • Bukky
  • Chris
  • David 
  • Gill
  • Liudvikas
  • Lorraine
  • Mirela 
  • Richard
  • Tuomo

Report a Problem Weeknotes : week ending: 2019-12-06

Weeknotes are a way for us to keep people informed about progress on the project.

The problem

  • 8 business areas (breaking out Markets from Parking and Estates Cleansing from Waste), approximately 60 high level problem types that can be reported in a variety of different ways 
  • A reporting approach that is structured around the service areas rather than the problem, for example if I want to report an issue of drug paraphernalia I can report it differently depending on if I am on the street, on an estate, in a park
  • Some of the online mechanisms for reporting problems fail or are not joined up to enable efficient recording / resolution 
  • Very few of the reporting mechanisms offer any explanation of the process or next steps live tracking – this generates repeat calls as the reporter doesn’t know if the item has been received, progressed, resolved, will generate no further action due to process 

Project goals

  • To enable Hackney residents and visitors to be able to use a single digital service to report a variety of issues that the Council is responsible for solving
  • For our call centre staff to be able to use the system to record issues raised by citizens via phone calls
  • For these issues to be routed to the appropriate part of the Council for actioning without the citizen needing to know the internal structure of the organisation
  • Through user research to be to demonstrate that citizens and staff are comfortable using the service
  • For the service to be integrated to our existing back-end digital services
  • Being able to report on our overall service metrics and specific information related to each integrated service area 

Sprint goals:

We aren’t sprinting yet :-). We’ll probably start doing that after the kick-off meeting (see below).

Good things

We have picked a partner to deliver this service. There has been a lot of great work by Louise Woollard who led an assessment of potential digital products to deliver this service.  We have picked FixMyStreet from mySociety and Louise and Matthew have lined up the procurement. We’re now building up to a kickoff meeting on 8/1/20 including representatives from Hackney service areas that are lining up to use the system and HackIT representatives who may need to be involved in the technical integration.

Many service areas in Hackney are lining up to use the service. The teams include the following.

  • Enforcement Officers in response to Noise and Antisocial Behaviour reports
  • Highways Engineers in response to Highways and Street Lighting reports
  • Parks Officers in response to Green Spaces, Ground Maintenance and Playground reports
  • Regulatory Services Officers in response to Commercial Noise and other Commercial reports
  • Housing Officers in response to Housing related reports
  • Environmental Operations Officers in response to Environmental reports
  • Parking Officers in response to parking / abandoned vehicle reports

Learned things

So much learning about what the council does! Some of the team have had to very quickly start to get up to speed with the many and varied service areas in the Council that the service will be integrating with.


We haven’t signed the contract yet. We don’t foresee any issue with this – we just need to get it over the line.

We’re still on the lookout for an excellent Service Manager. The remit of the service is very broad across a large number of council service areas. We need to have an enthusiastic Service Manager who has the time available to make the service a success.

We need to define service area Product Owners. Each of the service areas that are looking to use the service have a number of enthusiastic members of staff.  We need to work with them to identify who the key points of contact are.

We need to prioritse which service areas will be integrated first. Our contract with mySociety allocates us three free integrations with the Council back-end system. We’re currently developing some criteria to create an initial list of which systems will be integrated first so we can give everyone in the team realistic expectations.


We had a smaller team this week than we normally do. Thanks to everyone who worked on the project this week – Chris, David, F, Gill, Mirela, Richard.  

What’s next

In the next week we will do the following.

  • Deliver the criteria by which we’ll order the service area integrations and a draft list to discuss
  • Plan the kickoff meeting and send out invites
  • Continue the discussion around choosing a Service Manager

The following people will be working on the project:

  • David
  • Louise
  • Matthew
  • A number of people from Hackney service areas