Housing data in the cloud: Weeknote, w/c 03/02/2020

We finished our sprint on Wednesday. We have benefited significantly from working closely with colleagues from applications and infrastructure. Our main focus has been running some manual tests to see how our existing system reacted when presented with new data. We were particularly interested in: time stamps, key clusters, locking, conflicts and triggers. 

We also attempted our first data migration. We managed to move the database tables, but not the data objects. Unfortunately, the migration caused a problem in our existing system and our applications team had to reload some tables. A shout out to James and Nic who implemented the fix. We need to understand what went wrong so we can prevent it happening again.

It’s probably been our best sprint to date. Pace and energy was good, we were learning and iterating as we went along, and there was a strong understanding among the team about what we were trying to achieve. Our retrospective was characterised by honest reflection and a desire to keep on improving how we design sprints and work together. Every sprint is a test of the team’s resilience and creativity – the nature of this project is about untangling problems and encountering unknowns. 

I missed Chris, our product owner, this week. He is taking some well-earned holiday. He is unflappable and really good at explaining hard things in an accessible way. 

Our collaboration with AWS professional services starts next week! Our focus in the last couple of days has been on pulling together the information we need to support our work with AWS. This has included:

  • Tidying up our filing (definitely my weak point)
  • Sharing key documentation with AWS (a good test of relevance and understanding)
  • Summarising our needs for the sync process (based on our manual testing)
  • Putting the finishing touches on security and permission in our AWS cloud environment
  • Re-running the migration tool

We’ll be tackling our biggest unknown first – can we sync back data from the cloud into our existing system. We need to do this in a way that doesn’t disrupt availability and performance. I hope we will learn fast enough to provide an update next week!

It was great to meet colleagues from Camden Council on Thursday and to talk about this project in more detail. They are as keen to know the outcomes as we are. We know that other councils are experiencing similar challenges with legacy systems. It was a timely reminder that working in the open is vital to unlocking shared problems and accelerating change.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.