Virtual Council meetings, weeknote: w/c 30.03.2020

Legislation has been passed that allows Councils to hold their statutory meetings online during the COVID-19 pandemic. We are getting our heads around the implications of this and how we can implement something quickly which will fulfill our legal obligations.

It’s a genuine opportunity to do something differently. There’s been a lot of enthusiasm from our Council members and really thoughtful engagement on issues such digital inclusion. 

This week we have:

  • Identified the key user needs
  • Started drafting guidance for meeting participants
  • Drafted amendments to our Constitution in light of the new regulations

Our first virtual Council meeting is scheduled for 14th April – there’s nothing like a hard stop to focus attention. Next week we will be:

  • Agree on a tech solution to test based on a review of user needs
  • Reviewing what devices Council members are using to participate in virtual meetings
  • Testing our guidance – is it fit for purpose, do users understand it?
  • Running a test meeting with volunteers

It is great to be working with colleagues from different teams, in particular Governance and Legal. We are using a GoogleChat room for asynchronous discussion (keeping this out of emails where possible) and Trello to plan and deliver our actions. These collaborative tools are new to some the team. Everyone has got stuck in and we’re learning by doing. 

I’m really grateful for the fast (and thoughtful) work from LOTI and MHCLG Local Digital Team on this. Much of their advice we can copy and paste which is incredibly useful. 

We’re keen to talk to other Councils about what they are doing. Please get in touch via LocalGovDigital Slack channel or using the comments section. 

Housing Data in the Cloud: weeknote, w/c 09.03.2020

Movie characters minions backing out of a door and closing it.
It’s time to say goodbye

It’s not adieu, but au revoir.  

We’ve almost come to the end of Housing Data in the Cloud in its current guise. 

We know enough from our proof of concept to stop. What does stopping mean? This week, we have:

  • Pulled together key learnings
  • Reviewed our findings against the wider context
  • Made recommendations as to what should happen next
  • Proposed next steps

It’s hard to stop. It’s hard to disband a team of people who have forged strong working relationships. It’s hard to stop when we can see all the exciting possibilities this work offers. It’s hard to stop when we feel like we are just getting started. It’s hard to give up the responsibility for something when we’ve invested so much in its creation. 

But it is the right time to pause, collect our thoughts and determine what might happen next. 

In our show and tell this week, we talked about what we are proud of and what we have learnt during this phase. Also, the team was brave enough to reflect openly on what we could have done better. With such an experimental project, we failed quite regularly. And on failure we discovered it teaches us more than success – doesn’t make it any less uncomfortable or frustrating at the time though. 

Our parting gift is our sync-back model – co-designed with AWS professional services. We would love your feedback or questions. Fresh eyes will help us iterate it in future phases of work. Please pop ideas into the comments section.

Housing Data in the Cloud: Weeknote, w/c 24.02.2020

It’s been a quiet week on the project. Chelsea and Simon from AWS are coming on Monday to help us set up the sync in our cloud environment. After that, we can start running some small tests with our on prem database. The main focus this week has been devising those tests. 

We are looking at two things:

  • What happens on a “round trip” (on prem database -> AWS Postgres database -> changed record -> sync process back to on prem database)
  • What happens when records are added in a batch

I’m thinking about how to finish this phase of work well. Any tips on this would be particularly welcome in the comments section. I want our documentation to be as good as it can be and accessible for other teams. We’ve been doing this as we’ve gone along and my gut feeling is we are in a good place with this. 

Other things I am mulling over:

  • I want the team to have a strong sense of everything they have accomplished
  • Have a sense of ownership over the outcomes and be able to proactively shape the recommendations for “what next”
  • To celebrate the amount that we have learnt
  • To assess how we’ve worked together as multidisciplinary team and how we’ve collaborated with others
  • To reflect how we have communicated and engaged with others about our work

Housing Data in the Cloud: Weeknote, w/c 17.02.2020

Man playing invisible drums.
Drum roll please….

Chelsea and Simon from AWS have been working on a proof of concept sync-back – a key part of the hypothesis we’ve been trying to validate over the last couple of months. It’s been our biggest unknown and hence our biggest risk. 

Drum roll please….

The proof of concept has demonstrated that we can sync data from a cloud Postgres database back to our on premises SQL Server database. This is good news. The basic concept involves capturing the queries on our cloud database and queuing them to be replayed in our on prem database on a frequency of our choosing. The design is modular so we’re not relying on a single native tool to do all the heavy lifting.

So, we have taken a big leap forward in our understanding of what’s possible. The input from the AWS professional services team has been great. To move from the realm of the theoretical to the actual, we have to ask is it practicable? 

Steve, our data engineer, pointed me in the direction of Richard Feynman’s famous quote: “The rate of the development of science is not the rate at which you make observations alone but, much more important, the rate at which you create new things to test.”

With this in mind, our next step is to get the proof of concept cloud template set up in our AWS environment so our in-house team can run some tests with it. We need to explore:

  • How will our on prem SQL database behave when the sync process is applied?
  • Does this solution lend itself better to some types of data better than others?
  • What’s the best way to handle conflict resolution?
  • What happens when we increase the number of simultaneous queries?
  • How do we capture information generated after a record insert or update? (Triggers are proving fun)

In a (not so) small personal triumph, I’ve finally managed to coordinate team diaries and we have a show and tell booked in on 10th March. We’ve not been great at doing these in these phase of the project. This will be an overdue opportunity to share our progress with a wider group of colleagues.

Housing Data in the Cloud: weeknote, w/c 10.02.2020

Dog attempting to get a large stick through a small gap.
Big stick, narrow bridge. Problem solved.

Three things you need to know about housing data in the cloud this week.

Our work with AWS

We spent the day with AWS professional services on Monday exploring what our “sync-back” might look like. We have a proof of concept to test and expect initial findings next week. 

Hard work pays off

AWS were really positive about the cloud environment we’ve set up. The team has worked hard to get this in good shape. We’re relatively new to AWS cloud services so the learning curve still feels steep. The validation confirms that our skills and expertise is growing and we can feel confident about our in-house capabilities. Collaborating with the infrastructure team has been key to this. Knowledge shared is capacity strengthened. 

Keeping an open mind

The sync back proof of concept has spawned a fresh rush of creativity from the team. It’s offered an unexpected springboard to think more widely around some “how might we” type iterations and alternatives. The proof of concept offers some structure we can coalesce around. Perhaps the relatively safety of this (a convergence) releases the team to think more divergently again. Pop psychology aside, it is another timely reminder that working iteratively and (potentially) failing fast in this kind of problem space has significant benefits over incremental design.