Platform API weeknote, 20 May 2020

What is it?

The platform API project is part of an effort to separate the end-user interfaces, business logic and data of the various databases used by Hackney Council to deliver services to residents. 

What have we done?

The goal for this sprint was to complete the first iteration of the Mosaic resident contact information platform API and to set up the data migration service. In line with the philosophy of “fail fast”, we picked this one deliberately as the data source was complex, but not too complex. It would also be a good test of our data migration strategy; our options were CDC or DMS with MS replication. CDC failed with our data source but tests with DMS with MS replication were promising. MS replication takes advantage of the inbuilt SQL replication feature and could be used because Mosaic’s tables have primary keys. 

We started the week finishing off the essential tasks to build our “foundations”, things that will be of value to all APIs such as our deployment pipelines and Terraform scripts. This is part of making everything we do easily reusable. We also planned the following:

  • Create the repo for the Mosaic data source
  • Set up the database in the cloud for the migrated data
  • Set up the infrastructure needed for the service
  • Create the API with two GET endpoints – one for listing results, and one for viewing records

All these job stories have been templated for future reuse and also for a consistent framework to be followed, as the work required for the other data sources will be pretty much the same. 

Also this week we started work on our roadmap. We probably should have done this earlier but it was picked up in our first retro as a way to bring clarity to what we are doing, and when we’ll be doing it. After Mosaic, we will do Academy and Universal Housing; then in the next quarter we’ll tackle Qlik first. Work on the Academy has started, as its database is unusual in Hackney, and a solid understanding of how it works and is replicated is needed before we can do anything with it. 

We didn’t quite manage to finish the Mosaic tasks – another day is needed – but we’ve learned a lot. The next sprint will be shorter due to the bank holiday, and we’ve planned accordingly. We’re still finding our pace, but we’re almost there. 

+ posts

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.