Automated Testing weeknotes w/c 17.02.20


This week we have kicked off a new project as part of the DevOps programme looking at automated testing. The idea came from some of the work we did on strengthening support on the website before Christmas, where several members of the team were exposed to Cypress as a tool for writing automated tests. 

These team members identified this type of tool as something that could be used in their current work, which involves both them and colleagues in services manually testing our web based applications. This is an area where we can use modern automated testing tools to save people considerable time and allow HackIT to arrange deployments more regularly. 

This project will last 8 weeks, the idea being that we give the team a broad introduction to what tools are available and get them to start writing automated tests in that time. This will allow us to get a sense of where automated tests can quickly deliver value for HackIT and where we might need to carry out further work. 

The team for this automated testing project is: 

  • Rasit Arif
  • Richard Austin
  • Erdem Cankiran
  • Sharon Comery
  • Hamid Hazrati

All of the team except for Erdem and Hamid are in Applications teams. Erdem is an infrastructure apprentice. Hamid is a front-end developer from Digi2al who will be supporting us to develop our automated testing skills. 

What have we been doing so far?

  • The first week has been a bit of a sprint 0, designed to get everyone up to speed on the key concepts and start to get to know each other as a team. It has also been a chance for us to build out the product backlog. 
  • Hamid gave the team an overview of various automated testing options: Selenium IDE, Cucumber and Cypress. These tools all have different levels of complexity and abstraction. Selenium IDE is the most straightforward to get started with and install on our machines. It is what we are planning to start off with to build the team’s understanding of automated testing and its possibilities. 
  • We also had an introductory session on agile working. Most of the team have not worked on agile projects before, however working in HackIT they have picked up most of the key concepts and terminology. We went through the key ideas and gave the team a chance to ask any questions they wanted. 

What next?

  • Now that everyone has their feet under the table, an idea of the different types of tooling and we have a backlog we are ready to plan our first sprint, which we kick off this week. 
  • In the first week we also identified that the team’s current machines are unable to install certain tools so this week we will get the team access to machines that they need to install the right tools.

Getting Back In The Groove

Week notes Network Redesign 10/01/2020

Background & Recap

Let me start by welcoming everyone back after the Christmas break. I hope you were all able to get some rest and quality family time. It’s always a bit challenging getting back into the groove of things after some time off. However, there’s no time like the present so let’s get things underway.

For my new readers, the aim of the ‘Network Redesign’ is to enhance and future proof of the council network infrastructure. This will happen through the architectural redesign of the current council network and upgrading of associated hardware, (network switches & cables).

The new infrastructure design will have a ‘Web First’ strategy at its core and enable users (Council employees and members of the public to access council online services, from any of our offices without being bound to a local server but connecting directly to the cloud.

Last Sprint Goal

  • Complete cost estimates for potential firewall solution for the first quarter of 2020 (This has partially been achieved however due to sick leave elements of this task have rolled over into the upcoming Sprint)

Objectives for 2020 first quarter

Since the beginning of this project, I have had to up-skill my understanding of the different components which go into Wi-Fi installation and infrastructure redesign. From broadband cable installation, ‘Wayleave’ licensing to hardware purchasing of switches, to understanding the difference between these types of switches and their functions between a distribution and core levels. (See more on Core & Access Layer switches with this useful one-pager here)

The objective of our first quarter in 2020 will be to get the team and stakeholders for our project in a world where their understanding of our teams recommended solutions provide enough insight that they feel comfortable enough in their understanding to be able to order the relevant hardware solution fit for purpose and to meet a majority of our briefed requirements.

Sprint Goals

  • This current Sprint will see the team take a deeper dive into the requirements for purchasing a firewall for the data center and the distribution level.
  • Set up a Firewall test lab and review the results before submitting recommendations
  • Now that we have the ‘Wayleaves’ license approvals we are now doing the actual installation of the fiber optic cables for our identified sites.

Re-engineering Hackney content weeknotes 14/01/20

Site Improve screenshot
Site Improve screenshot

This project continues to throw up a multitude of potential avenues to explore. As part of the DevOps trio of projects at the end of last year (excellently DM-d by Felix), we cleared a load of outstanding bugs that had been, well, bugging us. Plus, the proto-support process that evolved out of the DevOps experiment has empowered our infrastructure and app support teams to respond with gusto if the site has a wobble. We’ll confirm it’s working if the ticket I just dropped in the support desk ends up in the right hands! 

We weren’t able to settle all of our irks in DevOps but, in the spirit of continuous improvement, we continue to push the bar where resources allow. The commercial properties service now has a more visual approach, akin to its real estate competitors. And the road safety pages have benefited from an audience-centric restructure. We’ve run a few workshops with services to promote the concept of user-centred design and, if your service would like help with this, give us a shout. We’ll also try and come up with workshop templates so you could run a session yourselves. 

Some services require more of a marketing focus than others. One example of this is Hackney Museum. We’re working with Niti and her team to create a museum website that holds its own against other museums’ and tourist attractions’ online offer. To achieve this – with minimal additional effort – our front-end developer Emma has created a Hackney WordPress theme using the WordPress CMS as the back-end and the HTML components built for the Intranet for the front-end. This equips us with a wider suite of components from which to devise a visual, enticing design that will appeal to visitors.

We can re-use the WordPress theme over and over again, where needs require. And, in the spirit of open source, we’re seeking to submit it to the WordPress theme repository so other organisations can repurpose it at no cost.

Some of the third party sites to which we link continue to use ancient templates, which disjoints the user experience. Working through a schedule of updating them is a project in itself, however, we now have the UI toolkit to aid suppliers. Museum Collections and News are up next for reskinning. New suppliers should be given the toolkit during onboarding (with requirements included in contracts) and the design team, led by Joanne, can assist with this. 

The Comms team continues to embed Site Improve into its business-as-usual routine to great effect. Thanks to Iain and Alan’s sterling efforts, we are pleased to report that all four metrics of quality assurance, accessibility, search engine optimisation and digital certainty have surpassed the government benchmark. 

We’ve put Hotjar back on the site and the DevOps blitz has increased from 55% to 65% the percentage of users scoring the site at least 4 out of 7 stars. The 35% less than happy with the current sitch are often complaining about embedded apps not working; and we’re looking at how we can feed this back to services to fix. On that note, Behrooz Mirmolavi will be flying in shortly from GLA to talk about his role as a website data analyst, especially around Google Analytics and Hotjar. All Hackney staff are invited so get your diaries prepped for 10am on 30th January. Save your spot or miss out!*

*(You don’t actually have to save your spot, just turn up)

Strengthening support on the Hackney website – Weeknotes week 3 w/c 09/12/19

What’s happened this week?

  • Rearchitecting the back-end: Dan has completed the rearchitecting of the backend for our staging and test environments. We are almost ready to update this for production. This will mean that we can go from having to run builds one at a time to a situation where we can run up to 5 concurrent builds. Omar has been working closely with Dan on this and getting his teeth into Terraform.
  • Navigation: Luca has completed the implementation of the website’s navigation, which should make it much easier when users are trying to move between different pages on the website.
  • Preparation for the end of the project: the team have started drawing up documentation and runbooks for supporting the website after Dan and Luca leave.

Challenges this week

  • The team’s time: it has continued to be difficult to get as much of the team’s time as we would like. This is partly because we have been unlucky and people have been off sick, however it is also a problem of people having multiple projects and existing commitments. We have tried to address some of these issues by coordinating with other managers and projects. We will see next week if it is effective.
  • The steep learning curve: members of the team have said that they find the learning curve really steep and that they aren’t sure how much of it they fully understand. This is expected, given that most of them were completely new to the principles of software development and DevOps when they started. By having the opportunity to practically try things out the team will hopefully be able to see how much they’ve learnt.

What’s next?

Next week is the final week of this phase of the DevOps programme. It will involve:

  • An end of phase show and tell
  • A hackathon to test how much the team have learnt
  • A phased handover of the DevOps knowledge that the HackIT teams need on the website
  • Continuing with refactoring the front end and providing learning opportunities for the team

What’s happened already?:

Weeknotes week 1 – w/c 25-11-19:

Show and tell week 2 – w/c 01-12-19 

(Over the next few weeknotes I will be experimenting with using different styles, please let me know if you have any feedback).

Strengthening support on the Hackney website – Weeknotes W/C 25/11/19

This third phase of the DevOps programme is focused on strengthening support on the Hackney website. We are using DevOps tools and approaches to make the website more reliable and sustainable. It is also testing one of the hypotheses from our initial discovery:

“Developers working alongside people with ‘ops’ (i.e. infrastructure / applications support) skills will increase velocity, resilience, security and stability”

This project will run between 25th November and the 20th December. 

Previously we carried out a discovery; and launched an initial alpha focused on cloud procurement and a DevOps pipeline approach.

The team

The team is made up of people from the applications, infrastructure and delivery teams as well as a DevOps engineer, Dan, and a Front-End developer, Luca, from Digi2al (who we have been working with on the DevOps programme). 

This is the first time we have applications people in our team, and it is a great opportunity to see how closely apps skills align with DevOps skills. Equally, for some of the team, this is the first time that they have worked on a development project using agile methodologies. As a result of all this, one of our key focuses is learning and development.

For the first time during the DevOps programme we have a team working on this almost full time, which is great as it allows us to more easily maintain focus and build momentum.

What we are doing?

This week we have been getting up to speed with the underlying technology of the website. Luca and Dan have been diving in at the deep end and quickly getting up to speed with the codebase. The rest of the team has been getting familiar with the technology and terms; and shadowing the more experienced members of the team. 

We have to strike a balance in this project between outputs and knowledge transfer. One week into this four week project, the HackIT team is primarily learning. While the  Digi2al duo has been identifying areas where the team can up-skill, providing introductory readings and running tech workshops.

There is an existing backlog from the project that delivered the website, which included a combination of bugs and small features. Product Owner on this project, Susan, has been looking through the backlog with Dan and Luca to identify tasks that add the most value and also make the website more maintainable and supportable in the future. 

This has resulted in an early focus on two areas: search and WordPress. Luca’s refactoring and simplifying of search will greatly improve the quality of search on the website and make future improvements easier. 

Secondly, Dan is working with Omar from Infrastructure to make the WordPress backend more reliable. It has a tendency to fall over during simultaneous deployments, requiring a manual restart. By restructuring the AWS architecture upon which WordPress sits, we should be able to save our content publishers  time and ensure the site is updated readily for our residents. 

What’s next?

We will agree on areas that our HackIT team members can look at over the next three weeks that will add value and broaden their learning. They are also continuing to shadow the Digi2al members of the team. 

Luca and Dan are working through the search and WordPress re-architecting, respectively, in addition to picking up minor bugs and code-reviewing other work that is taking place on the website. They are also running tech workshops and supporting the learning of the rest of the team by suggesting material to learn and exercises to try.