Modern Tools for Housing – Programme Week Notes 28/01/22

Somehow this week has flown over despite things going well across the board. I’ve been struggling to get time to work on longer-term things vs. the large pile of “10 minute jobs” that end up filling the day. I’ve got a few things moving (see below) but need to try harder next week. Meanwhile, our workstreams are continuing to do great work.

Finance – Silvia – HackIT workstream DM

  • Celebrations:
    • Two week sprint cycles commenced on 24th January 2022
    • Finance and Manage Arrears separated into individual workstreams
    • New members of the team welcome Furkan, Hugo, Faisal and Kate
    • Data Migration document approved and pleased that there is no impact on Manage My Home once Data Migration is complete
  • Challenges:
    • Delays with data migration documentation approval

Manage My Home – Yvonne – HackIT workstream DM

  • Celebrations:
    • First of new team members in Service Design and Business Analysis from Nudge on boarded!
  • Challenges:
    • New team getting used to the technical side as well as how we work. 
    • Being able to give timeframes yet on when we can deliver certain things (first process, worktray) 

Repairs – Sarah – HackIT workstream DM

  • Celebrations:
    • Demo of a possible future solution for online repairs that looked very promising
    • Design team synthesising the follow on research ready for review next week!

Challenges:

  • Repairs Hub doesn’t currently support multiple trade work orders, which is becoming a high priority problem that needs solving
  • Some operatives reported their bonus reports are incorrect due some data going into overtime instead of their bonus – we know about the problem and are awaiting managers to validate data so that we can fix it directly in our database and share the correct reports for the people affected 
  • Making changes to stories that were in the current sprint, we’ve moved out the ‘status updates’ epic and bought in some other mobile improvements where the requirements were clear and understood.

All our teams are going great guns this week. From Managed Arrears and Finance building up towards regular release cycles, Repairs working in the depot to onboard more operatives to folks from Nudge filling out the Manage My Home team so they can start sprinting again soon.

We had a demo from the team behind the open source, DLHUC funded Repairs Online. This is a resident facing application for raising repairs that we are looking into potentially introducing at Hackney. It looks like a good product but we need to do a ROI investigation to see if it’s a good fit for us.

I’m continuing to work closely with Nudge on the “soft reboot” of the programme. We’re discussing new Service Designer and Business Analyst roles to be able to address a lot of the medium-terms issues that we’ve not had the capacity to get to before. We’re also organising a “ways of working” workshop in a couple of weeks where as a programme we’re going to collaboratively define how we’re going to deliver our work for the next six months (including regular retros to improve it along the way).

As part of that we’re keen to iterate on our regular steering group meetings. To that end I’ve shared a set of options for the one we’re having next Friday and I’m looking forward to receiving feedback from everyone. In the future I’m suggesting that we might try an even more radical option and replace the agenda with lean coffee.

Finally, we didn’t get a date in the diary for the first Blue Sky Day in the calendar this week but we’re having a quick chat on Monday afternoon to check a few things and I expect invites to go out right after that.

As someone once said – Onwards! 🙂

Modern Tools for Housing – Programme Week Notes 21/01/22

Modern Tools for Housing – Programme Week Notes 21/01/22

A short one this time as it’s already a few days late and I’m keen to get it out the door. I was distracted by GovCamp for a couple of days (all session notes available here) but now it’s back into the swing of things. News from our workstreams.

Finance – Silvia – HackIT workstream DM

  • Celebrations:
    • Great progress has been made in our Programme Strategy and Planning meetings, with the Amido handover arranged for next week. 
    • PEN testing started at the beginning of this week
  • Challenges:
    • Additional effort is needed to ensure that all solutions (APIs and frontend) are adequately documented.  This will aid in sharing understanding of architecture, implementation, and approaches to problems needed to be solved.

Manage My Home – Yvonne – HackIT workstream DM

  • Celebrations:
    • Handover between Amido and Nudge
  • Challenges:
    • Lack of clarity in what roles we will have in the team. 
    • Lack of clarity on when those roles will be available to star. 

Repairs – Sarah – HackIT workstream DM

  • Celebrations:
    • Mobile onboarding continues to go well
    • Initial discussions regarding syncing voids into RH have kicked off to a positive start
    • Resident interviews to support our follow on research
    • Challenges:
    • Bugs causing stress with users, and tensions with the team.
    • Uncertainty on the shape of the new team

The biggest piece of news this week is that we’ve split our existing Finance team into separate workstreams for the new Housing Finance System and Managed Arreats. We’ve done this to reduce the amount of context switching for team members and to channel our focus into reestablished our Agile sprints so that we can manage our mixture of defect fixing and new feature work.

We’ve agreed the shape of the new team for Manage My Home and team members should start being added this week.

We have a meeting coming up soon to agree on the shape of the long-running HackIT-staff-only Repairs product team. Once that’s done we can start moving into the recruiting process (aided by our agency partner Nudge).

There’s a lot of discussion about reshaping the programme and confirming our ways of working now we’re moving from three agency partners to one and how that can fit into the long-term HackIT 3.0 transformation. Hopefully I’ll be able to report more on that soon.

The oft-mentioned in here but not-yet-done Blue Sky Days designed to refocus the programme from replacing Universal Housing to what we can really do with our new broad service should have the first session in the calendar by the end of the week.

So – nothing on fire at the moment but, as ever, much going on including lots of new starters joining the team with what, I’m sure, loads of great ideas on how we can improve things.

Modern Tools for Housing – Programme Week Notes 14/01/22

Our first full week back following the post-holidays firebreak and all the teams are getting back into full-stream working. The latest updates from them below.

Finance – Silvia – HackIT workstream DM

  • Celebrations:
    • Great progress has been made in our Programme Strategy and Planning meetings, with the Amido handover arranged for next week. 
    • PEN testing started at the beginning of this week
  • Challenges:
    • Additional effort is needed to ensure that all solutions (APIs and frontend) are adequately documented.  This will aid in sharing understanding of architecture, implementation, and approaches to problems needed to be solved.

Manage My Home – Yvonne – HackIT workstream DM

  • Celebrations:
    • Becoming clearer what team will look like with Nudge
  • Challenges:
    • Misconceptions and misunderstanding around data in the Person, Property and tenure Microservices and how other Housing services can access this data. 

Repairs – Sarah – HackIT workstream DM

  • Celebrations:
    • Onboarding for Mobile Working has kicked off to a very good start, with a good proportion of gas engineers and electricians now mobile, with the carpenters next up.
    • We’re no longer reading from the outdated API’s which were a snapshot of the pre cyber attack data. All property and tenure information is reading from the core data microservices, with contact names and numbers our next priority.
    • Challenges:
    • We’ve started the conversation about how the repairs team is going to evolve into a product team. There is inevitably going to be a period of time where we’re not going to be able to deliver value to users as quickly as we would like whilst we onboard new team members.
    • Devices.  We have been very fortunate to have been given new phones for the DLO reactive plumbers and drainage, but there are still many more trades to onboard

With the move to a single agency assisting HackIT across the whole of the programme it definitely feels like we’re in a period of transition. This is likely to last a little while as new folks come into the team, both to re-start MMH and at the programme-level, and we continue to rethink and discuss our overall shape and governance practices. This is combined with the overall current uncertainty around the impact of the “HackIT 3.0” department reorganisation. As always though the whole team is taking this as an opportunity to improve and is suggesting lots of great ways we can move forward.

Two of the things we’re discussing at the programme level are the introduction of some Programme Principles and / or a Programme Charter (based on the concept of a Team Charter).

For our governance we have a lot of things in the mix but we know we definitely want to revisit our roles and responsibilities definitions, particularly our Product Owners, and reexamine the purpose and execution of our Steering Group sessions which currently mostly act as reviews of recent work rather than forums addressing our difficult outstanding issues.

The good news is that I finally completed my self-imposed task to write up a full “state of the nation” document for MTFH. The bad news is that it came out to a behemoth of 14 pages which is too much for anyone in the team to need to digest. Instead I’ve almost finished pulling out the salient points into a draft public programme backlog (previously I just kept track of things in my personal one) which I’ll be circulating very soon. It also has a lot of items in it but I hope the way I’ve set it up will mean that it’s easy for us to slice it a number of ways including for highest priority things to cover in our new daily programme stand-ups, but also to have a new separate regular session to look at items that could delivery huge value in the long term but are unlikely to be started now as their benefits aren’t immediate.

There are many such items but one that I’m definitely keen to spend some time on is looking into how we can best support the new Link Work team that HackIT has put together. This is a multidisciplinary team from across our council service areas that is proactively supporting our most vulnerable residents.

As ever, a huge amount to do – both in the short term, but also making an effort on how we can help our staff and deliver excellent support for our residents by introducing new functionality that we’ve never seen before.

Modern Tools for Housing – Programme Week Notes 7/01/22

It’s our first week back after everyone’s had a very well deserved time away. We’re still on firebreak where the teams are easing back into delivering at pace. We also have folks still on holiday or off with covid so it’s a little slower in general at the moment. Back to normal next week.

Finance – Silvia – HackIT workstream DM

  • Next Show and Tell: 18th January 2022
  • Celebrations:
    • Data Migration Plan – Our Data Migration plan has been shared with Rashmi / Mirela detailing what steps we plan to take, what entities would the migration affect, what APIs are to be called and the changes that need to be made. The Charges data migration coding work was completed, coding began for the Accounts API, with the other APIs to follow next week.
    • Good progress has been made on the Leasehold Estimate Calculations work, with follow up meetings arranged next week with members of the Finance and Leasehold Service Team to answer some remaining queries.
  • Challenges:
    • We experienced some technical challenges this week regarding providing access to the required application and the creation of accounts for the Pen testers. These issues have now been resolved and the Pen Testing will start on Monday 13th January.
    • Data Migration will take place after the PEN testing.

Manage My Home – Yvonne – HackIT workstream DM

  • Celebrations:
    • Data Platform work 
  • Challenges:
    • Very small team

Repairs – Sarah – HackIT workstream DM

  • Celebrations:
    • Deployed an urgent bug fix for the operative split issue.  This applies to jobs raised since the deployment.
    • Started communications with gas engineers and electricians in preparation to onboard them from next week.
    • Challenges:
    • Lack of a team presence in the depot for onboarding is going to be quite a struggle but we’re thinking best about how to support operatives remotely as well as working with supervisors who will be in the office
    • Lacking a clear goal this sprint, lots of things to do with bonus, but less understood criteria on some of the other things

Bit of a minimal update this week as we’re still on firebreak. There’s lots of follow-up discussions after we concluded the new agency contract to continue to assist us in the delivery of the programme. Our first priority is to stand up a new team to deliver Manage My Home now that the previous agency has rolled off. At the same time we’re investigating how we can start planning how to transform the Repairs workstream into a product team – including our related recruiting strategy. Finally, we’re thinking about the overall structure of the programme itself and whether it’s set up to best deliver what we need for Hackney Housing in 2022.

I’m going to leave it at that for now – back with a lot more detail next week.

Housing Register Service Assessment 7/10/21

Housing Register Service Assessment 7/10/21

This document summarises the outcome of the Service Assessment that was undertaken on the Hackney Housing Register project in October 2021. The purpose of the service assessment was to determine whether the design and implementation of the Housing Register service met GDS Service Design standards, but also to provide specific feedback to the project team and to enable wider lessons learned to be shared with the rest of HackIT.

This summary does not include the evidence that was presented to the assessors by the team, as that can be found in the Trello board that was used to conduct the assessment.

This service assessment was carried out remotely, and we are grateful for the work done by both the project team and assessment panel in advance of the assessment meeting.

PanelTeam members
Cate McLaurin : Panel LeadMaya Knight: FG Delivery Manager
Darren McCormac: Delivery Management assessorBritt Wood: FG Principal Consultant
Richard Smith : User Researcher / Service Designer assessorHannah Mills: FG
Dan Harper-Wain: Product Management assessor (external)Ottla Arrigoni:FG
David Dean: Technical assessorChris Wensley: FG
Thomas Morris: FG
Darren Aitcheson: HackIT Delivery Manager

Related documents / links

Service vision

The vision for the new housing register is for people wanting to join the register to understand the full range of options available to them and their likelihood. It is simple to join for people who qualify, minimises failure demand, is easy to administer, sufficiently open and gives all stakeholders confidence in the fairness of the process. The underlying applications are secure, reliable and adaptable to the changing needs of users.

Summary of learning

  • The service and the team has generally met the standards required, with some particularly good practice that could be useful for other services to learn from or adopt.
  • The main concern of the assessors is regarding what will happen to the service once the development team rolls off and hands it over to Hackney to run and improve; it was unclear at the time of the assessment if there was an agreed plan for this. 
  • The team has done a good job of user research and has not just rushed into developing the system before fully understanding the needs of service users. However, it is important that an accessibility audit is conducted as soon as possible.
  • Whilst the team generally worked well in an iterative manner, and involved the LBH staff by embedding them within the team, it was quite late in the timeline before the first usable version of the service was delivered. It appears that LBH placed a fixed “go-live” date of October and a relatively fixed scope on the work at the beginning, which wasn’t particularly conducive to iterative development and Agile ways of working. This is something that may need to be addressed by LBH for future projects/products/services.
  • The time allocated for the service assessment itself was not sufficient, which led to some of the later standards feeling a little rushed. A minimum of 2 hours, and preferably more, should be set aside for service assessments.

Service Standard points in detail

1. Understand users and their needs (Assessor: Richard)

Develop a deep understanding of users and the problem you’re trying to solve for them.

Look at the full context to understand what the user is trying to achieve, not just the part where they have to interact with government.

Assessor comments

  • The team were able to demonstrate using a wide range of research methodologies to understand user needs and conduct usability research on the product.
  • Using potential product concepts (such as potentially adding nudge techniques as part of an application to sell the benefits and support of sourcing alternative private accommodation) at a really early stage as part of the discovery research teased out additional needs and behaviours quickly in an innovative way.
  • The team were also able to articulate where features had been iterated within the product due to the understanding they gathered from the usability research, including not using text messaging to inform residents of their place within the waiting list – a finding that would also back up previous research from other projects that considered using text messaging.
  • Whilst not recorded explicitly in the research plan, the team were able to robustly articulate the work they did to make sure research was conducted in an ethical way; especially considering the potential participation risks of residents on the housing register and any perceived benefits it may bring towards their application.

Assessor score: Partially met

Recommendations

  • Apart from a document detailing some demographic information of housing register applicants, there was anecdotal detail within the materials provided and during the assessment about telling the story of the resident – more about their motivations, life events that prompt joining the register, more about the varied barriers they experience to build up empathy with the user.
  • This would be extremely valuable, especially to those who aren’t as familiar with the project – such as those who might access research artifacts from the project via the User Research Library to which a team member should add their work so others can learn from it.

2. Solve a whole problem for users (Assessors: Dan & Richard)

Work towards creating a service that solves one whole problem for users, collaborating across organisational boundaries where necessary.

Assessor comments

  • There was good evidence to show the team took a holistic approach to scoping and designing the service. Whilst the original brief was to redesign the housing register form, the team unpicked this to explore the whole journey of accessing social housing (inc. alternative options), by mapping out the end to end service and analysing dropout/appeal data.
  • It was clear how the components created by the team fit into the wider HackIT housing vision, including the focus on setting clear expectations about the wait time, and advising users to access alternatives.
  • Perhaps due to the initial scope constraints, it wasn’t entirely clear how the overall service would develop over time, to ensure a coherent end-to-end experience for residents. The team articulated a number of areas they were keen to see improved, such as joining the application process up with the separate account users require to bid for properties, and keeping in touch with residents during the waiting period. However, there wasn’t a clear roadmap to indicate where these ideas might lead, or the level of resourcing they would require. 

Assessor score: Partially met

Recommendations

  • If it hasn’t already been explored, thought should also be given to how to maintain the quality of the register data during the waiting period, given that significant changes of circumstance are likely to happen to applicants during this time, that may change their priority or remove them from the register entirely.
  • It also wasn’t clear how much ownership the team had over the service landing page (or how they worked with others in Hackney, if it is owned elsewhere). For example, the ‘Join the register’ link on the orientation page appears to be broken at present. If it hasn’t already been done, the team should review the consistency and usability of these pages, in the context of the end-to-end journey of signing up to the register.

3. Provide a joined up experience across all channels (Assessors: Dan & Richard)

Work towards creating a service that meets users’ needs across all channels, including online, phone, paper and face to face.

Assessor comments

  • The service has been designed to operate effectively across a range of channels. The team have prioritised ease of contact for residents who struggle with the online service, with clear phone numbers and the use of a unique reference number to manage the transition between channels.
  • Staff are able to view residents’ applications, and act on their behalf (for example where residents aren’t able to use the online service at all).
  • There is also strong evidence for the decision the team took, about preferring email channels over SMS.
  • Staff have been closely involved in the development and prioritisation of the work – both through the service owner and housing register manager being embedded in the team, as well as by bringing wider staff into the decision making process.

Assessor score: Met

Recommendations

  • If it hasn’t already been explored, there is an opportunity during beta to test that the hand-off between the ‘apply’ step and the ‘provide evidence’ step works effectively in the online channel. These two steps currently appear as separate services, with an email linking residents to the second step, and could lead to drop-out.
  • Conduct additional usability research with agents whilst they are supporting residents through the system to check the tool meets their needs in a live environment.

4. Make the service simple to use (Assessors: Dan & Richard)

Build a service that’s simple, intuitive and comprehensible. And test it with users to make sure it works for them.

Assessor comments

  • The team conducted a significant amount of usability testing for the MVP, particularly given the tight project timescales. The team said they tested with a wide range of users and across tablets, mobiles and desktop devices.
  • They were able to articulate improvements that were made throughout testing to help users succeed first time, as well as areas that users still struggled with, such as understanding the ‘task list’ pattern and providing information about household composition. Whilst the latter two issues are yet to be resolved, the team have clear plans in place to address them.
  • The service also has a clear mechanism on each page for residents to seek help if they get stuck, and analytics will be in place to monitor any areas of difficulty they encounter.
  • The team struck a good balance between usability and counter-fraud measure, with regard to the risk of residents submitting multiple applications to ‘game’ the service. Rather than barring repeat applications (which could prevent users who had a genuine change of circumstances from accessing the service), they opted to solve these issues through a back-end application matching capability.
  • On the staff side, it was clear that the team had a good understanding of the needs of officers, particularly to be able to understand the status of a case at a glance and the available evidence showed these needs were met.

Assessor score: Met

Recommendations:

  • It will be important to monitor carefully for signs of further usability issues now the service is live, given the limited time available during alpha.
  • The team should ensure that the high level of contextual knowledge about the users of the service, and their needs, are handed over well to the HackIT team that takes on responsibility for the service, so that this context is applied to future iterations.

5. Make sure everyone can use the service (Assessors: Dan & Richard)

Provide a service that everyone can use, including people with disabilities or other legally protected characteristics. And people who don’t have access to the internet or lack the skills or confidence to use it.

Assessor comments

  • The team showed strong evidence of researching and usability testing across a wide range of accessibility needs. This included various visual impairments and neurodiverse conditions, as well as users with low levels of English literacy. It was clear that the findings of this research had actively shaped the design of the service, such as reducing the complexity of the content.
  • One of the most impressive examples was designing for non-binary users; here, the team’s work went beyond just content design, to re-working the service’s policy for calculating bedroom entitlement.

Assessor score: Not met

Recommendations

  • Further actions are needed to ensure the service is accessible and meets the relevant legislation. An accessibility audit is yet to be conducted. Accessibility issues are therefore likely to still be present in the live service, including for users with motor impairments (who didn’t appear to be reached through usability testing).
  • The findings from this audit should be incorporated into an accessibility statement, which should be published before the product is rolled out any further.

6. Have a multidisciplinary team (Assessor: Darren McC) 

Put in place a multidisciplinary team that can create and operate the service in a sustainable way.

Assessor comments

  • The standard is met – currently. It’s good that Hackney officers were embedded with the agency team, and a skills gap was addressed, though belatedly.

Assessor score: Met

Recommendations

  • Assessors were concerned that, once the agency has rolled off, there would be insufficient skill and capacity to maintain and iterate the product. Given what has been delivered is MVP, it is important that HackIT identify a new team to take this application on for maintenance and iteration. There is clearly more work to be done.

7.Use agile ways of working (Assessor: Darren McC)

Create the service using agile, iterative user-centred methods.

Assessor comments

  • There are some good aspects in how the team worked, considering they were constrained on budget and timescale. I’m not clear how they arrived at the MVP scope, but it is clear that it contains all the right minimum things for an application to be made and managed.

Assessors score: Partially met

Recommendations

  • The retros seem to have been treated as overhead, as the time available was compressed as the build went on. Thirty minutes every two weeks does not feel  long enough to dig into issues properly, to make a sustained change to process. The future team should allocate at least an hour every sprint to this (bearing in mind the Scrum Guide recommends 3 hours for 4 weeks).

8.Iterate and improve frequently (Assessor: Darren McC)

Make sure you have the capacity, resources and technical flexibility to iterate and improve the service frequently.

Assessor comments

  • The standard is met. There is evidence elsewhere that changes were made during the life of the build phase, based on feedback from users (e.g. emails instead of SMS), despite the challenges of a fixed project timeline.
  • The team also acted on weaker signals from users, building up a bank of assumptions to test in the future, where they had insufficient evidence to make immediate design changes.

Assessors score: Met

Recommendations

  • The team has also constructed a backlog for iterations once they have departed – the onus will be on HackIT to ensure that is acted upon, and balance this backlog against new evidence that emerges after launch.

9. Create a secure service which protects users’ privacy (Assessors: David)

Evaluate what data the service will be collecting, storing and providing.

Assessor comments

  • It appears as though reasonable assumptions have been made in the selection of a user identity framework. Developers only have access to resources that they should have. The data migration was done in line with acceptable practices, and sensible filters were chosen to discard irrelevant data.
  • Concern at lack of shared identity. Users signing up for this service are signing up only for this service, and their credentials are useless outside this system. This could lead to a greater incidence of password loss and is a missed opportunity.

Assessor score: Partially met.

Recommendations

  • Actively monitor cross government discussion and developments relating to authentication and citizen sign on for public digital services to inform Hackney’s future approach

10. Define what success looks like and publish performance data (Assessor: Dan)

Work out what success looks like for your service and identify metrics which will tell you what’s working and what can be improved, combined with user research.

Assessor comments

  • The team have developed a clear set of success measures for the service, which tie right back to the vision/principles of Hackney’s housing register policy. The right tools are/will be in place to monitor these once the service is live (including manual counting, time and motion studies, user research, Google Analytics and the planned introduction of Sentry).
  • It wasn’t clear from the evidence we saw whether or not the team will also actively monitor for failure scenarios. For example, one of the service goals is to deter housing register applications from residents who are likely to face extremely long wait times. It’s possible this could also deter  users with genuine priority need, who would be housed within shorter timescales.

Assessor score: Met. 

Recommendations

  • If they haven’t already explored this, the team should consider some of the potential negative outcomes of the service, and ensure they are able to measure any early indications of them where possible.
  • The main risk here looks to be ensuring a stable team is in place to pick up the measurement activity once the service is fully live, and act on what they observe (Link to point 6. Have a multidisciplinary team).

11.Choose the right tools and technology (Assessor: David)

Choose tools and technology that let you create a high quality service in a cost effective way. Minimise the cost of changing direction in future.

Assessor comments

  • The tech stack the team described is as follows:

.NET Core / C# for backend

API Gateway

Dynamo DB

React/NextJS for frontend

Serverless Framework

  • This is in line with Hackney’s standards. The main question the assessor had here was about the choice of database – DynamoDB and the team explained how this decision was arrived at.

Assessor score: Met.

12. Make new source code open (Assessor: David)

Make all new source code open and reusable, and publish it under appropriate licences. Or if this isn’t possible, provide a convincing explanation of why this can’t be done for specific subsets of the source code.

Assessor comments

  • We didn’t get to talk about this card specifically; however, the Hackney playbook was used in the development of the product and this was backed up by conversations with the team

Assessor score: Met.

13.Use and contribute to open standards, common components and patterns (Assessor: David)

Build on open standards and common components and patterns from inside and outside government.

Assessor comments

  • This section was not specifically discussed during the assessment meeting due to time constraints. However, the assessor was satisfied that the Hackney playbook was used in the development of the product and this was backed up by conversations with the team.
  • In addition, the government design system was used on the frontend, bringing this service in line with most others stylistically.

Assessor score: Met.

14. Operate a reliable service (Assessor: David)

Minimise service downtime and have a plan to deal with it when it does happen.

Assessor comments

  • The team ensured that observability was in place using CloudWatch and X-Ray – _de rigeur_ for AWS services and certainly in line with Hackney expectations. 
  • Swagger was used for API documentation and will be a great help to anyone interfacing with the service in the future. 
  • Tests were automated as part of the CI pipeline (unit and integration only) during the staging and production release phases. 
  • Infrastructure was defined in code.

Assessor score: Partially met

Recommendations

  • The main recommendation is around the use of end-to-end tests. The developers had not implemented them, did not plan to, and did not see their usefulness in this project. Suggest that with a high throughput, public facing app such as this, end to end tests are essential.