Tag: Consistency

Hackney NuGet Package

NuGet Package

Context

At Hackney, we believe in reusability and consistency that is applied to our API development process for all projects. As we started building more APIs we identified a lot of duplication of code from several product developments that took place on APIs with similar user needs. Hence we have identified a need to have a service that would store common code that can then be used between different APIs and contribute towards the open source community. After doing lots of research, we decided to start building our own NuGet packages which will help developers in faster development, reusable code and help us to achieve consistency across the board.  

Our Users and their needs

As a member of the Technical Design Authority team and the Security team, I would like to:

  • Ensure that the NuGet Packages have a secure authentication in place so that Security Compliance rules are not compromised.

As a Developer I would like to:

  • See consistency between APIs and Listeners so that any changes made to the data models can be reflected in both areas straight away
  • Reduce duplication of code across APIs so that development time can be used in exploring new features/functionalities and reusability can be achieved
  • Ensure any NuGet package created is secure so that security practices are not compromised

As a Solution Architect, I would like to:

  • Have awareness of the different NuGet Packages that we have so that they can be consumed across solutions that I design
  • Have an easy way to find NuGet Package documentation so that I can make informed decisions about whether the NuGet package can be consumed in new solutions

As a Product Manager, I would like to:

  • Ensure development time is reduced so that there is faster delivery to our users

As an External Agency developer, I would like to:

  • View the different NuGet packages that we have so that I can consume them within my APIs.
  • View the different NuGet packages so that I can create new ones that can be consumed across different services.

Our Vision

  • To introduce reusability between APIs developed for product development so that duplication of code is reduced across many product developments. This also enables a return of investment on the platform.
  • To be the source of truth for data models between APIs and Listeners so that data remain up to date at all times and thereby achieve consistency.
  • Better practices so that faster delivery of services to our residents is promoted as well tested feature.
  • Enables the team to be adhering to best practices as described.
  • Well tested and secure piece of code enables the team to emphasise security assurance and gives confidence that we are following the security first principle in depth.
  • To reduce the duplication of code so that there is better performance of applications 

Why would you need a Hackney NuGet repository?

We use NuGet packages to share code that is exclusive to an organisation or a project. This becomes more useful when having a microservice architecture where multiple services share common code, things like middleware, validation, logging, etc in order to have a strong ecosystem in place. 

Although Hackney has got the Base API template that is used when building APIs all the time and allows us to onboard new developers quickly, there is still a need for NuGet packages, here few examples:

  • Base API is a boilerplate code for being reused for new APIs which might contain a part of shared common code as not all services might need everything. 
  • If we update an existing shared code in the Base API, then we need to manually update all the services which are using the Base API. Instead, if we have a NuGet repository, we should just update the new version of the package inside the Base API and the services.
  • When a change is required we only need to do so in one place which helps to reduce the development time required to update code.
  • We might need to share multi-domains specific code that does not belong to the Base API. This will allow the source of truth to be available between different services and doesn’t require developers to update several APIs at the same time. 

What is a NuGet package and its benefits to our ecosystem?

A NuGet package is a single .Zip file that contains a compiled code that can then be consumed and shared between developers. Each package is deployed to a central NuGet repository that can then be installed within your services. 

The NuGet packages are secure as they require a Personal Access Token. Without the Personal Access Token, you will not be able to install the NuGet Package.  Not only does the approach of using NuGet Packages meet Hackney’s 7 approaches but it also provides a source of truth for standard data models. 

Our NuGet packages are reliable as it has the most up-to-date data models that can then be used across different repositories. Also, our NuGet packages contributes towards meeting the KPI metrics in order to justify the return of investment.

Here are our NuGet Packages if you would like to take a look: 

https://github.com/orgs/LBHackney-IT/packages

Our use cases for using in-house built NuGet packages

Within Hackney, we were able to identify a lot of APIs with the same user needs hence reusability can be accomplished across different APIs. Every time a change was required for a piece of code it was needed to update several different APIs. This increased the development time required. To reduce the duplication of code and development time required we identified a need of creating NuGet packages that would store common code which can then be consumed across several APIs. 

For example, within Hackney, we follow Event Driven Architecture where our APIs act as publishers and custom-built ‘listeners’ act as the consumers. The data model stored in the Listener should be the same as the one in the API – to prevent making any updates in multiple places we decided to have a NuGet package that would store the data model which can then be consumed by the API and Listeners. This will allow the Nuget package to be the source of truth and will promote consistency.

We have chosen NuGet as a tool to produce and consume packages as it contributes to efficiency in our development process, through the use of a central point of reference for all services that use a particular data model. 

NuGet packages can also be used to compile common code that can be shared across different APIs. For example, If an API is using DynamoDB as its database service, the converters might be required to convert the data type when the data goes in and out of DynamoDB. Rather than having the converters within each individual API, we decided to create a NuGet package that can then be consumed by all the APIs. This prevents duplication of code and reduces the time required in development. 

Using and building packages

Consuming our NuGet packages is the same as using any other public NuGet packages – the main task that is required to do is to add the package to the given project and then set up any configuration for authentication (if required). For the purpose of onboarding new developers, we ensure that we have extensive documentation in the READMEs of our repositories that is kept up-to-date, We also include the NuGet Packages that we have within our API Playbook and the Base API template… Simple as that! 

To ensure that consuming services are not affected by changes to our common packages, we use an automated versioning strategy such that every time an update is made to the NuGet Package the version of the package is calculated automatically. A service will only be able to consume new changes once they have updated the package version they are using in their respective project and deployed the changes. This allows the individual teams to check if there is a need for them to update the NuGet package via the NuGet package manager as part of the code update process.

When it comes to naming our NuGet Package, we like to ensure that our packages are clear and concise to allow all developers to be aware of the features that are available in the packages. All of our feature packages start off with Hackney.Core this helps developers to identify that this is a Hackney build NuGet package. All of our shared packages start off with Hackney.Shared this allows developers to identify that this data model is used across different services. 

We are looking forward to any collaboration and we are more than happy to receive any feedback for improvements as we believe in failing fast and learning from the experience and iterating our ecosystem.

Future plans

We have been able to see a lot of benefits from consuming NuGet packages and we would like to share this with different developers so that they can improve their ecosystem. Here are some of the future plans that we are considering taking into account:

  • Encourage developers from all projects across HackIT to consume our NuGet packages
  • Encourage other teams to create their own packages to prevent duplication of code between APIs across HackIT
  • To continue consuming these packages and find any possible areas of improvement required. 
  • Automate the process of notifying developers when an update has been made to a NuGet Package.