How Master Data supports better business intelligence

We are all visual beings now. Every day we absorb graphs, maps and informatics through many channels. We are comfortably stepping into them but often overwhelmed by the sheer amount on offer. So what we all need is well-designed, colourful, intuitive information, which allows users to toggle between summaries and details: Hackney dashboards.

The evolution of the Qlik BI project has been rapid, starting from “what is needed” accelerating quickly into “what is possible”, with new opportunities constantly emerging.

First, we created dashboards by engaging with colleagues who needed to replace their legacy reporting tools, in areas like Parking and Planning. We worked with the Housing Repairs team to show data they could have not easily see nor interact with using traditional reports. Our customers either knew or suspected that the answers to their challenging questions lurked in the data they collect every day but they were swamped with little time to analyse and draw meaningful conclusions.

Initial success came quick. Our customers received information which were not only facts or dry statistics but colourful, interactive and up to date information available to the most granular level. They further asked us to focus on the relationships between the figures, within which they are connected visually both in  depth and breadth.

As with all data projects we continued experimenting with exploration: both what is possible and how to make it approachable and accessible. We have style guides, we follow the local gov digital service standards, borrowed from other organisations. But in Hackney we have something unique. We have a pervading common denominator: master data indices which link disparate service data by a single reference key: unique address and customer reference numbers. We have been managing them for years –  insisting they are included in all new systems- and they allow data to flow between systems. Not only are they are very useful backgrounds link when serving our customers, they are very useful in all BI work to leverage much more meaningful insights.

So what could be gleaned from using our master data? Our data mining looked into the money element first: what is the overall debt? What properties cost us the most to serve? How can we promote people to pay by direct debit?  Which tenants could be illegally subletting their properties?

But by using these links we can do much more than just protect the ever dwindling public purse, we can improve the lives of our residents: We can get the full picture of who lives in a block of flats, rather than rely on one incomplete database;  understand which vulnerable tenants are being chased for rent debt when the long-term cost of our actions is far greater than money recouped; or understand the attributes and demographics of people affected by a new policy change such as the Council Tax Reduction Scheme. Most of all however, we aim to deliver dashboards which would inform us how to build a better relationship with our customers.

Qlik has provided us a canvas on which to illustrate these explorations with rich palette of further insights we are continuously working on.

Different lenses for understanding users

Ahead of user research week. I thought it might be useful to introduce some other service design principles, that it might be useful to think about when designing a new service, namely Life-Cycles.

Lifecycles can aid your design by giving an “outside-in” perspective of what customers experience across an entire sector.

Human Lifecycle– this describes how people behave in different key stages in their life. Thinking about this, gives you an overview of what really matters in people’s lives beyond what your organisation offers, and helps you understand how your organisation can support customers in major transitions in their lives e.g. from school attendance (Hackney Learning trust) to school leaver (ways into work), to employment, to renting home a home (choice based letting) to parenthood (children’s centres) and onto retirement (day centres for the elderly). What customers may need from your service will vary according to what phase they are in their lives and what transitions they are going through in their life.

The Consumer lifecycle – describes how people behave in a market when they make choices about their needs or wants. So if someone is going on a business trip they may use a number of services to meet their needs. How a service comes together and interacts (with those offered by your partners and competitors) affects the overall consumer experience. Understanding how people make choices enables you to design a service which supports them to make the right choices.

The Customer lifecycle – describes how customers become aware of a service, choose a service, pay for a contract, use that service, upgrade a contract, have incidents with that service and then either renew their contract or leave. Anyone whom has purchased a phone under contract knows that how you are treated during that lifecycle will reflect whether you renew your contract or go elsewhere. So thinking about that whole lifecycle and how a service deals with customers during the lifecycle can help increase customer loyalty, retain customers and optimise the contract holding experience. Commercial waste contracts is an area the council deals with where consideration of the customer lifecycle in designing a new service may help.

The user lifecycle – is a tool to help reduce costs, drive efficiencies and trigger new behaviours when people use the product or service. This tool helps you visualise the journey a user goes through maybe using multiple channels when they interact with your service. It enables you to visualise the service delivery across multiple channels to enable you to simplify and improve customer interactions.

In conclusion when designing a new service you need to do so through the lenses of the human, user,consumer and customer lifecycles.

Rent Arrears and the Sweet Science, v.01

As Digital Transformation Manager for Housing Services at Hackney Council, I get to run some really fun and interesting projects. This week saw the start of a great one however, as we kicked off a data science project which we hope will enable us to identify tenants most at risk of falling into rent arrears (it also marks the start of what will no doubt become a remarkably infrequent blog, but that’s another story).

Arrears are bad news for everybody; they cause stress for tenants who may lose their home and have to move out of the borough away from their support networks, and they reduce the amount of money we have to maintain homes and estates. When people fall into arrears we set up repayment plans, but for people on limited incomes the size of the repayment we can realistically expect them to be able to pay consistently on top of their rent is quite low, making debt stubbornly difficult to clear.

But what if there’s a better way, and we can predict those most at risk of falling into arrears so that interventions can be targeted to prevent the problem before it occurs? Within the council we have a Financial Inclusion team that helps residents with financial planning and can point them towards training to help them get better paying jobs. If we could better target that team’s resources to those most at risk then this wouldn’t just help reduce rent arrears, but give our tenants more control of their finances and help tackle unsecured debts, or payday loans.

To try and answer this question we’re working with a company called Pivigo, that run a programme to train candidates with PhD’s in quantitative disciplines to be data scientists. As part of the programme they need organisations to present them with real world problems to solve, so this week I met with our team to discuss the challenge. The team comprises:

  • Francesca Renzi, holder of a PhD in Nuclear Physics and winner of three research grants from the Umbria Region in Italy.
  • Philipp Ludersdorfer, holder of a PhD in Cognitive Neuroscience who has previously developed statistical models to predict outcomes and recovery of stroke patients.
  • Tom Northey, holder of a PhD in Bioinformatics and award winner at the TfL Data Science hackathon.

    Over the next month they will be analysing our data using techniques such as clustering, decision trees, and time-series modelling and building a model to try and quantify the risk of a resident falling into arrears. This model should then enable us to play with certain parameters such as anticipated inflation or wage growth rates to see how this may impact on our residents in given scenarios. Microsoft have given us free access to their Azure Machine Learning platform for the duration of the project, but the algorithm we develop will be platform-agnostic and available on Hackney Council’s GitHub repository. We hope that other local authorities and housing associations will test their data against it also and that we can work together to build upon and refine it.

    In a project dealing with such personal data as this privacy is of course incredibly important. Personally identifiable data is not needed to develop the model and so not included in any of the data sets used for analysis or testing. Similarly, should the project be successful and we create something that can be introduced to our working environment it wouldn’t be something that staff would be able to dig around in, but a tool that selectively highlights only those that may be considered vulnerable to the teams that can help them.

    The project is scheduled to run until 7th September and I’ll post again on what we’ve learned, but the end point is really just the MVP and I’m hoping that we can work with other local authorities and housing associations to develop this further.

  • Introducing the Hackney Agile Lifecycle

    Hi my name’s Michael Stevens, I work in Hackney ICT as head of business solutions (Hackney ICT’s team of business analysts/project managers and programme managers). This week I organised a team away day, the purpose of which was to help promote and embed our new agile ways of working the Hackney Agile life-cycle. A new process based on the Local Government Digital Service Standard and the design council’s double diamond process model which I have developed with the head of digital Matthew Cain

    • How was the away day designed ? as a series of role plays, with scripts for each part of the agile lifecycle. We asked attendees to work as part of a team, through a scenario using our agile lifecycle and produce the actual outputs they would in the real world. With one agile team per away day table. At the end of each stage a stage-gate presentation was given to the sponsoring group (sponsor) and permission sought to progress to the next stage.
    • What has the away day achieved? the away day is not the end of the journey in terms of agile, but an enabler for the start of a new chapter for the ICT dept. We will be doing follow up seminars to explore agile techniques in more depth such as Sprint planning , retrospectives and proto-typing, to help everyone in the unit build their knowledge and skills in agile techniques. Working in a more agile way is about us changing our emphasis, from being a support service that fixes things, to focus on being a strategic partner for our colleagues in other services in the council. Having an agile process in place will enable us to work more collaboratively as partners with our colleagues to deliver the excellent digital services of the future that our residents and partners need and deserve.

    To access the away day scenario & scripts check out our drop box.

    Developing digital services for all

    We’ve been working hard over the last few months to understand how to accelerate our delivery of digital services that are so good, people prefer to use them. It’s not a controversial idea, but we’ve got a lot of hard work to achieve it.

    The starting point is our HackIT manifesto which explains how we want to work and what’s important to us. We’ve produced this because we wanted to set out a clear and easily understood set of principles that we can use to make sure we’re all taking a consistent approach to delivering digital change at Hackney. The manifesto gives us a lens through which to examine everything we do.

    We are committing to the Local Government Digital Service Standard (the manifesto reminds us we’re part of a wider community working to deliver excellent digital local services). Everything we buy, commission or build will be assessed according to whether it meets the Standard. Important services will be assessed by external reviewers with expertise in digital services. This will help us make sure that our digital services put people first. If you are able to help as a reviewer, please get in touch.

    Next, we’re going to deliver our work using the Hackney Agile Lifecycle. This owes a huge debt to the UK Government Service Design Manual and similar approaches in the US and Australia. However, it’s rooted in the Design Council’s double diamond model of delivery. Anyone who’s working with us (Council officers, partners, other agencies) can understand our process. The job of creating digital services isn’t just an ICT thing- it’s a globally-recognised design challenge and we need to stand on the shoulders of giants.

    We’re opening up. We’re starting to publish our code on GitHub. We’re blogging about our work and what we learn and we’re spending time with residents to learn what they really need – and then making sure our services meet those needs. This will help us make decisions together. Our Tactics Manual brings together techniques from Google Ventures, 18F as well as our own experiences to help any team work collaboratively to tackle a problem at pace. Everything we’re producing is shared openly on GitHub and we’re really keen to hear from. others who want to copy, edit or propose improvements.

    Designing a new digital service is just the first step and the work mustn’t stop there. Typically it’s much easier to do a project well than it is to continuously improve a service. The fourth stage of our design lifecycle is: improve. It sets an expectation that once it’s fully operational a service will need continued leadership, review and focus to meet people’s changing expectations.

    Finally, we’re trying to move from mandating a way of doing something, towards finding many routes to high quality outcomes. We trust the team to find their preferred way of doing something – provided it meets users needs.

    This is the start of our journey. We’re fortunate that we can learn from the hard work of so many colleagues and share ideas for ways we can make public services better. We can play our part in designing digital services for everyone.

    Adopting the local government digital service standard

    Have you ever used the “GOV.UK” website to find some information or to use an online central government service?  If so, you may have seen that some of the pages are marked “Beta”. Here is an example at the time of writing:

    This does not mean that the information is wrong or the service is not yet ready.  Everything that is provided works as it should and the “Beta” marking is simply there to invite comments from users so that the webpage or service can be improved before it is formally considered to be “live”.  Once it is live, the “Beta” marking is removed but even then the information or service will still continue to be reviewed and updated throughout its life.

    This way of working is part of the Government Digital Strategy that was launched back in 2012 and set out how government intended to redesign its digital services so well that people prefer to use them.  The approach includes making usable versions of digital services available as quickly as possible, and obtaining user views early in the process. The criteria for this approach have been documented in a Digital Service Standard and this is used to assess whether a digital service has been created in accordance with the Strategy.

    Local Government has been following all of this with interest and an increasing number of councils are starting to adopt similar ways of working.  A steering group was formed last year and, with input from over 60 councils, a Local Government Digital Service Standard was created based on the one for central government. It suggests a common approach for local authorities to deliver good quality, user centred, value for money digital services.

    The idea of a “standard” might sound theoretical and maybe even bureaucratic, but in reality the points in the Standard are very practical and might be considered to be common sense. As such they provide a useful checklist to help Hackney create online services that people want to use.

    ICT has been exploring what the adoption of the Standard might mean for Hackney.  As well as sessions to exchange ideas with other councils, we have held three short, focused workshops to look at the implications of some of the points in the Standard.

    One of the main conclusions was that more engagement with service areas, users and perhaps residents will be needed during development than might have been the case previously. Service areas might find themselves invited to be part of project teams and to be given an early view of proposed developments and the chance to influence the direction. This will no doubt place greater demands on everyone’s time although the reward that is sought is the faster delivery of digital services that are intuitive to use, meet service area needs and can better respond to change.

    Within ICT we recognise that as well as finding ways to achieve this greater engagement, we will need to bring together a wider range of new technologies and services in the future. We also need to be able to assess the effectiveness of the digital services both through user and service area feedback and the use of appropriate monitoring tools.

    This will require new skills and methods in running projects, designing solutions, creating user interfaces that are simple and intuitive, measuring service performance and obtaining and acting upon comments about Hackney’s digital services. Good communication will be more important than ever. There is a lot for everyone to learn but also an opportunity to make full use of anything that already works well and to draw on the wider experience in digital services that is being gained across local government.




    Your data, protection and the right to know

    Hi, I’m Neil Lang, and I am helping Hackney Council prepare for new legislation about Data Protection and how we use your personal information. This is the first of a series of blogs I will be posting over the next year about how the Council works with your information and the steps we are taking to put you in the driving seat for how it is used.

    Have you ever wondered what information an organisation holds about you? In the information age, as we use online systems, we leave quite a digital footprint behind us. For example, when using free services from Google, you are making a deal. You get to use YouTube, search, Gmail, Drive and Google Maps for free.
    In exchange for this free use, you agree to share information about yourself that Google can share with advertisers to target their ads and make them more effective. For instance, airlines want to present to people who love to travel. Toy makers want to target children of the right age group and interests. Read on to find out how to find out what Google already knows about you!
    Google uses a lot of methods to learn about you. Some of it, you tell Google when you sign up for its services (your name, phone number, location, etc…) Other information may be less obvious. As you browse the internet, your searches all give clues to your interests — which Google collects for advertisers to direct their offerings to you. Of course, it’s not a perfect world, which is why from time to time you see adverts you might consider quite inappropriate!
    But Google also takes user privacy seriously. It knows what information it holds about you (that might seem obvious, but a lot of organisations don’t have a good handle on this, or the tools to readily discover it) – and Google is able to remove information about you so it doesn’t come up in searches (the so called ‘right to be forgotten’).

    New legislation

    Public authorities, like Hackney, also have to look after your personal information responsibly. Next May, a new piece of legislation – known as the General Data Protection Regulation – comes into force. This will replace our existing Data Protection Act, and introduces some important new rights about how we look after and use information about you.

    • It applies to both public and most private companies.
    • It reduces the time to respond to requests for information about you (known as ‘subject access requests’) from 40 days to one month;
    • it will require parental consent for processing children’s data;
    • it provides for a right to have your data corrected, or removed (with certain exceptions);
    • it requires clear consent to data processing;
    • it allows you to object to automated processing/profiling using your personal information;
    • and you may request (following a subject access request) to have your data provided to you in a portable, machine readable format.

    Hackney Borough Council is currently preparing to be able to fulfil all of these new rights, and wants to build solid digital trust with its residents in the way we store and use your data. Unlike profit-making commercial organisations such as Google, we simply want to use your information to deliver better services to you. In particular, residents often have concerns about information being shared between different parts of the Council. We will be transparent about how and why we do this, and we will only do this with your consent. Allowing us to do this will enable us to provide you with great, value for money services, at a time when all Councils are facing reductions in their overall funding.

    What does Google know about you?

    So how do you find out everything Google knows about you?
    By visiting a page called “Web & App Activity,” you can see what Google is watching.
    Then by visiting a site called “Ads Settings,” you can see what Google thinks it knows about you, and you can change what it’s telling advertisers about you.
    It’s not easy to find your “Web & App Activity” page. First, you have to be logged into Google. Once logged in, go to “” and click on “all time.”
    This brings up a long list of all the web pages you searched. You can delete them, but Google doesn’t make it particularly easy – it only lets you delete one day at a time. You’ll get a warning from Google suggesting that you don’t really want to delete this information (because, in truth, Google doesn’t want you to delete it). Don’t worry if you do, you won’t break the internet or your Google account if you hit the delete button!.
    Until next time, happy browsing!

    Understanding user need for technology in housing

    We are working with our colleagues in housing and FutureGov to understand what our tenants and leaseholders need from the Council’s housing service, and look at how technology can support those needs.

    The work has identified that too often many of our existing processes and systems are too complex for most residents. And because these have developed organically over time, it can often be hard for the service to be confident it has the information it needs to make the right decisions about how to handle problems and where to invest. That’s the starting point for a new piece of work to decide on what technology is needed – including whether the ‘one system’ approach is actually the right one, or whether we should move towards a more ‘loosely coupled’ approach.

    One of the most powerful findings of our research has been that the services and processes which support our tenants and leaseholders share many of the same characteristics as the services and processes needed for many other transactional services that the Council provides for our citizens. Reporting a blocked drain isn’t so very different from reporting a broken streetlight – or at least, it shouldn’t be.

    Happily, this work fits very neatly with three other workstreams in our programme to deliver digital change for everyone:

    1. our review of our CRM platform
    2. the need to renew our master data solution
    3. work to understand how we accelerate the delivery of services through Hackney’s One Account

    This is a great opportunity for us to think much more broadly about the underlying technology we use, and how we can build on it to make sure that we support successful, cost-effective customer journeys based on a single view of our customers and assets (homes, buildings etc).

    We are now assembling the team needed to make this happen. The team will comprise user research, service design and development skills, overseen by a Product Owner empowered to make decisions about the design of the service. We will reduce the risk of this complex project by working to Agile principles: breaking down the requirement into individual user stories, delivering them in fortnightly batches (or sprints) with the priority of testing working code directly with residents in weeks and using their feedback to drive our direction.

    So, rather than just buying a new housing solution, we’re planning to look at how far we can go by using generic solutions which we can use as a platform to build the end-to-end digital services that our residents need. Throughout the project we’ll be working openly. Nothing we do is likely to be unique to Hackney. If other local authorities, housing associations, or innovative software companies have part of the solution already, or want to work with us to fill a gap in user needs, we’d love to look at the potential to work together and share our learning.

    This project will be our next major step towards the ICT service that the Council wants, and our residents need. We’ll be consolidating the ideas we have already been working with: user-centred design, Agile, open source development and working to the local government digital service standard.

    This isn’t ICT-led or technology-led change. We want it to be genuine service transformation facilitated by the vision of the senior leadership of the organisation. We’re fortunate to work in such circumstances, which makes it all the more important that we deliver the right solutions for our tenants and leaseholders.

    New ways to publish health needs

    My name’s Katherine, and I started work in the Intelligence bit of the Public Health Team in 2014. One of my first jobs that summer was to look into commissioning a website for information on local health and wellbeing needs.

    Over the next two years, I wrote a lot of business cases, learned a lot about procurement processes, and missed a lot of deadlines. The one thing I didn’t do was test anything out. We’d already consulted with stakeholders in 2014 – why would we need to talk to them again before we’d commissioned a website for them to test?

    Then, at the start of 2017, we met with Matthew. Great, he said, I like your idea – now try it.

    The Intelligence bit of the Public Health Team has four members. We don’t build websites. We can whip you up a lovely infographic about vaccinations (good) or smoking (not good), and if you stand still near us long enough we’ll probably explain some statistics to you, but we don’t build websites.

    We built a website.

    We decided on the least possible amount of content we could put on the website to make it worth testing. We decided on the most important basic things that the website needed to do in order to be worth having. And then, using free online tools and a couple of half hour tutorials in Matthew’s office, we put it together.

    In two weeks of building this simple, quick, bare bones prototype, we learned more about our idea than we had in two years of talking it through. We refined it down, discarded the bits that didn’t work, and realised that we’d been making life needlessly complicated. We can now use free tools for things we thought we’d have to spend half our budget on.

    Over the next few weeks, we tested it out with actual humans. We asked them to do things on the website, and got them to talk us through the process. Pretty quickly, we learned a second thing: What people want out of this kind of website is several simple ways to search for content, not one super whizzy way.

    From this, we learned the difference between what stakeholders may say they want and what they actually want. We understood better how people will use our website, and what will make them abandon it. We now have a clear idea of what we want the site to look like and do – which will save us time and money when we bring in an external designer.

    That bit comes next. I’ll let you know how we do…

    Personalising our support service

    Hi, I’m Cate, and I work on the service desk.

    At Hackney we are constantly working to improve the way we support our colleagues. Over the last couple of years we’ve been changing the way our users contact us and now the majority of calls are logged online, rather than over the phone.

    This has provided our support teams with the opportunity to innovate and look at new ways that we can interact with our users – making our support more personal and responsive. We started this with drop-in support workshops, where users could speak with people from across ICT and get advice and support. And we’ve now extended on that, holding regular ‘pop up’ clinics at buildings across the Council’s campus on a weekly basis, providing support and advice as well as quickly swapping out basic equipment which users might not have had time to report to us as faulty. In the last few weeks we’ve also started to offer bookable appointments, making it possible for users to book a 15 minute slot to speak with one of our team to get help with issues they’re having or to ask for general advice on the systems they use.

    These changes have proven to be very popular with our users and we’re now looking to expand these services to include more specialised advice and support from other support teams across the ICT team.

    We have found that this more personal approach not only benefits our users but also gives our staff a better understanding of the daily challenges faced by users across the Council’s services. Another benefit is giving more variety to the work our support teams do and regular face-to-face contact which helps broadens their knowledge and experience.

    We are continuing to look for interesting ways that we can further improve the support we provide for our colleagues, and are now looking at introducing web chat and mobile-friendly access to our service as our next developments. We’d love to hear from support teams elsewhere to hear your ideas and see whether we could look to include those in our future changes. Do please get in touch!