Pilot or Perish

Pilot or Perish

Designers bring ideas to life and are able to convince organisations to build services that their customers will love, but fail to communicate how these will bring value to business. At the same time, businesses spend millions to communicate their ideas through business cases that fail to convince customers.

This article by Livework Partners Melvin Brand Flu and Lavrans Løvlie was originally published in Touchpoint Journal, Service Design Network (Volume 5 Number 1, May 2013)

Neither beautiful designs nor detailed spreadsheets give (senior) management enough certainty to invest ambitiously in new services – but pilots will.

Livework is working with clients in different industries to conduct ‘real life’ pilots with actual clients in real business scenarios and settings. The outcomes challenge often long-held beliefs about customers and the organisation and guide the design and implementation of services that work in the real world.

Pilots that combine the creativity of design with the analytical rigour of business consulting provide a powerful approach for launching services that can perform in the market. Before we look at why, we need to understand how both design practice and business cases are risky propositions on their own.

Tangible visions or 'corporate entertainment'

Organisations thirst for exciting visions and smart strategies that are wonderfully presented. Great service visualisations bring designers praise and design buyers acclaim from their peers, but often fail in the challenging process of going all the way to market. When the applause for a brilliantly envisioned customer experience fades and enthusiasm meets with business realities, the need for reassurance slowly kills the persuasive vision.

A thrilling vision of the future ends up serving as a moment of engaging corporate entertainment. For an organisation this has value in itself – and there is certainly a market for it – but ultimately it does not carry enough weight to make an impact in the real world.

Tangible visions or 'corporate entertainment'
Pilots help validate both market viability, impact on the organisation as well as economic value and customer experience.

User insights vs. customer behaviour

Qualitative design research is great at bringing a customer experiences to life inside an organisation. It is also a quick and cost-efficient way to pinpoint the most important needs and opportunities for innovation.

However, most design researchers have had to answer the question ‘So, how can I trust a conclusion based on what a handful of people say?’ This uncertainty is not caused by businesses trusting numbers over people. It is caused by an approach that does not tell them much about how customers behave en masse. Understanding how different people behave differently at large scales is needed to judge commercial value and to make crucial priorities and decisions in practice.

Qualitative research produces convincing stories, but needs to be backed up by evidence that acting on them will make an impact on large groups of customers.

User insights vs. customer behaviour
Full-scale subway station pilot of a new information system for Ruter (Transport for Oslo).

Marrying insights with quantitative data only gets you so far

Quantitative research and data analysis might reveal useful patterns and trends but are not necessarily a good predictor of customer behaviour. Validating these with customers can still throw up false positives that point in the wrong directions.

As an example, a bank invested heavily in enabling account holders to manage their household finances via an online tool, expecting to strengthen their relationship with their customers, as well as attract new ones. Research clearly showed the interest and need of the customers to better manage their finances, customer clinics confirmed this and user testing made the tool easy to use. Two years after its launch, the uptake is disappointingly low, despite the strong push to promote and redesign the service.

Solving the right problems

Visual and tangible ideas have incredibly high value. Designers have a wealth of techniques to help communicate a customer experience. When managers need to make decisions that involve effort and risk, it is invaluable to see what a service will look and feel like for customers.

At the same time, the designer’s strength in seeing the service from a customer perspective is a weakness in terms of overlooking the factors that really will make the idea sink or swim. A service concept that may make perfect sense for customers may simply be unachievable by the organisation. The reasons will often be found a long distance away from the design brief. It can be in a detail of an IT system, in an organisational structure that is too costly to change or a corporate culture that will demand a high political price for being challenged.

Designers are great at making services simple for customers, but often underestimate how complex they are for organisations to realise. Design can make a convincing argument that customers will appreciate and value the service, but also need to prove that the idea is achievable for the business.

As a case in point, a regional council dealing with children with disabilities wanted to simplify the service and make it more transparent to families and caregivers. While everyone involved in the project bought into the proposed elegant solutions and designs, it did not address the fundamental question: how to do more with less. In addition to this, senior management and stakeholders could only see the massive reorganisation and redistribution of power required, and stopped the project.

Why business cases cost organisations billions

Most organisations assure the viability of an initiative by building a business case that has to meet certain targets, such as return on investment (ROI), or an industry specific measure such as average revenue per user (ARPU). Ultimately, a business case is supposed to offer management a rationale for committing resources to a project.

Unfortunately, businesses have a graveyard of unsuccessful projects and initiatives that have cost hundreds of millions but failed to deliver, even though the business case was accepted. Often the business case is massaged, negotiated and adjusted internally to reach management acceptance, failing to test the assumptions in the market and with customers.

A global brand had a solid business case for upgrading core IT systems which was based on one billion of additional revenue. Since this programme only touched internal operational systems, customers would not experience a better service. Where would any significant increase of sales come from?

Build pilots to demonstrate and convince

Service pilots create a confidence in success that neither design nor business cases can do on their own. They merge design skill with business analysis in a way that produces effects beyond both approaches.

Build pilots to demonstrate and convince
Testing a new ticket machine for Transport for Oslo

Pilots accelerate learning

When you trial services in a real-life situation, you gain insights that are simply unachievable on the drawing board or in a spread sheet. When you move from theory to practice you will gain insights and (dis-) prove assumptions from day one. Pilots can be adjusted and changed in rapid iterations, and helps the organisation learn quickly from the behaviour of customers, staff and systems.

In testing a new service members of staff from different departments as well as consultants and trainers performed the same front line roles. Because of the diverse background and level of experience of the people servicing the customers, the organisation very quickly learned the best ways of interacting and supporting the customer. This informed frontline staff behaviours and policies even beyond the scope of the new service.

Pilots enable true customer involvement

When you ask them, customers will say one thing and do something else. The same goes for staff. When you run a real service at small scale, you can observe actual behaviour, see the results and engage with customers and staff about their experience. Pilots enable customer involvement beyond voicing their opinions through surveys, observations and workshops. They let you both understand and measure what really means something in the everyday life of customers and staff.

The bank with its financial tool for households found out after launch that even though customer really liked the idea, they were not interested in changing their behaviour and use the application to take control of their finances. Piloting with real life customers instead of controlled groups would have exposed the human tendency towards socially desired responses to topics such as financial responsibility. People would like to think of themselves as taking charge, but in reality they didn’t care enough to sit down and work it through.

Pilots engage and energise the organisation

Pilots need to be run by the people who ultimately will deliver the service, not by consultants. This engages staff, from the top to the bottom, in making service better for themselves and for customers in hands-on ways. Pilots inspire because they shift the focus from performance in daily routines, to asking ‘how can we do this better?’  Pilots breed a culture for continual improvement.

In a pilot, sales agents were given lightweight, easy-to-use tools to support quick and extremely customer-friendly instructions to customers. During the pilot, an internal competition grew between agents as to who was the best at servicing the pilots. The pilot tools, through their immediate feedback, for the first time gave agents meaningful feedback on their performance and, since the results were public, it motivated the agents in ways incentive schemes had not.

Pilots can fail

Setting up a pilot for it to work for customers and staff means it will most likely be successful. Designing a pilot to learn with the customer and staff might actually mean failure in terms of business objective. Understanding why something does not work is sometimes more valuable than seeing how something can work.

When piloting a new online-offline service concept to support new customers, the project team faced significant internal resistance from several departments. The pilots failed to make any impact on customers and were discontinued under heavy political pressure. However, through the pilots, a painful process flaw was exposed in the running of the existing service. This enabled the organisation to improve its current online and offline services significantly by eliminating the redundant process.

Pilots help identify real economic value. If you design them with the business

Pilots allow you to test, validate and adjust the business case to actual numbers. When pilots are designed to align closely with business goals, they provide the data needed to model the economic performance of the service when scaled up. This means that numbers can be compared to historical data and business forecasts. Pilots build a body of evidence that enables confident decisions.

Changing the call script for an insurance call centre enabled agents to show more empathy and offer a better service to customers. The data analysis revealed that even though the experience of these customers was significantly better, it did not contribute to the organisation’s bottom line due to the limited number of customers who were impacted.

Pilots will tell you more than the business case

A business case will describe a model for the financial value of a service. By definition, a model can’t incorporate the complexity that businesses and customers experience on a daily basis. Pilots allow organisations to identify crucial gaps in the model, and highlight opportunities that were not obvious at first sight. Pilots turn economic theory into hard evidence.

In a retail pilot, customers were offered a service that would enable them to walk out of a mobile phone shop with a fully functioning phone with all their settings and content transferred from the old one. The pilot proved the willingness to pay for the service was very high, which means it had the potential for commercial success. However, the pilot also revealed the tremendous pull this service had on attracting new customers. This had greater economic value and supported a more interesting business case.

If you think pilots are expensive, try failure

In an interesting paradox of corporate behaviour, managers are willing to invest hundreds of millions to bring a new service to market, without properly testing it in real-life situations first. Unfortunately, the cost of both fundamental issues about the proposition and small irritations in interactions will multiply when a service is launched and scaled.

Bringing new services to market demands a lot of any organisation, and rightly so, as the risks are significant. Managers both need to be convinced and they need evidence that their efforts will bring success. Pilots create both and, more importantly, they allow businesses to test, learn and refine their ideas in a real-world context that involves customers and staff.

Pilots also unite the skills of business consulting with design skill, to produce results that impact beyond both. Service designers have the skill to imagine experiences that make sense to customers and build, run and iterate service pilots. In addition, designers have the skill to design stories that convince managers and staff. Pilots built on solid business logic enable organisations to detail new service propositions based on evidence.

If you want your idea to become a successful reality for customers and for the business, pilot it to make sure it makes sense.