Before Innform, the team behind the hospitality training startup was making eLearning training tools for multinational brands. And despite these companies’ indisputable training knowhow, a forward-thinking leadership and good budgets – they still allowed us to follow the same path that any product team would in their position. Educational software is just like any other software: it needs to be something people want. In software development, we call the step that confirms this essential criterion, Product Validation.
Over the years, I’ve had the pleasure to lead several eLearning projects from idea to launch. Most of these tools are training and developing thousands of employees worldwide (in some cases hundreds of thousands) – we managed to build platforms that work, and that people liked. It’s easier said than done, but once you understand how product validation works, designing with uncertainty will become a thing of the past.
So here’s the big secret: Establish a system of product validation.
Let’s unpack this.
Any design team, even those new to the game, will realise at one point or another (hopefully before too much time and money has been spent!) that testing a product prototype with end users is probably a good idea. It will provide the company with valuable answers for a small fraction of what they’d spent if they shipped a final product (considering it’s a success, that is!)
Take car manufacturers, for example: Before Toyota decides how to spend $1bn to design, launch and market a new model, they’ll produce several prototypes (known as concept vehicles) for their customers to look at, try out and reflect on. They will observe their reactions and assess the car in the real world for safety, drivability, longevity and so on. For a tiny relative cost, Toyota will know, with sufficient confidence, which of their new car models will be going places.
Agile manufacturing, rapid prototyping, bootstrapping, an MVP approach: Call it what you like. These are processes based around continuous product-design validation, and it makes perfect business sense.
But what is the number one mistake design teams make when it comes to product validation?
They run product validation tests only at times of uncertainty.
They are reactive, rather than proactive, and will apply the breaks to test a product only when customers, stakeholders or CEOs become nervous. This is by all means good practice – and a wise move if we’re reasonably unsure of a product direction.
A good design process is based on systematic and consistent validation, rather than inspiration or urgent corrective action.
But there is a better way, one which offers a great deal of peace of mind and that makes stakeholders far more likely to be engaged and supportive of the overall process: An ongoing system of product validation. Good products are born out of a consisten, reliable and systematic design process. One based more on evidence than inspiration and urgent corrective action.
So consistency is key, now let’s look at how we can execute a system of product validation
A system of Product Validation
When you manage to get a rhythm going in your product validation efforts, you leave nothing to chance. Here are 10 ways that will help you arrive to a water-tight software solution on time, and budget!
1. Weekly one-on-one user testing sessions generates momentum as well as ongoing insight [weekly]
Today, most design teams will agree that user testing is an absolute must. Without it we are blind, spitballing ideas and going simply with hunches. Bad news for our client’s allocated budgets, or that precious investment we just raised.
But rather than doing these remotely, or out sourcing testing to a user testing company – try doing them yourself and in person. A 5 minute interaction with a user using your prototype will teach you more than 5 days of agonising contemplation – I promise you.
Here’s how we run weekly prototype-based user tests at Innform
- Find 10 non-users who fit your predefined user personas
- Split them in 5 groups of two. The sessions will take place weekly for 5 weeks. At the end of the 5th week, you would have planned another 5 sessions with 10 new users. Note: The more users, the merrier!
- Invite them to your office and offer an incentive (ex: an Amazon discount voucher)
- Be clear with what you want them to try, keep it down to 30 minutes and cover three product scenarios, max! As your testers get tired, you will start getting distorted feedback.
- Ask them to think aloud, you want to hear that internal monologue – that’s where the magic happens
- Do not help them unless they are totally stuck and find no way of reaching the next step
- Ask them for their opinion, what they’d change, what they like, what they don’t.
Setting up a user testing ritual becomes doable when it is planned ahead of time and executed in a lean fashion (good is good enough!)
A great prototyping tool which we love to use and can highly recommend is Marvel. It’s ideal to test the initial stages of your product journeys and also integrates perfectly with your UI design tool of choice.
A 5 minute interaction with a user using your prototype will teach you more than 5 days of agonising contemplation.
2. User feedback and public boards [twice weekly]
They are free (albeit sometimes frustrating) sources of pure validation gold.
We sometimes forget that we’re part of a community, a community of people who want to use the stuff we make. Imagine how strange it would be for a school not to have a deep understanding of it’s society’s longterm ambitions. Or how impractical it would be had restaurant chains stopped learning about their consumers’ changing eating habits. And would we trust an airline that ignored every other complaint whenever disappointed passengers are double booked, missing their flight? Feedback helps us improve our products and services, and most often it comes to us from our community. The truth is they care: about their outcome and about the brand they grew fond of, so listening is the least we can do.
Today, I find operating a project in isolation and away from the community, a laughable proposition. Not only because of the peripheral blindness that has so often toppled even the most formidable software companies, but because I genuinely believe products have an inherent human component that we just can’t do without – it’s the secret sauce, if you like.
Luckily today we can receive feedback for free via public Trello boards, forums and review websites. It isn’t always pleasant to read and often tests our patience (most of us do our utmost to build the best possible product!) but somewhere buried in that criticism (and mind you, feedback is often constructive) is a nugget of pure gold.
Set up public forums, support centres, live chats and other ways by which your customers can reach you whenever they have feedback – and check it regularly. Run it past your product roadmap, and if it is something that will help delight customers in the future, put it on your todo list.
Remember: Every bit of constructive feedback is positive validation. People will not complain about a product unless they care about it on some level. So take it as a positive sign that your community has embraced your solution but wants it to be better.
A super useful SaaS tool for user feedback is Hotjar, which allows you to understand how users are actually using your tool.
Software products have an inherent human component that we just can’t go without.
3. Run regular accessibility and usability tests [daily]
You heard it before: There is no need to reinvent the wheel! And most of the time this is absolute gospel.
Today we stand on the shoulders of giants. Almost everything has been experienced and solved by someone else who came before us. I call them generous souls who went out of their way to diligently journal their pain with a process, and writing up their tried and tested solution. We are ever grateful.
One of these timeless pieces of wisdom is accessibility and usability principals. The two terms are closely related and identical in the sense that they are design commandments to help us build products that are intuitive to our users – in other words, they are rules that guide designers to build good quality software.
I won’t be going into detail about these principals in this post, but I’ll be writing more about this later. The great news is that checking our in-progress prototype on these fundamentals can lead to a pre-validated product. They allow us to present our testers with a prototype that is already fit for consumption, unhindered by silly UX roadblocks.
When we use these tools to pre-validate prototypes, we show users something of quality every time. This way they can focus on testing our innovations, rather than working their way through an half-baked interface.
4. Focus groups stimulate creativity [monthly]
Running a monthly focus group is like yoga – it can be uncomfortable in the moment but you know it will pay dividends in the long term.
Think about it: Kindred spirits in a room, a magical synergy, a healthy flow of ideas and possible solutions. OK – they can be a challenge to organise and moderate, but a focus group acts as a corrective measure every time we veer off the path. Assuming we invite the right group to the party, you’ll get product validation and ideas for the price of one.
Focus groups can be expensive and time consuming, but generally one session per month is more than enough to help your team gain a deeper understanding of the ‘real world’ the product will exist in.
I’ve heard of many cases where a focus group will hands down reject a product that is half-way through production. That’s bad news, of course, but what often happens is that the same group will offer alternative ideas, tweaks to the original plan or radical improvements that the design team hadn’t yet thought of – The echo chamber can be a dangerous place, and a focus group can be a great way to bring us off that tunnel and offer us a refreshingly subjective point-of-view.
5. Regular stakeholder checkpoints [every 2 weeks, or at the end of every ‘sprint’]
Modern design teams who believe in sprints religiously will segregate a product team from its stakeholders. This division is meant to help designers and engineers focus on the job at hand (usually in 10-day bursts) without external interference. I agree with this, it instills a discipline in a process that helps both parties get the job done right (even though it sometimes doesn’t feel that way – we all like a bit of control!).
Having said all that, we have to keep stakeholders in the loop. I propose a call to happen every ten days, during which everyone is updated with progress, delays and potential challenges ahead. A good practice effort that can help in these calls is a product backlog, which holds every idea and bit of feedback that is produced by these sessions. Each idea/feature/bug is tagged with a priority value and assigned to the team in a later sprint.
Regular checkpoints with stakeholders will help you establish a sense of trust with the global team, and will put everyone involved more at ease. Transparency, honesty, good communication: these values are important in any relationship, let alone one of which happiness is conditioned by adherence to deadlines and assurance of quality.
Yes, the user and customer always comes first – we must above all please them. But keep in mind that your stakeholders (the people commissioning the project, providing the budget, or have a direct interest in its success) will have invaluable industry wisdom that can help shape the product, and in turn validate the solution.
Our industry’s greatest strength is the ease with which we can react and change tack. It supports the modern world’s urgency to transform, optimise and surpass.
6. A fluid specification list [review bi-monthly]
In the olden times, software projects kicked off with a the all-important ‘specification sign-off’. A list of detailed features and components that will go on to make up the end product. The granularity is sometimes headache-inducing (and rightly so, this document will protect buyers and vendors alike) but we used to get one thing wrong: They were set in stone, often to the detriment software success.
The clue is in the name: Software is malleable, and the world has accepted it to be as such. Virtually every industry understands that software exists to serve their ever-changing, technologically driven business. Technology reshaped the way people think and therefore behave – the way they do business and buy products. Can we afford to ignore this immutable truth? I don’t believe we can. Then how can we expect a software that was moulded 24 months ago be implemented today without any hiccups?
What this means is that a product that was by all accounts validated one or two years ago may very well be missing the mark today – and that is totally fine, in fact our industry’s biggest strength is the ease with which we can react and change tack. This is why the software industry has become one of the most essential and rapidly growing sectors today – it supports the modern world’s urgency to transform, optimise and surpass.
Keep your specifications flexible, and avoid being rigid whenever business requirements change. Delivering a product in modules may be a healthier option than shipping a DVD in a world of Netflix.
7. Keeping up with game-changing technology [daily]
When we choose to build (and validate) software products, we take on a responsibility. One which our stakeholders, users and customers do not always fully understand – this makes it an even bigger responsibility.
Imagine a world in which aircraft manufacturers decide to stop research and development, or become complacent about safety improvements to their aircrafts. Imagine what would happen where it Google would decide to stop updating roads and addresses on Google Maps, we’d be driving on railroads or up the wrong side of the road in no time! (not to mentioned turning up late for Christmas lunches and Superbowl parties)
Unlike other trades like Architecture or Accounting, our duty to remain up-to-date with technological advances remains somewhat of an unwritten rule. Failing to do so can result in overly complex software architecture, inefficient and poorly backed-up databases, expensive hosting solutions, substandard downtime and a clunky or slow app.
In an age of copious news and accessible knowledge, all of these repercussions (and there are many!) are simply unacceptable.
Keeping a daily tech news routine is healthy, inspiring and just the right thing to do if we want to be in the best possible position to deliver quality to our customers. It is absolutely essential if we want to validate a proposed solution in a market with endless possibilities.
Product Hunt is a great way to keep in touch with what’s happening out there!
8. What already exists, and how can we use it to speed things up?
Many of our awesome software ideas have been done before. It’s simply a fact – for better or for worse, we’ve reached a saturation point in which great software companies have thought of almost everything within existing budgetary and technological boundaries – and developed it to near perfection.
Thankfully, much of these products come with something called an API (Application Programming Interface). An API allows other software to exchange data with an another existing product, thereby leveraging the latter’s established features, user-base, account authentication, processing power and so on.
Building a solution that already exists (unless it is to disrupt and radically improve upon it) is a ludicrous thing to do. Why spend months or years building something when we can simply integrate with an existing solution?
Integrations can help your product become marketable very quickly – and it will allow designers to iterate it into a usable standard in much less time than if they had to build all the solutions themselves. This presents us with a big advantage: We can validate software quicker because users can use a fuller extent of the tool.
9. Deploying in stages
I see many talented design teams which for some reason or another, choose to launch products in one ceremonious, potentially nerve-wrecking event. Eric Reis who wrote the brilliant ‘Lean Startup’ compares this act of insanity with launching software products like rockets, as opposed to evolving them like cars – with rockets you only get one shot. Why put up with the risk?
What I suggest is to develop a one-team mentality ahead of project kickoff. This means that there is no ‘us and them’ between us and the stakeholders. We are one team pulling for the same successful outcome. So from day one, we’ll explain that we work in an agile way, that we embrace rapid prototyping and package work in sprints, and that we deploy products in stages.
Shipping bits of software gradually alleviates a lot of the anxieties of launch, because it gives the one-team space to breathe, to change its mind and reconsider their options. It gives designers the time to react to critical UX erros, and developers to iron out high priority bugs. It helps us understand what the users like, what they don’t and what they simply don’t get. It helps us manage budgets, use time more sensibly and communicate updates or problems more effectively with our customer base.
In short, it helps us validate the product.
To use my favourite analogy: If your goal is to create the world’s best ham sandwich, focus on putting a world class piece of ham between two delicious slices of bread. Maybe add a touch of quality butter. Take your time to get that sandwich right before you move on to the mustard, the pickles and the tomatoes – because if your customers don’t like your mustardy creation, it will be a pain to identify the culprit.
Was it the acidity in the tomatoes?
Is the mustard too strong?
Are the pickles too much for their pallet?
Make a great, simple ham sandwich, then add the ingredients one after the other, testing and validating your additions every step of the way.
10. Learn to accept feedback, don’t fear it: You’re not building this just for you!
Let’s be honest, we all know the feeling.
Everyone is totally OK with criticism and absolutely devoid of ego until the first poor review hits the inbox (When it’s public, it hurts so much more!)
Deep down, a complaining customer wants your product to succeed, because they see how it can add value to their lives.
In the next few weeks, I’m going to write a post on how to seek feedback and encourage our customers to share it with us before they take to Twitter and vent their frustrations publicly. But for now, let’s focus on what you can do to handle feedback like a professional.
Firstly, thank the person who took the time to share their thoughts with you. As I said before, believe it or not most people will do this out of love for your company and product. Deep down, they want it to succeed, because they find the product adds value to their lives.
Secondly, read the comment from their point of view. Empathy is the backbone of good customer experience – if your instinct is to ‘fight’ and build a counter argument to neutralise the negativity, then resist it. It won’t do you or your product any favours, not to mention that you will very likely lose a customer for ever.
Finally, look at the feedback in the context of your overall product roadmap. Sometimes users will suggest things that are simply not in line with our longterm product vision. In these cases, thank your customer for their time and ongoing interest but explain that this may not necessarily add value to the community at large. Implement that which fits your plan in the long term and that your resources allow for in the short term.
A culture of openness will help us work along side our customers, and not simply for them. It promotes a sense of ongoing product validation that incentivises paid users to invest further in the product, and motivates our team to strive for higher standards.
In closing, let’s consider the one thing that does not change: and that’s change it self! At the heart of technology is evolution, and irrelevance is what we fear most when managing the lifecycle of a product – will it continue to be valid for years to come? If we apply a system that validates products and new features consistently, rather than only when the need arises, will guarantee a greater chance of longterm success.
See you next time! – Michael Azzopardi from Innform
To register for Innform Beta for free and try out the Ready-made courses, head to innform.io 🙂