Vanity Metrics in Innovation: Evaluating Impact
Vanity Metrics in Innovation limit industries’ ability to support, evaluate, and celebrate real innovation. One company may define innovation as merely pivoting to step 1.1 (from 1.0) where others may need leaps and bounds to 3.0 before claiming innovation has been achieved.
The first is inevitably short-sighted where the other may create the next visionary product but fail to have an existing problem to solve. The reason this happens is because there is often no threshold or bold definitions for innovation to guide the next great leap.
Vanity metrics can be associated with core innovations but are unrelated to the numbers that matter (TechCrunch, 2011).
Change by definition is not innovation. Innovation is more than just a new idea, process, or product. It represents a new solution to an exist problem, or better yet…..a new solution to a problem no one realized even existed.
Solutions become outdated as competitors develop and introduce better solutions. But the problems that until now have had no solution continue to represent the largest opportunities for innovation and to make an impact.
Projects and investments to support innovation need to have a context to evaluate impact, relevance to add value to the customer, and interface with available processes and assets and provide additional solutions. Businesses need to move beyond vanity metrics and identify goals and measures that create real impact.
Innovation is not useful if created, evaluated, and implemented in a vacuum.
Evaluating Value
| Context: | Is an incremental gain in the current environment while or following the industry a display of innovation? |
| Relevance: | Is developing a new, or expanding upon existing, innovation that solves no present customer problems a display of innovation? |
| Interface: | Is developing a new functionality that is scalable but lacks customization, personalization, and cannot be used by the target market? |
These questions are raised with consideration of current claims of innovation in the healthcare, technology, policy, and digital space. But where is the impact? True innovation requires cutting edge leadership.
The Social Value Portal provides the following definition for evaluating impact:
“Social Value refers to wider financial and non-financial impacts of programs, organisations and interventions, including the well-being of individuals and communities, social capital and the environment (2017).”
Measuring Value
The Social Portal provides the following criterion for evaluating innovation:
| Is it objective and verifiable – will an independent audit arrive at the same answer? |
| The issue here is whether or not the solution provided by the product/ service offering actually solves the problem. Is it actually a solution or is it merely another incomplete answer to an existing problem? Is the solution accessible, relevant, and effective or does it create new problems by its implementation? Does it leave gaps in the solution? |
| Does it use financial proxies – is it reliant on attributed financial values and if so are they universally agreed? |
| The issue here is whether the financial KPI’s attributed to the solution are agreed upon by all stakeholders. Are financial values aligned? Is the product/ service offering affordable? Sustainable? Do they improve accessibility or create a barrier? |
| Is it sustainable or is it too costly to undertake? |
| The issue here is the solution’s impact remains over time or if a dependency on the product/ service offering is created? Is the necessary ecosystem in place to sustain access to the solution overtime, if needed? |
| Can it translate into other metrics for comparison purposes? |
| The issue is if the impact can be confirmed by related and interdependent KPI’s? Or are other essential key performance indicators compromised? Are relative gains and costs isolated or distributed throughout the ecosystem, with some customers benefiting more than others? |
| Can you benchmark with it to compare performance between organisations? |
| The issue here is if the product/ service offering would create more gains in other business models, environments, and/or contexts? |
| Is it compliant to legislative and industry frameworks? |
| The issue here is whether the product/ service offering is aligned with the higher level values, ethics, and priorities of legislative and industry frameworks. This also prioritizes context fit where the product/ service offering will have enough access to the industries, markets, businesses, and customer groups it seeks to serve. |
| Does it cover all aspects relevant to your business such as, the environment, people, monetary value or hyper-locality? |
| This issue here is whether the product/ service offering is developed, and promoted, with consideration of its environmental, social, and human impact. Or are negative consequences created that need more solutions? Are the greatest gains received by business investors or by customers? |
| Is it applicable across public, private, third and community sectors? |
| The issue here is whether the benefit of the product/ service offering is available to all relevant sectors. Is it only available to specific sectors? The highest bidder? Or limited due to homeostatic mechanisms that seek to support the status quo? |
Conclusion
The definition of innovation takes on significant social characteristics that influence the construct’s impact, relevance, and accessibility. When every change, whether forwards or backwards, is celebrated as an innovation the ability to evaluate return on investment (ROI) is diminished. Distinguishing between vanity metrics and innovations that have a real impact is crucial for evaluating opportunities for further research and development.
Vanity metrics distort impact and by definition are unrelated to core innovations. Strategic efforts to support innovation require a structured approach that incorporates key metrics, formal strategies, delineation of roles, and effective benchmarks. Although all modifications along the innovation development curve can potentially add value not all have a direct impact.
Travis Barker, MPA GCPM
Innovate Vancouver
Consulting@innovatevancouver.org
Innovate Vancouver is a Technology and Business Information Consulting Service (TBICS) located in Vancouver, BC. Contact Innovate Vancouver to help with your new project.
Resources:
What is Social Value. (2017, September 24). Retrieved December 21, 2017, from https://socialvalueportal.com/what-is-social-value/