Public V Private Sector Innovation – The Basis For Success
A lot has been written over the years about both public and private sector innovation. Following the wave of managerialist reform in the 1980s and 90s it has been widely believed that the Public Sector could improve its innovation performance by looking to the Private sector. That is not the conclusion we drew from a recent comparative study of Private Sector CEOs and Public sector heads of agencies experience of innovation. Innovation was commonly pursued for different reasons irrespective of whether in the public or private sector. The approach adopted differed primarily based on the degree of uncertainty presented by the environment and whether the innovation was in response to an unexpected situation or part of a deliberate repositioning. The means available for being proactive, as well as the options available for managing uncertainty in the different contexts, most explained the difference between the sectors and the likelihood of a successful outcome.
The backdrop to the debate
Over the past few decades public sector innovation has been a hot topic in many countries. This has been in response to rapidly changing national and global circumstances requiring increased innovation in both policy and delivery to meet the needs of diverse stakeholders within limited budgets. While the need for innovation has increased there is a general perception that the public sector lacks the capacity to deliver it. This perception has been reinforced in the research literature, with the public sector frequently characterised as conservative, bureaucratic and reluctant to change. However, much of this past commentary has been based more on opinion (and perhaps a little prejudicial stereotyping) rather than solid evidence. There have been few direct comparisons made between the public and private sectors approach to innovation and none that considered both successful and unsuccessful innovations. As with all areas of public management, innovation in the public sector has been influenced by changing ideological conceptions of governance and public management. The New Public Management (NPM) of the 1980’s widely advocated the adoption of private sector management principles in Government. One of the implications has been a focus on the similarities between the public and private sectors in their approache to innovation, rather than the differences. We sought to understand what is unique about the public sector and what implications this has on the approach to innovation most appropriate to the public sector context.
How we did it
We collected 84 stories of innovation from the 25 CEOs and 20 Public Sector leaders (generally heads or deputy heads of Government Departments). Forty two of these stories were of innovation experiences which were successful and a further 42 unsuccessful. Detailed qualitative analysis was then undertaken to identify patterns within and between these stories. What this analysis overwhelmingly revealed was that, regardless of whether it was the Public Sector or the Private Sector, the way the leader thought about innovation was driven by the context they found themselves in and the problems they needed to solve – not some higher meaning or concept of innovation. Understanding and accounting for the context in which the innovation occurs is therefore crucial to the adoption of the best approach. The stories were drawn from a wide variety of contexts so we looked for those contextual characteristics that were common. Two characteristics emerged:
– The Level of Uncertainty the CEO/Head held about both their organizational situation and the environment it was operating in; – The Level of Pro-activity inherent to the CEO/Heads situation – whether the innovation was part of a planned strategy or a response to external triggers that needed to be incorporated.
Public V Private: What are the differences?
The first and most obvious difference was the existance of three quite distinct approaches to innovation in the private sector. Following the wider literature we labelled these incremental, evolutionary and revolutionary. The public sector, howerver, only displayed two, which we have called:
– Ministerial: innovation that occurs through interaction with and on behalf of the government’s political appointee; and – Departmental: innovation that occurs within a department and has been initiated internally and led internally.
Interestingly, and contrary to what many might expect, relatively few Public Sector innovations could be classified as incremental – characterized by low levels of uncertainty. This may reflect the generally more complex environment which the Public Sector confronts – particularly the diversity of stakeholders and interests which must be managed during any change to existing process. Secondly, the private sector interviews showed that the approach taken by the CEOs to different types of innovation can have a significant impact on the likelihood of success or failure. The same can be said of the public sector but the reasons for this are completely different.
Irrespective of whether a private sector CEO was reacting to an organizational circumstance or proactively innovating there was little difference in their likely success or failure. In the public sector the difference was dramatic. Indeed there was only one successful innovation from a reactive context in the public sector. Conversely the complexity or uncertainty appeared to have little impact on success for the public sector indeed the public sector had more revolutionary successes than failures suggesting a well developed innovation capability when circumstances are right – a finding which challenges those negative stereotypes!
There is a case for comparison or benchmarking between ‘Departmental’ innovation and the private sector. However ‘Ministerial’ innovation, presents such a significantly different innovation context that comparison with private sector approaches is of limited value. For example, comparisons are sometimes made between the role of the Board and that of the Minister and Government in terms of oversight of executive functioning. When it comes to innovation, the Board will generally take its lead from the corporate executive. In the public sector, in addition to performing an oversight role, the Government is an important source of innovation initiatives. Departments have an obligation to pursue political initiatives and these may be introduced with relatively little advance warning and with limited scope for modification or adaptation at the Departmental level. Consequently, public sector managers are far more likely to find themselves reacting than are their private sector counterparts.
A further and particularly significant difference is that the private sector assumes and accepts that failures are a normal part of innovation. The failures are acceptable as long as the successes outweigh the losses from a commercial point of view. This is reflected in the use of probability based approaches – an approach completely absent in the public sector profiles. In the private sector, return on investment is the ultimate measure of success. In this context, speed to market can be more important than a perfectly implemented idea. Removal of all uncertainty associated with the idea is a luxury that it cannot always afford nor indeed always need. By contrast, failure is not acceptable in the public sector due to the attendant political risks.
Historically the public sector, in many Western Democracies at least, has been very successful in the implementation of quite complex and revolutionary innovations – not least the extensive reforms of the 80s and 90s. However, it has arguably succeeded because it can use time as a resource to reduce uncertainty in a way that the private sector cannot. Innovation in the public sector then is highly sensitive to time and the quality of the idea, in a way that does not exist in the private sector.
It is significant then that of the thirty public sector stories collected we only had one successful story where the innovation was initiated in a reactive context. To put it another way, where the public service had little influence over the idea or the timing of the implementation, the chances of failure were substantially increased. The concern is that the public services in many countries may increasingly be confronting an innovation environment where reduced influence over the nature of the idea and the timing is the norm. The implication of this is that it removes some of the key strengths of public sector innovation, by reducing the time taken to implement complex public policy, and the ability of the public service to temper bad ideas through the reduction of uncertainty. If this trend is believed likely to continue, new models are needed designed to deal specifically with this environment.