Can Industry can learn from Public policy and bureaucracy led best practices

As I get a chance to reflect on the three decades of working in the corporate world and last year in Public policy, get to realize that both these worlds appear to be so very different, yet they are so much more alike. There are a number of things that Industry and the Corporate world can learn from the Public Policy and Bureaucracy driven best practices.

I have delivered many complex projects across computer systems, hardware, software, products, SI, networking, communications, consulting and with many projects converging all these elements into a single program. I have also had the privilege to work on future of technology, Future of Industries and future of consumer behavior (have written about these multiple times in my previous publications) under the Technovation charter during the last 15 years. A looking back exercise makes a lot of sense today.

As I find myself analyzing how these projects could have been executed with greater efficiency and effectiveness, the introspection brings a blend of emotions: a mix of confidence from accumulated experience and a subtle awareness of the ways things might have been approached differently. I am happy to share some of my learnings, observations and best practices.

In the pre-digital days, when most applications and users were behind the firewall and the use cases were much simpler. Those were the 80386, Novell, Yahoo, Window NT days. The developmental technologies were complex labor intensive, and code reusability factors were limited. We were dealing with structured data, and leveraged a ‘system of records’ approach user engagement. The data was historical, helped generate insights for forecasting, decision making and planning. Novell Networks, MS Backoffice and ERP systems helped achieve the economies of scale, but need trained tech personal at every touch point. This sort of summarizes my first decade- years between 1992 to 2000.

The Internet and eCommerce then brought in a new perspective to tech adoption- observed most between 1997 to 2005. In the later years, adoption of Mobile, Cloud and edge technologies led to an expanded and boundaryless data center. Social media had a big role in tech percolation. In the last few years, convergence of technologies along with IT and OT integration led to increased democratization of tech in the industry. This is ultimately and gradually leading us to the Digitally powered world in the coming years. My next two decades between 2001 and 2020 were all anchored on these learnings.

Over all these evolutions, years and multiple transformations, we realize that even though we have strong frameworks and project management best practices, the average failure rates of tech projects are high. Adopting agile and DevOps practices has been shown to reduce failure rates, but a large number of projects still struggle due to organizational and cultural barriers, lack of training, or resistance to change. Improvements in areas such as increased project management awareness, better-defined success metrics, and stakeholder engagement, have helped improve success rates, though failure remains a significant risk in software development. Studies show that approximately 70% of software projects face some form of failure, which can range from partial to complete. "Failure" here includes projects not meeting goals, missing deadlines, or exceeding budgets. The Standish Group's CHAOS Report frequently cites a 20-30% rate of projects that are abandoned or fail to deliver any usable output.

Bureaucrats and Public policy driven projects are successful

In case of Public Policy driven projects, even before technology being digitally adopted, Bureaucracy led public policy programs inherently leveraged a ‘system of engagement’ mode. Bureaucrats have always been ‘hands-on’, and ‘always-on’ mode of engagement. There is a level of passionate involvement that drives the Officers, being both purpose driven and visionary thinkers. Resilient Leadership does make a big impact on Public policy, and many decisions directly impacting life and death situations at population scale.

In todays Digitally transformed environment, important fundamentals drive success-

- Data transparency, data volume, data variety and data velocity along with stakeholder involvement and engagement has always been high, Digital and augmented technology can drive transformational benefits. Public policy systems are better equipped on this front.

- System of engagement, with the entire workforce working towards the end users at all times. One thing that stands out in Public policy is the level of understanding of the behavioral dynamics of common citizen the Bureaucrats have. Their ability to keep things simple, such that it is adoptable by the most illiterate of the society and the Constitutional need to think and address population scale as a fundamental driver is what keeps the lights on in our surroundings today. We have seen the negative impact in Countries when these systems' fail- be it in Bangladesh or the most recent unfortunate incidents in Spain.

Another point I want to touch upon is the pace of activities. While business world appears to be fast paced, the public policy systems can grow up to population scale exponentially and in very short bursts. The speed at which things happen and the volume activities at population scale can impede stability. Be it a challenge, a pattern or a solution, in Public policy systems, things spread speedily and scales to population impact.

There are other reasons that have resulted in improved success rates of the large transformational Public policy projects-

1. Policy Continuity and Political Backing: Ensuring consistent political and administrative support
2. Bottom-Up Reforms: Empowering local-level officials and involving them in planning improves buy-in and ensure reforms are tailored to practical needs.
3. Use of Pilot Programs and Gradual Scaling: Testing reforms through pilot projects before national rollout help identify challenges and make adjustments, increasing the chances of successful adoption.
4. Increased Collaboration with the Private Sector: Public-private partnerships brings in expertise, technology, and efficiency needed to manage large-scale bureaucratic changes.

Effort and cost creep are ongoing risks in large-scale projects, often driven by evolving requirements, unexpected complexities, and the challenge of aligning diverse stakeholders. As users adapt to new systems, unintended outcomes—both positive and negative—often surface, requiring refinements or adjustments to ensure the intended benefits. Incorporating flexibility for iterative improvements, setting clear and measurable objectives, and maintaining open communication with stakeholders can help manage risks. Hence, a value driven benefits model (refer my earlier post) can become the Future of Governance.

In conclusion, today’s industry ecosystems are uniquely positioned to adopt and benefit from best practices refined over decades, especially in a post-pandemic landscape that has accelerated digital transformation. The rapid shift to digital has heightened the need for resilient, adaptive frameworks that support seamless integration across various domains. By learning from established strategies in Public Policy, industries can not only enhance their operational resilience, but also drive innovation through improved collaboration for population scale transformations.