Employee-Led AI Experimentation

BCG and Columbia Business School research reveals that employee-centric organizations are seven times more likely to achieve AI maturity than their peers — and that maturity is built bottom-up, not top-down. Yet 74% of companies can't scale AI beyond pilots, and 42% abandoned most initiatives in 2025 alone. The disconnect points to a fundamental strategic error: treating AI adoption as a technology deployment when it's actually a workforce capability challenge.

The question for your next planning cycle isn't whether to adopt AI. It's why your $50K pilot is being outperformed by the marketing analyst paying $20 a month for ChatGPT.

The CTO of a Fortune 500 financial services company recently described their AI strategy as "a graveyard of pilots." Over 18 months, they had launched 47 AI initiatives. Forty-three delivered nothing measurable. The remaining four showed promise but couldn't scale beyond their test environments. Total investment: $23 million. P&L impact: zero.

This story isn't unusual. It's the norm.

S&P Global's 2025 survey of over 1,000 enterprises found that 42% of companies abandoned most of their AI initiatives this year, up from just 17% in 2024. The average organization scrapped 46% of AI proofs-of-concept before they reached production. BCG reports that 74% of companies struggle to achieve and scale value from AI. The pattern is consistent across industries: ambitious pilots, substantial investments, and results that quietly disappear from quarterly reports.

Yet something curious is happening in parallel. The same companies watching their official AI programs sputter are home to employees who've figured out how to use AI effectively on their own. McKinsey found that 90% of knowledge workers use personal AI tools for work, even though only 40% have officially sanctioned access. These shadow AI users aren't waiting for permission—they're shipping better work, faster.

The question isn't whether your organization should adopt AI. The question is why the $50,000 pilot your IT department spent months planning is being outperformed by the marketing analyst who pays $20 a month for ChatGPT Pro.

Is the Model Context Protocol on your radar? Has it become a point of contention between developers keen to use MCP servers and security teams concerned about the lack of guardrails?

Stacklok is working with leaders across industries to bring the Model Context Protocol into production on a secure, scalable platform. Curate a registry of trusted MCP servers. Control auth via an MCP gateway.

Learn more at stacklok.com or join us at an upcoming MCP roadshow stop in San Diego, Austin, Atlanta, Boston, New York, or Chicago.

🎙️ AI Confidential Podcast - Are LLMs Dead?

☕️ AI Tangle - Ads Invade ChatGPT, Nvidia Hits $5T, and AI Starts Making Work Harder

🔮 AI Lesson - Build AI Skills That Remember How You Work

🎯 The AI Marketing Advantage - ChatGPT Just Entered the Ads Game — And Marketers Aren’t Ready

💡 AI CIO - Fresh Minds Outsmart the Experts

 📚 AIOS - This is an evolving project. I started with a 14-day free AI email course to get smart on AI. But the next evolution will be a ChatGPT Super-user Course and a course on How to Build AI Agents.

The companies winning with AI aren't running bigger pilots. They're running thousands of smaller experiments.

The enterprise AI playbook has followed a predictable pattern: identify a high-value use case, assemble a cross-functional team, partner with a major vendor, run a pilot, prove ROI, then scale. It's the same approach that worked for ERP systems, cloud migrations, and digital transformation initiatives.

It's also failing spectacularly with AI.

The pilot model assumes you can identify the right use case in advance, control the variables, and measure success through traditional metrics. AI doesn't work that way. The highest-value applications often emerge from unexpected places—the customer service rep who discovers a better way to summarize calls, the analyst who figures out how to automate their weekly reports, the product manager who uses AI to synthesize customer feedback at scale.

When 90% of employees are already experimenting with AI tools outside official channels, the pilot isn't discovering new possibilities. It's constraining them.

The data tells a clear story about what's failing. Organizations are spending disproportionately on the wrong priorities—over 50% of AI budgets go to sales, marketing, and customer operations, while back-office automation, which shows the highest demonstrable ROI, remains underfunded. Companies build when they should buy, with only 33% success rates for internal AI builds compared to 67% for vendor partnerships. And perhaps most critically, they focus on technology when they should focus on workflow redesign.

The Stanford Digital Economy Lab found that 42% of AI initiatives in 2025 were abandoned, with "model fetishism"—the obsession with having the newest, most powerful AI—cited as a primary cause. Teams spend months evaluating whether to use GPT-4 or Claude 3.5, even though the real bottleneck is whether anyone has mapped the process the AI is supposed to improve.

Consider the math. A formal AI pilot involves: stakeholder alignment (2-4 weeks), vendor evaluation (4-8 weeks), security and compliance review (4-6 weeks), development and integration (8-12 weeks), user acceptance testing (2-4 weeks), and training and rollout (2-4 weeks). Total timeline: 5-9 months before results are visible.

An employee with a ChatGPT subscription experiments immediately, iterates daily, and either proves value or moves on within a week. The feedback loops are measured in hours, not quarters.

This isn't about rogue employees circumventing controls. It's about the scientific method. Science advances through many small experiments with quick feedback, not through massive centralized initiatives. Breakthroughs happen when researchers can test hypotheses rapidly, fail cheaply, and iterate.

The same principle applies to AI adoption. The 77% of employees using generative AI at work—with only 28% having formal guidance—aren't being irresponsible. They're running thousands of parallel experiments that no pilot program could replicate.

The research on employee-led AI use is compelling. A Nielsen Norman Group study found a 66% increase in average productivity when employees integrated AI into their workflows on their own terms. A Science journal publication documented a 40% time reduction and an 18% improvement in quality for professional writing tasks. MIT research on software developers found a 26% increase in output, with effects concentrated in mid-level complexity work, where human judgment and AI capability intersect most productively.

But here's what matters most for long-term strategy: AI assistance increases employee creativity, not just efficiency. An Academy of Management Journal field experiment found that AI assistance elevated creative problem-solving in sales contexts, with the effect "much more pronounced for higher-skilled employees." Employees exposed to intelligent AI assistants report greater creative self-efficacy when organizational infrastructure supports experimentation.

This distinction—creativity versus efficiency—explains why the replacement approach is failing while the augmentation approach succeeds.

Companies rushing to replace workers with AI are learning expensive lessons.

Klarna, the buy-now-pay-later giant, provided a case study in aggressive AI replacement. CEO Sebastian Siemiatkowski bragged that AI bots were doing the work of "700 full-time agents" and stopped hiring for a year. The company partnered with OpenAI, slashed customer service and marketing headcount, and claimed $10 million in savings.

By early 2025, customer service ratings had dipped, user complaints increased, and Siemiatkowski was forced to acknowledge that "cost unfortunately seems to have been too predominant evaluation factor." Klarna is now hiring human customer service agents again and facing net losses of $99 million in Q1 2025—more than double the same period in the prior year. The company paused its highly anticipated IPO. Siemiatkowski now says: "There will always be a human if you want."

Duolingo followed a similar path. The language learning company cut 10% of its contractor workforce in January 2024 and laid off over 100 contract writers, translators, and curriculum experts. CEO Luis von Ahn announced AI would "eventually replace all contractors." Users complained that the content felt "repetitive, robotic," lacking the playful tone that made Duolingo iconic. Language teachers and linguists raised concerns about educational accuracy and cultural nuance. A massive social media backlash followed. Von Ahn walked back his stance, acknowledging potential quality impacts.

A survey of 1,000 UK C-suite executives by Orgvue revealed the pattern isn't isolated. Of leaders who made redundancies due to AI adoption, 55% admit they made the wrong decisions. The consequences included widespread internal confusion, employee turnover, and—notably—declines in productivity, the opposite of the intended outcome.

The Brookings Institution research tells a different story about what happens when companies invest in AI without the replacement mindset. Analyzing firms from 2010-2018, researchers found AI investments associated with 13% increase in trademarks, 24% increase in product patents, but only 1% increase in process patents. Companies were using AI for product innovation, not just cost-cutting.

The researchers concluded: "It does not appear the main use of AI has been to cut costs and replace human workers." The primary effect was sales growth and expansion through increased product innovation. And critically, the overall relationship between AI adoption and employment was positive.

This isn't about being soft on efficiency. It's about understanding where AI creates value. McKinsey found that workflow redesign has the biggest impact on AI's business value—not the sophistication of the model, not the scale of the pilot, but whether the work itself is reconceived to leverage what AI does well while preserving what humans do better.

The formula that emerges from successful implementations is consistent: 70% investment in people, process, and adoption; 30% in technology. Organizations treating AI as a technology deployment problem get the ratio backward.

Phase 1: Sanctioned Access (Weeks 1-4)

Provide official AI tool access to all knowledge workers. This isn't about control—it's about visibility. When employees use shadow AI, you can't see what's working. When they use sanctioned tools, experimentation becomes observable.

The security concerns that typically delay this phase are valid but often overstated. The employees are already using AI. You're choosing between invisible, uncontrolled experimentation and visible, guideline-bounded experimentation.

Phase 2: Use Case Identification (Ongoing)

Instead of top-down use case selection, establish mechanisms to surface what employees are already doing. Regular check-ins, internal showcases, Slack channels for sharing AI wins—the goal is to make successful experiments visible so others can learn from them.

The best use cases often come from unexpected places. The analyst who figures out how to automate a tedious weekly report. The customer service rep who discovers a better way to summarize complex issues. The product manager who uses AI to synthesize hundreds of feedback entries. These discoveries happen at the edges, not in pilot program planning meetings.

Phase 3: Scaling What Works (Months 3-6)

When experiments prove value, invest in making them systematic. This might mean enterprise licensing, custom integrations, or workflow redesign to embed AI assistance into standard processes.

But be selective. Not every successful experiment needs to scale. Some will remain individual productivity tools. Others will transform entire functions. The experimentation phase reveals which is which.

Key Success Factors:

Mistaking pilot success for scalability: A pilot in controlled conditions with dedicated support doesn't predict enterprise-wide adoption. The high failure rate isn't about failed pilots—it's about pilots that can't survive contact with real organizational complexity.

Underinvesting in workflow redesign: AI doesn't improve bad processes. It accelerates them. Before any AI implementation, map the current workflow and ask: if we were starting from scratch, how would we design this? The technology decision comes after the process decision.

Measuring the wrong things: Traditional IT metrics—uptime, user adoption, feature utilization—miss what matters. The right questions: What decisions are people making differently? What work is no longer being done? What new capabilities have emerged?

Pursuing replacement when augmentation creates more value: The research is clear—companies that position AI as a tool for workers outperform those positioning it as a replacement for workers. And the companies that tried replacement are walking it back.

The productivity evidence is substantial when AI augments rather than replaces:

Beyond productivity, the innovation case is stronger still. The Brookings research found that AI-investing firms saw a 24% increase in product patents. When AI handles routine cognitive work, humans focus on judgment, creativity, and complex problem-solving—exactly the work that creates competitive differentiation.

ROI Considerations:

Expect 2-4 year timelines for meaningful returns. Quick wins exist but sustainable advantage requires workflow transformation. Organizations reporting faster ROI typically had higher AI maturity before beginning—they were ready for the technology.

Competitive Implications:

The companies succeeding with AI aren't those with the biggest pilots or the most sophisticated models. They're the ones that figured out how to run thousands of small experiments simultaneously, surfacing insights from the people closest to the work.

Your competitors' advantage isn't their technology budget. It's whether their employees feel empowered to experiment.

The strategic implication is straightforward: stop treating AI adoption as a technology implementation and start treating it as a workforce capability challenge.

This means redirecting budget from centralized pilots to distributed access. It means building measurement systems that can see what employees are already doing with AI. It means accepting that the best use cases will emerge from experimentation, not planning.

For leaders who've watched expensive pilots fail, this is actually good news. The path forward doesn't require more investment—it requires different investment. Enable your people. Make experimentation visible. Scale what works.

The displacement fears are real. Entry-level positions are disappearing—skills atrophy when AI handles too much. But the companies that rushed to replace workers are retreating. The evidence favors augmentation, and augmentation requires human involvement.

One question for your next planning cycle: How many experiments are your employees running right now that you can't see? And what would change if you could?

You built the front end. Now make it work.

Join Jordan van Maanen from Make.com on February 24, Tuesday at 12 PM EST to learn how to connect tools like Softr and Zoho Forms to real automations using webhooks, data mapping, and multi-step workflows.

Stop building static apps. Start building automated systems.

I appreciate your support.