The ML Democratization Paradox.
More important than using Machine Learning, apply its Core Principles
Ignorance of the basics
Through the popularization of Tools like ChatGPT, society is experiencing an unprecedented democratization of machine learning. Users in Companies across the globe routinely leverage sophisticated ML algorithms through native-language, easy-to-use prompts and often rely on the truth derived from predictive user behaviour analytics in CRM systems or to generate recommendations in marketing platforms.
As in the age when personal computing became popular, given the ease of use, we eagerly consume the outputs of the machines but forget and later or even learn anymore the fundamental principles that make these machines effective. In the era of Artificial Intelligence and the underlying building blocks, the most valuable insights from machine learning are probably not the models themselves, but the conceptual frameworks that they are based on.
These frameworks—for example, the concept of loss functions—are a kind of basic toolbox for data scientists and AI architects, but this knowledge isn’t distributed across the organization. Given the ignorance, many Leaders of vital functions in organizations rely, for example, on fundamentally flawed data-gathering methods that would be immediately rejected in any machine-learning context.
For example, the Loss Function is a nearly perfect measurement method
Every machine learning algorithm relies on the loss function. This simple concept from stochastics represents one of the most powerful measurement frameworks ever developed. A loss function isn’t based on subjective questions or a more or less biased sentiment, but rather measures the precise gap between actual and desired outcomes, providing an objective and more important actionable measure of performance.
When an ML algorithm needs to predict house prices, it would never ask a homeowner how much they think this home is worth; instead, it measures the difference between the predicted and actual prices, weights those errors by their business impact, and, with each run, minimizes that gap. The result is an incremental improvement that is easy to measure and much more objective than subjective perception. This same principle—measuring deviation from optimal outcomes rather than collecting Intentions and opinions—could revolutionize how we measure, for example, customer satisfaction.
Customer Satisfaction Measurement and its questionable foundation.
While ML algorithms optimize for rigorous loss functions, many business leaders and decision-makers continue to rely on metrics that would immediately be rejected in any ML context. For example, both Net Promoter Score (NPS) and Customer Satisfaction (CSAT), which we want to observe given their ubiquity and flaws:
The NPS Problem:
While pragmatic, because based on a single question: "How likely are you to recommend us?", the score is considered a kind of holy grail for great customer satisfaction. However, it suffers from a fatal weakness: It ignores the cultural context; Germans rate completely differently from, for example, Brazilians. Additionally, it ignores the action that actually drives value (a "promoter" who never buys again is worthless), and even worse, it provides absolutely no hint or insight into what or where to improve. Most critically, NPS measures only hypothetical intention rather than actual behaviour. In the ML world, it means training an algorithm on survey responses rather than on outcomes.
The CSAT Problem:
CSAT asks customers to score their satisfaction on a 1:10 scale. Beyond the already discussed cultural and contextual biases, CSAT fails to weigh what matters. A customer "satisfied" with a minor feature receives the same weight as one dissatisfied with core functionality. Positive and negative scores are just summed, and a 0 counts the same as a 6 as one negative score point. There's no loss function, no optimization target—just an average of opinions disconnected from the real business impact.
Both Methods represent measurement theater: the appearance of rigor without substance is quadrupling the negative impact of Goodhart's Law - When a measure becomes a target, it ceases to be a good measure.
What a Loss Function Approach could look like;
When applying the loss function to measure customer satisfaction, you would measure the gap between customers' actual behavior and optimal behavior. A potential model could look like:
Behavioral Deviation Index:
- Expected repurchase rate vs. actual repurchase rate
- Predicted lifetime value vs. realized value
- Target product utilization vs. actual utilization
- Baselined quantity support tickets vs. actual support workload support
- Expected referrals generated vs. actual referrals collected
Ideally, each of those deviations would be weighted by its business impact. For example, losing a high-spending customer (high weight) would generate a much larger "loss" than losing a one-time shopper or similarly underutilizing a minor feature (low weight). The aggregated view would provide an objective measure of how far your customer experience deviates from optimal—and more importantly, where you're losing the most value. So, you would measure reality, not perception.
A way out
It seems that complex mathematics, such as the stochastic methods that underpin machine learning, are still confined to the working spaces of data scientists and ML engineers who understand loss functions, gradient descent, and optimization. These specialists, often considered nerds, build the systems that learn and improve automatically, but these conceptual frameworks are not spreading beyond the borders of the narrowest technical domain.
Business leaders, meanwhile, have become heavy consumers of the outputs without absorbing the underlying principles. They trust, for example, ML-powered recommendations for inventory management, but at the same time continue measuring customer satisfaction as if the last 30 years of ML development never happened.
The tools have been democratized; yet the thinking has not. This represents a massive missed opportunity.
The insights that make ML robust — rigorous measurement of deviation from targets, automatic weighting by impact, continuous optimization, and they shouldn’t be confined to the Technical labs. They are conceptual frameworks that could transform business measurement across domains.
We stand at the Planckpoint.
Machine learning has given us both powerful tools and powerful conceptual frameworks. We've enthusiastically adopted the former while ignoring the latter. There is a profound irony: Organizations are trusting ML algorithms to optimize, for example, inventory and personalize marketing, yet the same organizations measure most essential business outcomes using methods that no ML Developer would ever accept as valid.
The solution is not to augment the AI/ML “Tool-Party”, but rather to systematically apply the founding principles to business measurement. Loss functions work because they measure quantifiable, objective gaps and weight them by their impact. That same logic should apply when organizations are measuring customer satisfaction and employee engagement. The question is not whether we can apply these principles beyond the Labs—it's why we haven't already.
The democratization of AI and ML will be complete not when everyone can use the tools, but when everyone applies the architects' thought process who have built them.
The evolution in business measurement will come from borrowing the mindset, not just the models, of machine learning.
