KPMG has deployed a dashboard that allows its consultants to monitor how frequently their colleagues use AI tools. The firm says it hopes the dashboard encourages more "frequent and sophisticated" AI use among the roughly 10,000 workers in its US advisory practice. The announcement is specific enough to be informative and general enough to reveal something that KPMG's leadership may not have intended: the firm has confused activity measurement with competence measurement, and in doing so has reproduced one of the most persistent errors in organizational learning theory.
Frequency as a Proxy for Capability
The dashboard tracks how often consultants use AI. This is a legible metric. It is also, on its own, a weak one. The assumption embedded in the design is that frequency of use correlates with quality of use, and that making frequency visible will drive both upward simultaneously. This assumption does not hold under scrutiny. What KPMG has built is a surveillance instrument oriented toward behavioral compliance, not a developmental instrument oriented toward competence formation. The distinction matters more than it might appear.
Algorithmic literacy research documents a persistent gap between awareness and capability. Workers who know that AI tools exist, and who use them regularly, do not automatically develop the structural understanding needed to use them effectively (Kellogg, Valentine, & Christin, 2020). Gagrain, Naab, and Grub (2024) distinguish between awareness-level literacy and schema-level literacy, where the former involves knowing a system exists and the latter involves understanding the structural logic that governs the system's outputs. KPMG's dashboard appears designed to increase the former. There is no evidence in the announcement that it addresses the latter.
The Competence Inversion Problem, Again
Classical organizational learning interventions assume that workers arrive with baseline competence and need nudges to apply it consistently. Leaderboards and usage dashboards make sense in that framework. The problem is that AI-mediated work does not fit that framework. As the ALC framework I develop in my dissertation argues, algorithmic environments invert the classical assumption: competence is not pre-existing but develops endogenously through interaction with the system. This means that pushing workers to interact more frequently, without building the schema that makes those interactions productive, may produce a false signal of organizational readiness.
Hatano and Inagaki (1986) draw a foundational distinction between routine expertise and adaptive expertise. Routine expertise is procedural: it involves applying learned sequences to familiar problems. Adaptive expertise involves understanding why procedures work, which enables reconfiguration when the environment changes. A consultant who uses an AI tool fifty times to draft client memos has accumulated routine exposure. Whether that consultant understands why the tool produces certain outputs, and can therefore adapt when the task changes, is a separate question entirely. The KPMG dashboard measures the first. It is silent on the second.
What the BCG Survey Adds
A recent BCG survey of 625 business leaders found that more than 60% of CEOs feel their boards are "rushing" AI transformation. This context matters when interpreting the KPMG announcement. Dashboards that measure AI usage frequency are, in part, a response to board-level pressure to demonstrate organizational commitment to AI adoption. They produce visible, reportable artifacts: usage rates, adoption curves, comparative rankings. These artifacts satisfy governance demands without necessarily addressing the underlying question of whether workers are becoming more capable.
This is not a cynical observation. It is a structural one. Boards operate at a distance from operational detail and reasonably rely on quantifiable signals. The problem, as Rahman (2021) notes in the context of platform governance more broadly, is that the metrics organizations use to manage algorithmic work tend to reflect what is measurable rather than what is consequential. Usage frequency is measurable. The quality of a consultant's mental model of how a language model processes ambiguous client queries is not. Organizations optimize for the former because the latter resists easy quantification.
A Measurement Design That Would Actually Work
This is not an argument against dashboards. It is an argument for measuring the right things. An instrument designed to develop genuine AI competence would need to track transfer, not just usage: specifically, whether workers apply AI effectively across task types they have not previously encountered. Gentner's (1983) structure-mapping theory suggests that transfer is evidence of schema acquisition, not procedural repetition. If KPMG's consultants are using AI more frequently but only on the same narrow task types, the dashboard is measuring habit, not learning.
The broader implication for organizational theory is that the current wave of corporate AI adoption is generating new versions of an old measurement problem. Firms are investing in instruments that make behavior visible while leaving competence opaque. Until organizations separate frequency metrics from quality metrics, the gap between reported AI adoption and realized AI capability will continue to widen - and boards will continue to receive dashboards that confirm progress without measuring it.
References
Gagrain, A., Naab, T. K., & Grub, J. (2024). Algorithmic media use and algorithm literacy. New Media & Society.
Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7(2), 155-170.
Hatano, G., & Inagaki, K. (1986). Two courses of expertise. In H. Stevenson, H. Azuma, & K. Hakuta (Eds.), Child development and education in Japan (pp. 262-272). Freeman.
Kellogg, K. C., Valentine, M. A., & Christin, A. (2020). Algorithms at work: The new contested terrain of control. Academy of Management Annals, 14(1), 366-410.
Rahman, K. S. (2021). The invisible cage: Workers' reactivity to opaque algorithmic evaluations. Administrative Science Quarterly, 66(4), 945-988.
Roger Hunt