Humans and AI, what (and how) do we think?

Today, many of us are questioning the pros and cons of AI ‘knowledge’ tools. They offer increased efficiencies, but they are also challenging our faith in (and understanding of) information, knowledge, and epistemological accountability. As knowledge workers, we find ourselves sandwiched awkwardly between the experience (and promise) of AI-provisioned comfort and the desire (and need) for human-provisioned growth. Navigating this emerging paradigm shift — in which our knowledge practices, productions, and consumptions are being re-invented before our very eyes — leaves many of us struggling to find the right balance of industry push, AI delegation, and individual integrity.

So, just how different is this new age of AI-supported thinking (or to put it another way, are we talking calculators or chain reactions)?      

Some scholars are suggesting that, in the light of this latest socio-technical evolution, we must now revise our fundamental understanding of human thought. Where once there was only System 1 and System 2 (gut instinct vs cool intellect), there are now suggestions of a third, described as System 3, or indeed System 0. These ‘third ways’ attempt to describe how humans are now thinking, reasoning, and decision-making with the aid of AI knowledge tools. And, more specifically, how these new integrated modalities have given rise to the need for a new understanding of cognition. The System 3 proponents talk not just of AI-induced human cognitive offloading and overriding, but also of cognitive surrender. This latter state is described as a transfer of control in which humans adopt “AI outputs with minimal scrutiny, and overriding intuition and deliberation”. And often culminating in “adopting the AI’s judgement as their own”. The System 0 proponents describe it slightly less alarmingly as “an artificial, non-biological underlying layer of distributed intelligence that interacts with and augments both intuitive [System 1] and analytical [System 2] thinking”. But they also go on to say this new form of thinking raises important epistemological and ethical concerns, such as dependency, shifting norms, and reduced critical thinking.

These thought provocations would suggest that AI information tools are not just another turn on the technological wheel, but rather represent a fundamental paradigm shift in our knowledge practices. Confronted with this significant re-appraisal of all that we hold dear, context, comparisons, and critiques can be helpful. For instance, it is worth reflecting on the reactions of some when the printing press was invented in the 15th century, the consequences of which were the dissemination of information, stories, and knowledge at a scale never seen before. Leading the charge against the monstrosities of the ‘mechanical movable type printing press’ were the Benedictine monks:

“They shamelessly print, at a negligible price, material which may, alas, inflame impressionable youths, while a true writer dies of hunger. Cure (if you will) the plague which is doing away with the laws of all decency, and curb the printers. They persist in their sick vices, setting Tibullus in type, while a young girl reads Ovid to learn sinfulness. Through printing, tender boys and gentle girls, chaste without foul stain, take in whatever mars purity of mind or body; they encourage wantonness, and swallow up huge gain from it” Fillippo de Strata.

Shifting from monastic mania back to the rigours of science, the System 0 proponents describe AI as an extended form of distributed cognition, the notion of which provides us with a useful structure to help contemplate the question of AI r/evolutions. The notion of distributed cognition has been around for a fair while, and in essence describes thinking that happens not just in an individual’s mind, but that eventuates as a product of thoughts distributed across other people as well as other tools and environments. Examples of distributed cognition range from the humble shopping list compiled collectively via post-its on a fridge door, to the confident GPS system (sometimes) taking us down awkward lanes and unhelpful cul-de-sacs. According to this definition therefore, AI is just another tool that is being harnessed by our well-practiced distributive thinking habits, thereby falling more into the papyrus to printing to predictive algorithms pattern rather than the paradigm shifting need for an entirely new model of human thinking.

But does that really cover it? Is the pervasive integration of a single thinking tool that helps us do anything from managing our diaries, to writing reports, sending emails, coding websites, advising on lifestyle choices (we could go on), is this merely an amped up form of distributed cognition? Or is it, as the third system theorists suggest, doing something more?

The answer is (as ever) yes and no. AI does represent a form of distributed cognition, but no, it’s not as we’ve ever seen it before. And (ironically) in reality AI is actually a somewhat undistributed form of distributed cognition. Because distributed implies a diversity of informational options — across people, objects and place, whereas AI information, although trained on the oh-so-wide world web, has been finely milled, ultra-processed, and artificially reconstituted according to the directives of a handful of extraordinarily well-funded (not to mention geo-politically powerful) technology companies.

And there are other things that also set AI apart from the more established processes of distributed cognition. For instance, the persuasive nature of generative AI’s anthropomorphic conversational interface means that we as humans can’t help but speak to it as we would another human (just think about that uncomfortable feeling of social dissonance brought on when attempting to remove conversational politesse to save on time or tokens!). And we know from both research and headlines, that dialoguing with an AI agent over sustained periods of time produces outcomes similarly associated with human-to-human interpersonal relationships, including emotional investment, reliance, and identity fusion.  

So, to return to the question of calculators or chain reactions, the answer might be better contended with from a more retrospective viewpoint. All technologies have the potential to be disruptive, but their disruptive potential only exists in the minds and (subsequent) actions of their human users. Understanding the potential for AI disruption, whether to military operations, traffic management, or indeed, the foundations of human thought requires (ironically) human thought. And given what we know about the impact of confounds on empirical truth-seeking, ensuring that we investigate the question of human thought before full cognitive surrender, would appear to be wise.