- 1 Abstract
- 2 Introduction
- 3 1. The view from AI
- 4 2. The functions of intelligence
- 5 3. The structure of intelligence
- 6 4. The development of natural intelligence
- 7 5. So is artificial intelligence possible?
- 8 6. Conclusions
A nearly complete essay on artificial intelligence, which is full of (to me) interesting things but which degenerates into a notebook that even I can’t quite follow towards the end.
Pretty much everything that matters about the cognitive mind that we have learned in the last fifty years or so was taught us either by Chomsky or by Turing.
Jerry Fodor, Times Literary Supplement, 13 September 2002
Computers are useless. They can only give you answers.
This essay is designed to refute the view that a true (‘strong’) artificial intelligence could be based on computation. It is argued that the essential shortcoming of computation is its inability to deal with certain normal intelligent functions, notably insight and criticism, against which the mathematical logic of computation is compared. The reason why computation is incapable of this is that no system or artefact that is controlled mathematically can simultaneously obey the rules of mathematics (such as that the meaning of terms cannot be changed in the course of a mathematical structure’s execution) and also be capable of insight or criticism (both of which require that at least some terms change their meaning). It may be true that any stable result of intelligent activity can be described mathematically, but all intelligent activity and experience consists at least partly in the production of new and unpredictable outcomes.
There aren’t many truly great moments in history. For me, perhaps the greatest so far was the day human beings landed on the Moon, as if saying to the universe at large, ‘Here we are!’ So very different from, say, the battle of Actium (where civilisation nearly turned its back on western Europe) or the invention of the printing press or computer. They were just human beings talking to one another, whereas when Neil Armstrong stepped off that ladder and onto the Moon, humanity stepped out onto a completely different stage. There aren’t likely to be many more moments like that, if only because most of the really important changes we can look forward to – bodily immortality, universal enlightenment, that sort of thing – are more likely to be the results of gradual processes than discrete accomplishments.
But there is at least one more truly historic event we can predict with some confidence: the day we first look an alien species in the eye, and it looks right back. What is more, it may well be in a laboratory on Earth rather than in deep space or through a radio telescope that another intelligent species first declares, ‘And here we are too’.
The search for artificial intelligence (and here I am concerned solely with ‘strong’ AI – the creation of artificial intelligence properly so called) is undoubtedly one of humanity’s greatest quests. Given the logic of the relationship between intelligence and technology, it is also highly likely to come to a successful conclusion (Robinson 2004). However, like any other adventure, it is unlikely to succeed unless the adventurers take along a clear enough image of the grail they are looking for, or at the very least a map of where to find it. In fact, starting out with such a picture or map is especially important in this case, given that these particular adventurers are not looking for something that already exists; rather, artificial intelligence is something they are trying to make. So we need, if not a blueprint or a photograph of intelligence, then at least a definition. After all, we did not have the luxury of starting research into artificial flight armed with blueprints for a Boeing 747, but it is quite certain that everyone who ever tried to build a flying machine had a more or less clear idea of what it meant to fly.
Where then is the definition of the intelligence that the AI community is trying to build? As far as I am aware, there are three general kinds of definition currently in use: the partial (intelligence as language, as complex computation, as…), the trivial (the Turing Test, chess-playing machines, and so on) and the wrong (various allegedly intelligent tasks). And even that leaves out the very large proportion of AI research – perhaps the majority — that starts from no clear definition of any kind. A few bullet points, perhaps, or maybe a task that an intelligent being would surely be able to perform (by no means the same thing as an intelligent task); but a credible definition of intelligence? As often as not, not.
Once upon a time this need not have been a fundamental problem: until the early twentieth century intelligence might well have been best described in terms of a list of skills or perhaps as too unknown to have a prior definition. But unfortunately neither of these positions has been tenable for many decades – indeed, since before the invention of the computer.
First and foremost, serious scientific conclusions as to the nature of natural intelligence have been well established since at least the 1920’s, as a result of which we have a very clear picture of what natural intelligence is. This naturalistic research continues to proceed with vigour, to generate new and original results and above all else to focus on the real activity of demonstrably intelligent entities, namely human beings.
Furthermore, this research strongly suggests that there is no list of specific abilities can be considered as inherently representative of intelligence. Any form of activity can be considered intelligent if it is done intelligently, and the fact that an action is done by an intelligent being is no evidence of that actions’ intelligence when performed by another entity, artificial or otherwise. As intelligent beings, human beings are much better at mathematics than any other being with which we are acquainted, and for a long time the great exhibitions of mathematical ability (such as Euclid’s Elements) were held up as paragons of intelligence’s accomplishments. However, a pocket calculator can perform most aspects of mathematics far better than any human being – once, that is, the underlying reasoning has been performed by the mathematician, and then implemented by the engineer, who together make it possible to simulate mathematics in a machine. More generally, although indispensable to any complete notion of intelligence, neither the concept nor the substance of intelligence is exhausted by its functional skills and its empirical knowledge, nor even by its systems of culture and technology, no matter how vast or subtle the latter may be.
Despite this, the AI community has always proceeded as though intelligence were all but unexplored territory. This has led them into the truly fundamental error committed by all those who start out on a journey with no map and no clear destination in mind: they end up going around in circles. Indeed, AI researchers have generally operated in a doubly inappropriate manner: by applying the means that computers make available to commonsense notions of what it means to be intelligent. As a method for developing preliminary intuitions about what intelligence is all about it’s not a completely absurd approach, but as a research strategy in a field where the basic definitions were well established long before AI was even imagined, it makes no more sense than trying to create artificial life by applying the methods of gardening to the average vicar’s concept of life.
Nor is the situation saved by appealing to concepts of intelligence derived from so-called ‘cognitive science’. After all, this discipline is very much the product of investigating broadly intelligent forms of activity by means of models derived from computing. But that is only to say that it assumes the very thing that AI research sets out to discover, namely whether computational methods are capable of implementing intelligence proper. No wonder that it is such a productive strategy – if productivity is measured solely by the volume of research generated. Yet this approach does not seem to have got us much closer to a credible idea of what it means to be intelligent, or to the creation of a genuine artificial intelligence. As I shall argue at length below, this should not come as a surprise: if the purpose of cognitive science is to create tools for creating a genuine yet artificial intelligence, then to define and study intelligence in terms of the logic of computers simply forces AI research into a vicious and ultimately sterile circle.
Before starting my account of AI, I should repeat that it is only with ‘strong AI’ that I am concerned here. And in strong AI, the ultimate objective is surely be to reproduce in artificial form the essentials of natural intelligence. This in turn defines the central question when evaluating the claims of AI, namely the extent to which its creations do indeed reproduce the original in some fundamental sense. The definition of this ‘fundamental sense’ must be taken from the nature of intelligence rather than from any assumptions about predefined methods or acceptable results (e.g., that it must be computable), and this in turn needs to be traceable to the features of human nature we are most willing to describe as intelligent.
Of course, it is highly debatable to what extent an artificial intelligence would be like a human being. Human beings may be the best instance of an incontestably intelligent being currently available, but it is most unlikely that it is our humanity that defines our intelligence. On the contrary, it is surely our intelligence that defines our humanity. Nevertheless, it will probably only be when we have either met our first Little Green Man or produced a genuine artificial intelligence that we will be able to distinguish unequivocally between intelligence in general and human beings in particular. So although creating artificial human beings would not be a very sensible goal for AI research, we’d need a pretty compelling reason to start from anywhere other than intelligence as found among ourselves.
1. The view from AI
Like many under-defined disciplines, AI seems to constantly veer back and forth between two superficially complementary yet somehow mutually inextricable poles. At one extreme it returns again and again to the empirical and functional features of the intelligence it is trying to build. This takes the form of various attempts to create the impression of intelligence, as though that would be the same as creating intelligence as such. In other words, the search for a strong AI – one that is spontaneously capable of fully intelligent activity and experience – is defined and tested in essentially weak terms – i.e., the ability to simulate this or that expression of intelligence. That this is an invalid elision would be inescapable were it not for the fact that the difference between these empirical and functional surfaces and ‘intelligence proper’ is often expressly denied. The archetypal form of the empirical cum functional test for intelligence is the Turing test, which, although often criticised, has yet to be transcended. This ‘test’ is based precisely on the assumption that if one can give a successful impression of intelligence then one has created intelligence itself.
But this is by no means the only strategy for AI. There is also an equally important school that concerns itself with the real depths of artificial intelligence, on the core logic and ultimate rationale for research of this kind. As with the functional-empirical school, there are many sects, but all take as their starting point a single great argument that was advanced even before the dawn of AI research (but which also takes us back to Alan Turing), namely the ‘Church-Turing thesis’.
Turing’s appearance on both sides of this equation is by no means a coincidence, for as I shall argue in this section, the Turing test, like all other functional-empirical tests, is a completely inadequate measure of human or any other kind of natural intelligence; but at the same time, the reason why AI models of intelligence never escape from functional-empirical models of this kind is that the underlying conceptual framework derived from the Church-Turing thesis is, like any computational model, incapable of defining a non-functional, non-empirical test.
1.1. The Turing test
Alan Turing set out his famous test for intelligence in 1950. Leaving aside the question of exactly what the test really means – and this is by no means universally agreed – it has generally been interpreted as saying that if you cannot tell the difference between a human being and a computing system (given that your access to both is limited to a question-and-answer session conducted via a computer terminal), then there is no difference: the machine is as intelligent as the human being.
Turing’s personal expectations in this regard were low: he expected that by the year 2000 machines would be able to deceive human beings for five minutes at least seventy percent of the time. However, leaving aside the inability even now of machines with vastly greater processing power than Turing could ever have envisaged to pass this test, it must be recognised that it is actually a very poor test.
First and foremost, testing intelligence by looking for any particular concrete expression of intelligence as such is almost certainly misguided. For reasons that will be made clearer below, there is not, and cannot be, any concrete form of activity that qualifies as unequivocally intelligent, and quite possibly no particular instance of an otherwise truly intelligent activity that could not be mimicked by an entirely unintelligent system. That is why it would do no good to try to build an artificial intelligence on the basis of any concrete picture of intelligence – a complete intellectual profile of Albert Einstein, perhaps, or the possession of language or a proof that, whatever else it did, any group of real intelligences would naturally invent feudalism within n centuries of their creation. Of course, we do need to start from a definition of some kind, and fortunately such a definition is not hard to come by. Unfortunately, not only is this definition completely lacking in the kind of concrete content any empirical or functional test presupposes, but it does not come from, and is most unlikely to be accepted by, the present-day AI community.
Secondly, the Turing test is actually devoid of meaningful or decisive content. After all, I may be able to vouch for the fact that my interlocutor is a human being without being able to say just what it was that convinced me. There are no specific aspects of my interlocutor’s communications to me that I am supposed to evaluate against any predefined model of intelligence. On the contrary, much of the attraction of the Turing test lies in its freedom from any actual criteria for passing or failing other than the opinions of the participants. As a result, a dozen people may agree that a given system passes the test without anyone being any the wiser as to what any of us think intelligence is, even subjectively, let alone providing a repeatable test based on an objective definition. Indeed, Turing’s criteria are so weak and so open that they are all but bound to generate false positives and false negatives in extraordinary profusion. For example, chimpanzees have a serious claim to genuine (if limited) intelligence (Parker and McKinney 1999), but as Hopper and others have observed, there is no chance that one would pass the Turing test. Again there are solutions to this problem, such as basing any such test on a more rigorous analysis of intelligence as it is found in nature, and again such analysis is not hard to come by. But again that analysis does not come from AI, cognitive science, or any of their allies.
Thirdly, the test assumes that a mimic is the same as the real thing. This is the real crux of the matter and the real reason why the Turing test is so misleading. It is a straight functional-empirical test: can this system fool us into believing that it is intelligent by acting and looking like it is intelligent? This is rather like arguing that anything that could create convincing animal tracks must be an animal, yet my children have owned shoes that would fit that particular bill. And in any case, given the constraints that surround the Turing test, what would it really tell us if an computer system could meet all its requirements? If the only way I could communicate with it was through a computer terminal, then I’m sure that a computer model could easily fool me into believing that it was the planet Saturn in orbit about the Sun, connected to me via a telescope and a display screen. Does the fact that I cannot tell otherwise oblige me to believe that what I am seeing on that screen really the planet Saturn? How would I know whether it was planet of any kind, orbiting anything whatsoever? Maybe it is, maybe it isn’t – but this certainly isn’t a test of it, one way or the other. On this basis, I should believe that there is such a place as Middle Earth and that it is under threat from orcs.
How much less then is it a test of a computer systems’ intelligence to check how functionally or empirically like real intelligence it is. Nor therefore is it a sensible strategy to try to build an artificial intelligence by assembling a functional or empirical simulacrum. After all, if a magician looks like they are pulling eggs from my ears, am I supposed to conclude that chickens are roosting in my brain? (If I believed that, perhaps they are.) Least of all is it a credible approach to creating an artificial intelligence to introduce elements of randomness or even deliberate mistakes (both stratagems suggested by Turing himself and echoed by other researchers ever since) on the grounds that that would make the machine ‘seem’ more pathetically human. The fact that such an approach would make it harder to tell the difference between a human being and a machine is completely irrelevant so long as we are asking is whether either possesses a real intelligence, as opposed to asking whether we could be fooled by a fake.
Finally, the Turing test is not a test of anything that, thanks to researchers such as Piaget, his colleagues, his successors and even his critics, we already know to characterise natural intelligence, as elucidated at great length, in great empirical detail, and with detailed analysis and validation across a huge range of psychic and social domains. So it is quite wrong to assert that:
The [Turing test] may be abandoned one day, because more information on how the mind works may be obtained and we may have better means to detect another entity’s cognitive capacities. But today, we do not have much to look at that is more informative than behavior.
1.2. Turing’s conception of intelligence
So what kind of intelligent being did Turing have in mind when he formulated his famous test? It turns out that, when he said that ‘A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine’, he meant that ‘the Turing machine is a model, idealised in certain respects, of a human being calculating in accordance with an effective procedure’. In other words, Turing’s starting point was not so much that a machine could model a human being but rather that a human being could model a machine. Certainly the Turing machine was not intended by Turing to be a model of human beings or intelligence in general, and certainly not a model of Romeo and Juliet falling in love, Beethoven writing the Eroica, Kant pondering the starry heavens above and the moral law within, or you and me fighting on opposite sides of the barricades. It wasn’t even a general model of work. On the contrary, it was a model for dealing with, in Turing’s own words, ‘problems which can be solved by human clerical labour, working to fixed rules, and without understanding’. Turing even described electronic computers as ‘intended to carry out any definite rule of thumb process which could have been done by a human operator working in a disciplined but unintelligent manner’. And again, ‘The human computer is supposed to be following rules; he has no authority to deviate from them in any detail‘. So far is this from any intelligible model of an intelligent human being that it is almost a specification for Searle’s ‘Chinese room’ – the very antithesis of intelligent action.
That this rather witless individual is what Turing had in mind is supported by the form of determinism to which Turing – and by implication every computational model – was referring when defining a machine that could solve the Entscheidungsproblem.
It was not the determinism of physics, or chemistry, or of biological cells, that was involved in Hilbert’s question about decidability. It was something more abstract. It was the quality of being fixed in advance, in such a way that nothing new could arise… One could describe in advance exactly how the machine would behave in any contingency… He would consider machines which at any time would be in any one of a finite number of possible ‘configurations’. Then if, as with a typewriter keyboard, there were only a finite number of things that could be done to the machine, a complete account of the behaviour of the machine could be given, once for all, in finite form… And so he imagine machines that were, in effect, super-typewriters… The whole point was that it should [work] without the interference of human judgment, imagination or intelligence.
Perhaps there is a degree of artistic license here, perhaps not. But from Turing’s point of view, the utterly mechanical machine that could decide a mathematical problem, with its discrete and finite states, its ‘notes of instruction’ and tables of behaviour resembling a kind of implacable Laplacean bureaucracy, is in this sense essentially the same machine that would lie at the heart of a true artificial intelligence. Not that Turing (or many of his successors) felt no qualms about this view, but these seem to have been marginal qualifications that needed to be dealt with rather than fundamental difficulties that might derail the whole programme. No wonder that, when reading of Turing’s ‘intelligent machinery’ bravely facing up to his test, one senses both the extreme logical rigour and the utter, utter stupidity of such a device. It could be programmed to anything that it could be programmed to do, but would have no idea of what it was doing. In other words, the Turing machine is the very antithesis of a human being, which cannot be programmed to do anything, and can never detach its actions from either their goals or its own responsibilities for their consequences.
So the shortcomings of the Turing test are not accidental, and its flaws explain why Turing’s assertion that we should apply the same criteria to machine intelligence as to our human neighbours’ intelligence is so faulty: it is a perfectly valid requirement in principle, but if we expected no more of our neighbours that they should be ale to pass the Turing test, then we would be living in an extraordinarily shallow world. Yet the limitations of Turing’s test follows directly from the inadequacy of his assumptions about what would constitute either intelligence or a valid test. To put it as charitably as possible, the contrast between Turing-like tests and tests of intelligent proper can perhaps be summarised by contrasting intelligence as such with the kinds of concrete ability, skill, ability as so on, which can be collectively termed ‘cleverness’. Many systems, biological and mechanical, are demonstrably clever. Plainly that does not make them intelligent.
Nor is this a matter of abstract methodology: the flaws go right down to the most practical detail. For example, Turing believed that ‘the question and answer method seems to be suitable for introducing almost any one of the fields of human endeavour that we wish to include’. True, as Turing argued, it would make no sense to challenge a computer to a beauty contest any more than one it would be a valid test of my athleticism to make me race against an airplane. Yet there are many aspects of intelligence that are excluded by this approach, especially once one takes into account the types of question one is not allowed to ask. Yes/no questions are in order, and since 1995 Loebner Prize contestants have had to respond to questions on to any topic whatsoever. However, transcripts of the leading performers suggest that a question like ‘What do you think of Picasso?’ (which Turing expressly excluded in his original paper) has yet to be met with a direct answer that indicated any aesthetic appreciation: the leading stratagems still seem to be to prevaricate and deceive rather than to exhibit genuine intelligence. I suspect that ‘Isn’t it a lovely day?’ and ‘Are you afraid of dying?’ are out too. This rather suggests that the only kind of conversation allowed is an expressly unintelligent one. Clearly you would not want to trapped at a party with a Turing machine – not even for five minutes.
1.3. The Church-Turing thesis
So the Turing test is a bad test, and it is not hard to see why other AI tests are equally flawed. But perhaps such tests should not be used as a stick with which to belabour AI as a whole: if they are bad tests, why don’t we just forget them?
Unfortunately, it is a lot harder to discard the real cause for complaint and the real reason why most tests of ‘machine intelligence’ have been no more than elaborations and variations on Turing’s theme. The Turing test is a trivial functional-empirical test, not only in its ‘deceive human beings for five minutes at least seventy percent of the time’ form but in principle. And the reason why AI as it is currently constituted is never going to get to an adequate test of intelligence proper is that the very basis of its approach makes the same mistake as the Turing test, namely its misplaced faith in computation. And computation in turn is the wrong starting point for any attempt to build a true artificial intelligence because it must by its very nature constantly return to functional and empirical descriptions, definitions and tests, precisely because it is computational. In other words, the reason why we can’t just forget the Turing test is because it is (in many ways) the practical embodiment of the theoretical kernel of AI: the Church-Turing thesis.
The Church-Turing thesis – informally, that ‘every effective computation can be carried out by a Turing machine’ – is the technical heart of computationalism. It has been interpreted by some philosophers of mind as meaning that all forms of intelligent activity, whether defined as ‘computation’ as such, as a specific form of neural activity, or otherwise, can be exhaustively analysed in terms of the computation of data or the operations of a hypothetical Turing machine. So is the Church-Turing thesis really open to this kind of interpretation? If it is, then the way to AI seems to be open. But if not, then computationalism is destined to be a discarded hypothesis rather than a secure foundation for artificial intelligence, and AI research must find a stronger basis.
The Church-Turing thesis is by no means uncontentious in AI research. Various authors have argued that it cannot legitimately be applied in this way, at least not without a great deal of auxiliary speculation, which would rather undermine the point of appealing to the thesis in the first place. For example, even if intelligent activity were machine-like (either loosely or in the technical sense in which computationalists use this word), the Church-Turing thesis does not actually imply that a Turing machine could simulate the behaviour of any machine, or that any physical, organic, intelligent or even computational system could be simulated by a Turing machine. As Copeland puts it:
The Church-Turing thesis does not entail that the brain (or the mind, or consciousness) can be modelled by a Turing machine program, not even in conjunction with the belief that the brain (or mind, etc.) is scientifically explicable, or exhibits a systematic pattern of responses to the environment, or is ‘rule-governed’ (etc.).
However, my own reservations about the Church-Turing thesis do not concern its interpretation. Rather, it seems to me that no computational model can exceed the bounds of the functional and the empirical, and so penetrate into the structural and existential depths of intelligence proper. The reason for this is that computationalism is an application of mathematics, and all mathematical definitions, descriptions and controls – archetypally in the form of mathematical formulae – assume that all the terms of the definition, description, control, and so on, remain fixed for the duration. Given the role played in intelligence by the many forms of activity that actively change the terms of the activity under way even while it is taking place, plainly no mathematical account could be given of an intelligent action.
Or rather, it may well be possible to give a mathematical model for any established pattern of activity, including any pattern established by intelligence itself; however, it is not the repetition of established patterns that constitutes the intelligence of an activity. Rather, it is the original insight that something new was required – quite often that an existing rule or model or value or goal or item of knowledge was false or invalid – and that this or that novel reconstruction or amendment of the current ‘terms’ of activity would solve the problem. Even a meta-model that succeeded in formulating how such a change takes place would not suffice, given that the mere knowledge that such a formulation was applicable would cause one to take yet a further step back, to a yet more abstract plane on which the meta-model no longer applied. That is, after all, exactly what the mathematician was doing when they formulated the meta-model in the first place. And it is there, and not in the ability to use the meta-model, that the intelligent of mathematics lies.
Hence the inadequacy of mathematical explanations of intelligence – and so, a fortiori, of computationalism. To put it at its simplest, not only are intelligent functions computable but that very computability would be conclusive evidence for its lack of intelligence,
[review notes on mathematics, logic & development.]
If computationalism is limited in the same manner as all other mathematics, then the only approach it can adopt to anything is functional or empirical. Or rather, it finds the hidden alter egos of the functional and the empirical completely inaccessible, and must resort to the functional and the empirical for want of a more penetrating alternative.
2. The functions of intelligence
If the Turing test will not do, what is it about natural intelligence that is so impossible for computation in general and current approaches to artificial intelligence in particular to grasp? Innumerable answers have been offered to this question, and even defining intelligence is beyond the scope of a single essay. Conversely, if it is wrong (not to mention impossible) to define intelligence and intelligent knowledge in purely functional or empirical terms, what alternative is there? The answer comes from a problem that is seldom recognised (indeed, is bound to be ignored) by AI’s constant attempt to square intelligence with computationalism. There is no intelligence in the passive reception of ‘data’, in the ‘processing’ of that data into an internal result or output of some kind, static storage of internal data, the calculation of new ‘weights’ in a neural net, and so on. For where mathematics is obliged to insist that the terms in which it operates are held constant, if only for the duration of the current operation, one can only be certain that one is dealing with intelligence proper when exactly the reverse occurs, and the terms of the current function, empirical manifestation, and so on, are challenged and perhaps overthrown not only as a result but actually in the course and as a defining feature of its current activity.
The specifically intelligent quality of natural intelligence emerges not through any special predisposition towards, capacity for or the accumulation of empirical knowledge or functional skills, but through the organisation and re-organisation of the structures and methods whereby knowledge and activity of any kind is acquired and applied. What is more, these regular self-transformations are intelligent not because of their regularity of even their mere occurrence, but because they are the direct and immediate product of intelligent activity itself. They are the very heart of intelligence, all else being merely its transient and superficial embodiments, each of which is cast off as soon as it is found wanting. Conversely, a static intelligence, an intelligence that is not constantly aware of, rehearsing and (where necessary) revising its own goals, reasons and values, is not intelligent at all. That is why naturally intelligent activity and experience can both be highly organised and yet possess no innate biases of any proto-rational kind, and why to be intelligent is defined not by any particular functional or empirical test but, paradoxically enough, by the absence of any such tests.
Not that intelligence lacks highly characteristic functions. All intelligent beings imitate, play reason and judge. They have language and imagination and ideologies and science. However, all these functions differ from the functions of all preceding forms of matter in that they lack any determinate forms or expressions or embodiments. To play a game may well be to follow a fixed set of rules with predetermined objective, and so on. But in direct proportion to how closely it is constrained by such structures, play lacks its intelligent core, which is its playfulness. To be playful is to play with all aspects of the activity in question, including rules, goals and all the rest. That is perhaps why the choice of chess – so eminently suited to computation – was in fact fatal to so much research in artificial intelligence: of all games, chess is the least playful.
However, my main examples of this lack of predeterminate forms, expressions and embodiments not only exemplify this characteristic lack of concreteness just as well as imitation or play but are also clearly exemplified by the very activity in which researchers into intelligence of all kinds are necessarily engaging, namely science. Hence the present very simple, very narrow approach, in which I propose that all intelligent activity – and no non-intelligent activity – is characterised by (among other things) two phenomena, namely insight and criticism. Hence the significance of the remark by Picasso at the head of this essay: for the ability to answer questions – a typical element of any standard approach to artificial intelligence – is a very secondary talent in comparison with the ability to grasp what is the right question in the first place, to ponder what is taken for granted, and the ability to doubt, to question, to ask ‘What if?’, and so on. And of course, such abilities rest directly on the human talents for insight and criticism.
That insight is a deterministic process may be debatable, although I suspect that, with sufficient knowledge any given insight could be explained, although only post hoc. However, that it is deterministic does not entail that it is either mechanical or reducible to anything other than itself – two common assumptions about the phenomena of intelligence that pervade not only AI but the sciences as a whole. Determination requires only that events come about solely as a result of prior conditions, which is by no means the same as saying that these conditions are fixed and immutable or that there is any ‘underlying’ or ‘external’ structure by which these events are ‘really’ caused. On the contrary, human beings are constantly doing completely novel things such as speaking sentences that have never been spoken before (by them anyway), solving unprecedented problems, and so on, being struck by novel connections, and so on, without any sudden change in either their abilities or the experience of these novelties being radically different from their experience of everyday life. Indeed, every object a human being constructs is a novel, unprecedented construction, existentially speaking; why then should we try to expunge the idea of novelty in intelligent action by reducing the deterministic to the mechanical. Likewise, by entering into a situation as an insightful agent rather than either tracking it blindly or imposing a processing method on it, an intelligent being is open to all the possibilities of the situation, including all those elements of my object’s existence that are not expressed in the current situation or do not relate to my current state, perspective, interests, and so on. Indeed, the ability to comprehend that things and events exist existentially, so to speak – in their own right, on their own terms and for their own sakes – is absolutely central to any theory of natural intelligence. Conversely, why should we expect a truly mechanical system to be capable of originality, novelty or insight?
In terms of Turing’s original problem – Hilbert’s Entscheidungsproblem – the relationship between insight and decidability is relatively straightforward. If a system of ideas is not powerful enough to show whether or not a given proposition was decidable, then a genuine insight would be needed to extend and deepen that system so that it could make that decision. But such an insight is by definition not part of the original system. If, on the other hand, it is possible to see whether or not a given proposition was decidable, then no insight would be required. In the former case, insight could not be modelled by the system in question, and in the latter, nothing needs to be modelled. Likewise for any computational system or ‘definite method’ (in Turing’s sense): nothing it can model constitutes an insight, and insight itself is precisely what cannot be modelled.
What is more, any activity that is not characterised by insight, or at least the possibility that it may be interrupted, enriched or even wholly derailed by insight is, solely by virtue of that lack, not intelligent. After all, although an action may replicate an intelligent action in every empirical or functional respect (in other words, it may mimic it perfectly), if there is not even the possibility that the very terms of the action in question may be challenged – as embodying false assumptions, for example, as being wrongly organised and structured, as not really tending to the action in question’s overall goal or being faithful to the agent’s purpose or values, or just plain as not making sense – then although it may have been instigated, designed and even carried out by an intelligent being, it is not intelligent in itself.
If this suggestion is correct, then for a given research stratagem to be on the right track it must be able to show that any artificial intelligence it produced would be capable of insight. Furthermore, artificially intelligent activity must be characterised by insight, not as one talent among many but as potentially an integral part of all its activity. For while it is true that all human beings are capable of being thoroughly uninsightful and uncritical, it is precisely this kind of activity that raises questions about their actual intelligence. There also appear to be no forms of strictly intelligent activity that do not posit their own goals, values, objectives, and so on, or that prevent the agent from tracking the relationship between such factors and the action itself (ie, insight).
I must emphasise that insight is not special. I might just as easily have selected explanation, proof, justification or self-justification, prudence, not to mention any number of other exemplars, all of which must be at least implied by all forms of specifically intelligent knowledge and activity, and all of which are both presuppose and inform insight. For example, regarding sympathy, if you experience another as being the same as (identical with, equivalent to, etc.) yourself, your attitude to it is the same (identical, equivalent, etc.) as your attitude to yourself. Of course, it never is quite identical – in particular, you usually know it is not you – so sympathy is never quite complete. However, sympathy in general is a natural corollary of insight. Although few of these may be present in any given intelligent action, they are all capable of being imported into action if it is genuinely intelligent. Or rather, to act intelligently is, among other things, to take all these into account, however implicit this accounting may normally be. For what all these qualities share is the unique system of causality to which natural intelligence is subject, namely rationality. In a rational system, neither action nor belief is ‘caused’ in the sense that they occur by virtue of structures and forces that operate without regard to the meanings, values and implications and without the possibility of evaluation, justification or conclusion. In other words, a rational system is a system of causation that is driven by reasons (in the widest sense), and not by causes in any of the many senses that term has in the natural sciences. In a rational context, insight plays a role that is not only fundamental but also obvious.
Insight has been recognised as an essential attribute of human cognition ever since Plato and his descendants. Aristotle even defined true knowledge in terms of insight, regarding empirical experience as a very poor relation to knowledge until it was reconstructed on rational terms. The scientific association of intelligence with insight goes back at least to John Romanes, Darwin’s partisan and acolyte. Indeed, Romanes defined intelligence in terms of its ability to infer the imperceptible from the perceptible. In other words, a natural intelligence is insightful because is makes sense of things in terms of their structures and their relationship to the world, as opposed either what they do (their functioning) or what they look like (their empirical surfaces). As a result, intelligence takes outward data into account, but it is not content with data until it has explained it and come to terms with it, until it knows what it means or of what value it is, why and in what sense it actually is what it is, and until it fully apprehends not only what it is but also that it is, with all that that implies for its relationship to other ‘data’, independently of its relationship to the intelligence in question. In other words, natural intelligence is never content with the facts of life until it knows how to grasp them existentially and comprehend them structurally. In even fewer words, intelligence is rational.
In terms of Turing’s original machine, with its ability to write and erase discrete symbols, plodding back and forth along its a single, effectively endless tape, is there any way that such a contraption could ever recognise that one of the symbols was in some sense ambiguous or inappropriate? Of course, were it pre-programmed to recognise certain forms of conflict or omission there would be no problem, but then in what sense would it be identifying a problem, as opposed to just processing another class of predefined data? Conversely, what if, like a newborn baby, it were not so programmed? What if, as Piaget demonstrated, the development of higher levels of insight and effectiveness relies on nothing more than the occurrence of contradictions within the activity in question, as problems that arose, unforeseen, only as the programme was being executed? Or indeed, that external hints and instructions were not only unnecessary but, in the absence of the internal capacity to profit from the agent’s own original insights, completely useless?
Of course, this is a very difficult notion – that something radically new could spring (or at least struggle) into existence not through a specific, pre-existing capability being deployed but through the system in question struggling with an unforeseen difficulty, which moreover arises out of a flaw in its own nature, and yet it is overcome anyway. No wonder that theorists are practically queuing up to attribute such talents to heredity, as though that in some way solved the problem. I can walk – it must be evolution; I can think – it must be evolution; my wishes always come true – it must be … But where exactly does the chain break down? Is it any more legitimate to credit evolution with the creation of reason or reflexivity than it would be to credit it with infallibility or knowing the recipe for fairy dust?
The ability to ‘see’ beyond the surfaces of things in this way confers some interesting abilities. For example, it allows the intelligence in question to identify, resolve, tolerate and also exploit ambiguity. What are, from an empirical or functional point of view, two separate sets of data are, from an insightful point of view, two aspects of the same thing. This is evident in the most ordinary features of human intercourse, as in the case of irony. This is not a special skill added on to ‘standard’ intelligence, but its perfectly routine expression. Any test of intelligence should certainly oblige both interlocutors to deal with irony – a test that the Turing test conspicuously ignores.
This is only one aspect of the way in which insight enables intelligence to see both that a single reality can seem very different when addressed from a different perspectives, and that appearances can remain the same even though the underlying reality has changed. Another is the ability to perform experiments, which would be impossible without the ability to envisage a structure that transcends the merely empirical, such as a law. Insight allows us to grasp the profound disconnection between the data a source provides and the nature of source considered in its own right, on its own terms, and for its own sake. Insight not only lets us have explicit notions of sameness and difference but also tells us that sameness and difference may be indicated but they are not defined by the data we receive. They are moreover only ‘indicated’ to a being prepared for such indications – an ability which could never arise from the mere processing of data. So insight is an meaningless notion at any pre-intelligent level: only a handful of primates, cetaceans and a few other species are able to see beyond the data given. What is more, insight enables us to see not only the continuities and discontinuities in existence but also, at higher levels of development, the processes and mechanisms (forms, rules, procedures, and so on) through which identity is managed independently of surface changes. It even allows us to recognise that things continue to exist even though we no longer experience them directly.
In short, any intelligent being is capable of cognitive operations that could not possibly be analysed in terms of the Church-Turing thesis precisely because this thesis expressly assumes a finite number of exact instructions. Leaving aside the question of whether any intelligent act requires exact instructions (what would be intelligent about that?), it is perfectly clear that the simplest forms of normal rationality – including those in which the most avid fan of strong AI engage all day long – exceed these limits. The simplest abstraction has the effect not just of adding new ‘instructions’ to the current activity but also of potentially changing the meaning of all previous instructions. And what else are insight and criticism other than, respectively, the creation and the application of ways of grasping the situation at hand that are not present in the existing ‘instructions’? A higher perspective of this kind is not just an addition but a qualitative change. This in turn transforms the ‘effective’ or ‘mechanical’ significance (as AI terminology has it) of the current ‘instructions’, the stability of whose semantics is taken for granted by the Church-Turing thesis. But the reality of dynamic qualitative changes in perspective of this kind, and even of level of cognitive structure, is well attested, not only in the scientific literature but in everyone’s everyday experience.
And even that simple observation ignores innumerable questions. For example, if it is true that all rational acts involve a degree of insight, criticism, and so on, then it is not at all obvious that a cognitive act can be exhausted by ‘instructions’ of any kind, or could be described even superficially as an ‘input-output function’. Even the most determined attempt on the part of an intelligent being to follow instructions mechanically will inevitably lead it to ask itself whether it is following them ‘properly’ – a question no computational device would ever ask itself. Nor is it obvious that the simplest addition to (let alone insight into) an intelligent cognitive ‘instruction set’ could be accounted for by simply re-computing previous states of the ‘system’ in the manner of a Turing machine. On the contrary, it seems highly unlikely that any change could ever be made by an intelligent being that did not involve it pondering what it was doing on some thoroughly non-functional, non-empirical plane. Still less has it ever been shown that a yet higher process such as abstraction can be computed in this sense.
The centrality of such abilities to intelligence is obvious. Without insight, the abstractions on which every intelligent action is based – its assumptions and axioms, the values and goals to which it is directed, the processes and methods to be applied, and so on – would be entirely inaccessible. It would be impossible to tell whether one was doing the right thing or how well one as doing in any but the most trivial sense. Nor is insight a narrowly intellectual capacity. Without insight, it would be impossible to do anything intelligent, such as construct machines to fly to the Moon, suspect that we are being deceived, conduct an experiment or believe (or disbelieve) in God.
Nor is it only computational models that are undermined by these peculiar capabilities. A normal biological activity such as navigation is transformed out of all recognition by the ability to make and use maps – a relatively simple expression of insight. After all, an arctic tern may be able to navigate almost from pole to pole, but I doubt that it could replicate my proven ability to navigate the route to my various places of work each morning. What is more, its inability to do so is the direct expression of its lack of insight, not only into what I do every day but also into what it does every year. That is why, unlike any non-intelligent being, I could navigate not only the distance to my office but also to literally any point of the entire planet. As for the rest of the visible universe…
This is not simply a matter of human beings possessing a special talent for navigation. As my wife will confirm, I am a useless navigator with a dreadful sense of direction, but I have managed to get from south England to Scotland, Johannesburg, central Djakarta and Wilmington, Delaware without too much effort. Admittedly I had the help of many other intelligent beings, but none of them had any innate talent for navigation either. The arctic tern, by contrast, has a very definite in-built ability to perform this particular navigational task, but seems to be completely helpless when faced with other, superficially more simple journeys. And it is intelligence that explains why the tern’s ability is so limited and mine unlimited: to extend it the tern must wait for evolution, whereas any intelligent being can navigate from anywhere to anywhere, pretty much right now.
For the human ability to navigate is only the application to the problem of movement through the terrain of ordinary intelligent abilities of a much more general order: the ability to understand correspondences and anticipate changes in perspective, a completely generalised grasp of the structure as well as the content of space (as opposed to the tern’s apparently specialised and limited grasp of features in space – which are neither structure nor content), and so on. As for the more special (though no more specialised) abilities intelligent beings demonstrate, such as writing and reading a story, conducting a scientific experiment, running an industrial economy, explaining the nature of intelligence, and so on, it is quite inconceivable that these could be carried out without an enormous capacity for (among many, many other things) insight. And none of them seem to rely on pre-programmed abilities comparable to the arctic tern’s talent for navigation.
As I have already, said, the view of intelligence I am applying here is more or less that advocated by Jean Piaget. Assuming that Piaget’s account is correct – and the evidence generally supports him in every arena from child development and cross-cultural studies to anthropology and primatology – the kinds of abilities described above are normal forms and consequences of insight of which any natural intelligence would be capable. Any AI research looking for a plausible definition of intelligence must surely begin somewhere near here.
Yet this only emphasises a fundamental paradox for the existing AI community. This description of an intelligent being’s insight into things and events is couched in structural and existential rather than empirical or functional terms. It is the nature and being of things to which rationality in this sense addresses itself, quite independently of what things do or how they are experienced – on the one hand their internal structure, their intrinsic properties and propensities; and on the other their existence, their ontological status, their ‘thusness’. Conversely, it is not the ‘data’ with which things provide us that allow us insight into them, nor any mathematics or other (eg, analogue) computation we might apply to them. After all, the ability to grasp the rational status of an object as such – its objectivity, in fact – demands the ability to grasp precisely those aspects of an object that are neither defined nor represented by ‘data’, but which explain why there are any data at all.
Unfortunately, it is precisely this rational element of human nature that went missing when Turing assumed that the (human) computer’s behaviour as ‘at any moment determined by the symbols he is observing, and his “state of mind” at that moment’, that the actions of the mind can be exhausted by program-like instructions stated in a finite and completely pre-defined language, or that ‘We may compare a man in the process of computing a real number to a machine which is only capable of a fixed number of conditions’. Although the former could be readily extended to allow for any amount of background knowledge and implications, and the latter is a sensible constraint if you want to simulate intelligent actions, even taken together they can not account for the possibility of a new idea (ie, a new ‘state of mind’) arising as part of the very business of performing the computation. But this is a much more characteristically intelligent outcome of an action than any bravura technical performance or command of the data.
It is of course the Turing version that lies at the heart of artificial intelligence – an interpretation of human nature that excludes anything genuinely intelligent. For Turing, action is completely determined by the ‘configuration’ the agent is in, by the way the current ‘table of behaviour’ exhaustively determines what each eventuality signifies and what to do about it, and by the data with which it is presented. Nor (for reasons that will be presented shortly) is the situation redeemed by the inclusion of the most prodigious capacity for learning. And if one’s goal is to create a mechanical simulacrum human nature, then this is a valid strategy: otherwise, how could any form of activity be mechanised, and technology of any kind be created? But from the point of view of a strong but genuinely intelligent artificial intelligence, it completely begs the essential question, namely the extent to which human nature is capable of mechanisation at all.
None of this is particularly original. The idea that a distinction needs to be drawn between valid knowledge based on the structural and existential nature of the situation and mere beliefs and opinions – even factually correct beliefs and true opinions — gained from our empirical and functional acquaintance with the world goes back to Plato’s dialogues and has been reiterated at regular intervals ever since, up to and beyond Kant’s categories. Nor does one have to advocate dividing reason into unreliable belief of the external world and the certainties of the ‘soul’s own business’: as ever, Piaget’s dialectical account of the relationship between knowledge, the knower and the known makes this quite unnecessary. In fact Piaget’s account is exactly what makes it possible both to have a rigorously rationalistic epistemology and to entertain the prospect of an artificial intelligence that could be developed in a laboratory (as opposed to relying on, say, evolution to throw up a suitable hopeful monsters, or perhaps it emerging only from the mind of God).
This independence of the functional and the empirical is not only intrinsic to all forms of intelligence activity but also present from the very start of intelligence. As far as infant cognitive development is concerned – an obvious starting point for any well founded AI programme — the classic example of a non-empirical, non-functional property is the already mentioned quality of ‘object permanence’. The infant is able to grasp an object’s permanence when it recognises that objects continue to exist independently of its empirical or functional contact with them. During the earliest, ‘sensorimotor’ stage of cognitive development, the infant must once have had direct physical acquaintance (through its own practical perception or behaviour) with an object before it can bestow permanence on it. In other words, while intelligence proper is being constructed, the infant is still working with objects’ functional and empirical properties. However, it is not only the empirical and functional properties of its objects that concern it; it is already busily constructing its rational properties, including both the first hints of the structural logic of objectivity in general and the existential recognition of the object’s continuing existence even in its absence.
Nor are these rational conclusions merely generalisations from its empirical and functional acquaintance with objects. There is after all nothing in my handling or seeing a ball that could tell me that the ball still exists when it is in the shed and I am in the kitchen. Rather, object permanence follows from the infant’s active engagement with objects, for which empirical and functional data may offer an index of an object’s presence or absence, but tells the infant nothing at all about either its internal structure or, a fortiori, its existence.
In later stages, however, when more abstract logico-mathematical structures are created, it develops the ability to deduce the nature and properties of objects completely independently of any practical engagement with them, to define abstract objects (such as ‘identity’, ‘justice’, ‘object’ or ‘artificial intelligence’) that have no concrete empirical or functional expressions, and to construct entirely imaginary objects, some of which (devils, phlogiston, free enterprise, and so on) come to wield great practical power, even to the point of engendering vast scientific and political programmes that dwarf AI research.
There are two further rational properties apart from object permanence that are equally difficult to explain in computational terms, but which should be mentioned briefly here. These are an object’s autonomy and its universality, which, taken together with permanence, are the principal aspects of its existence; indeed, taken together, they might even be said to define that existence.
If the permanence of an object is its existence independently of any experience (awareness, consciousness, etc.) of it, its autonomy is its ability to act on its own terms, in its own right and for its own sake. Not that all objects are quite as autonomous as that: not many rocks could be said to act ‘for the their own sakes’, at least not in the sense that an animal does, and certainly not like a human being. In fact, only an intelligent being could be said to capable of full autonomy. An object’s autonomy, like that of an intelligent being, is proportionate to its general material level. For example, an organism is more autonomous than a chemical structure, which is in turn more autonomous than a purely physical entity. But all object’s have at least some autonomy – even a rock resists. However, insofar as a would-be artificial intelligence limits itself to data gathered from its own perspective (ie, consists of applying its internal algorithms to exogenous data), it is necessarily as incapable of detecting an object’s autonomy as its permanence.
As for an object’s universality, this is the continuity and consistency of the object in all circumstances. This implies little more than that it is permanent and autonomous in relation to all other objects. However, universality should not be confused with durability. Like its permanence and its autonomy, the exact terms of an object’s universality can be modified by many forces and factors, not least its functional and empirical relationships to other objects. This book, that building and the planet Saturn are all different from what they were when they first came into existence. But that every object always possesses some degree of permanence, autonomy and universality is an invariant of its existence, and moreover an invariant that, like all genuine invariants, is not expressed in and cannot be derived from any data about it. And again, insofar as a would-be artificial intelligence limits itself to its own perspective, it is incapable of recognising an object’s universality.
From both structural and existential points of view permanence, autonomy and universality completely transform intelligence’s relationship to the universe it inhabits, not only by comparison with any current approach to artificial intelligence but also any non-intelligent structure. It certainly distinguishes intelligence from organic life as a whole. However, this not being an essay on intelligence in general, this remarkable fact need not detain us here. In general terms the significance of our objects’ structural and existential character is that there is a reality that exists independently (though not regardless) of appearances; that objects are ‘others’ that exist independently (though not regardless) of our selves; and that there is a realm of absence that exists independently (though not regardless) of the present. As Daniel Dennett put it:
Many organisms ‘experience’ the sun, and even guide their lives by its passage. A sunflower may track the sun in a minimal way, twisting to face it as it crosses the sky, maximising its daily exposure to sunlight, but it can’t cope with an intervening umbrella… But we human beings don’t just track the sun, we make an ontological discovery about the sun: it’s the sun!
Perhaps the fact that a sunflower’s movements could probably be modelled in information processing or computational terms tells us something about contemporary AI’s notions of intelligence. But as intelligent beings we experience the sun by applying our concept of the sun – our insight into its structural a existential nature as the sun – rather than by simply responding to a particular range and intensity of electromagnetic radiation, gravity, and so on, to what the sun is doing to us, and so on. Dennett’s moment of ‘ontological discovery’ is indeed one of those ‘sudden jumps of clicks from one quite definite state to another’ of which Turing wrote when trying to explain digital computers, but in this case the jump is from inside the situation to outside it, from ordinary seeing to that ‘seeing things invisible’ that consists precisely of transcending the empirical and the functional, and so throws natural intelligence outside the realm of computation and computability.
How then is it possible to have any grasp of an object that neither is based on nor results in the computation or processing of information? In Piaget and Inhelder’s own words:
An operational system derives its content from a series of abstractions of the subject’s actions, and not from particular features or properties of objects.
In other words, permanence, autonomy and universality arise from coordinating, reflecting on and synthesising the various kinds of activity through which we engage objects, other subjects and the world, and so from constructing both higher level capabilities and higher level object properties. That is, it is through the reconstruction of the forms of activity that we produce such insights. The results of this process of ‘reflecting abstraction’ are yet higher level structures capable of yet higher level processing.
Clearly, then, natural intelligence is not ‘insightful’ in the manner of a computer program, which is to say, by mechanically ‘processing’ or ‘computing’ the ‘data’ generated through action and experience into a ‘state’ or ‘result’. But again it is not at all clear that the proponents of AI are even aware of the problem, let alone that they are approaching a solution. On the contrary, as far as I can tell, AI systems seem to be much more like sunflowers than sunbathers. Hence, in sum, the unsuitability of computation to any account, analysis or automation of natural intelligence.
To understand insight’s wider ramifications it is essential to recognise that, as natural intelligence advances, insights consist less and less of static conclusions or beliefs. Rather, as each new stage in the development of intelligence unfolds, a new level of insight is generated that includes not only a deeper grasp of things and events but also a more profound comprehension of the nature of both reality and knowledge. As a result, insight is both more powerful and more contingent on all the other insights at the developing intelligence’s disposal. That is, the very process whereby insights are formulated is increasingly also the process whereby they are integrated with (and so both inform and are informed by) the larger system of knowledge at the intelligence in question’s disposal. Conversely, any insight that is taken for granted ceases to be a genuine insight and is relegated to the status of a mute fact – the least rational and least mature of all categories of knowledge other than data itself.
This capacity for self-integration does not arise from any additional ‘program’ that applies itself to individual insights. For each new insight to be informed by related insights, methods and other factors is inherent in the nature of insights in general. That is the very process whereby insights are created in the first place: through the application of structures of activity that themselves arose out of previous action on objects, and so already possess the power to generate structure, organisation, meaning, significance, value, and so on. That is why the relationship between knowledge and the methods used to construct it matures as knowledge itself advances, from the simplest, most superficial and static forms of knowledge proper to a given stage in the development of intelligence (ie, data), in which the link between knowledge and the method used to construct it goes completely unrecognised, up through facts, information, apprehension, relativity and yet higher forms of insight, and then on to yet higher global stages of cognitive maturity, each of which not only reflects a new level of insight but also is supported by a new overall level of integrity. Likewise for its various forms of action. In Piagetian terms, intelligence progresses from its sensorimotor origins through pre-operational, concrete operational and formal operational stages.
Indeed, the very notion that knowledge could be based on data is extremely problematic. However little it may be recognised, not even intelligence’s least mature productions, let alone its insights, are ever as primitive as mere data. Indeed, the notion of truly ‘raw’ data is credible (or required) only in a universe in which knowledge is simply ‘read off’ from objects, without any active engagement with existing knowledge, and so without any enrichment of new knowledge by the methods whereby it came to be known in the first place. This is the universe of computation, in which data is processed and the results stored, yet not automatically organised, coordinated, drawn into implications, grounded in axioms and assumptions, and otherwise enriched by the very process whereby it is known, except insofar as these ordering methods and storage systems are either explicitly built into isolated computations or explicitly built into a connected system. As a result, nothing new can arise from the mere act of knowing something. The same could never be said of even the least mature intelligence (or the simplest non-intelligent organism), whose method of knowing is precisely to structure, organise, integrate, imply, coordinate and otherwise impose order on even the simplest sensory and motor actions, based on the totality of other structures of activity at its disposal at that point.
In short, ‘data’ may be a sensible category of knowledge for a computational system, but it is quite inapplicable to intelligence. And this is the nub of the matter, for it is the very isolation of knowledge from rational method that eliminates the possibility of not only of further insight but also the formation of all the higher forms of knowledge and activity. Conversely, is it possible for a computational system to have any form of knowledge other than data, and moreover data in this rawest, wholly non-rational, non-structural, non-existential sense? If so, then all may still be well, but if not, then whence will arise all the many aspects of intelligence proper on which any such advance necessarily relies?
Leaving aside the progressive qualitative enrichment of intelligence’s knowledge and activity that comes from its creation of higher and higher levels of knowledge and insight, a key part of what distinguishes each of these new levels from its predecessors is the greater extension and intension of the various forms of validation to which it subjects itself. Again this is created not by pre-defined specialised functions (and certainly not ‘programs’) designed to ensure integrity. Rather, it comes from the spontaneous application of all the neighbouring forms of knowledge and activity to one another. This happens not because intelligence has any natural urge to engage either in self-criticism or in the elucidation of its concepts, but because all forms of intelligence are forms of activity, and so need a positive reason not to apply themselves to anything to which they might be applicable. That is why a small child really will reach for the moon: not because it is predisposed to engage in such a peculiarly fruitless activity but because it has yet to learn how to restrict its efforts to more realistic targets.
It is also by this route that each new form of action and knowledge is subject to the yet higher levels of its integration to which I just referred: the exceptional flexibility of intelligence ensures that a huge range of prior knowledge is lurking directly behind any current act of knowing, ready to inject its implications into the emerging knowledge. This leads to the integration of new knowledge with itself (coherence) and with other knowledge and actions (consistency), with their own functions (correctness) and throughout their spontaneous scope (completeness), and (in the case of functional and empirical actions and knowledge) with the changing circumstances to which they apply (currency). In this way, axioms are compared with one another, assumptions with values, methods with goals, and so on. What is more, any modestly mature intelligence will be able to compare the abstractions on which the activity or experience at hand is based with their own or other, quite external values, methods, assumptions, etc. on every level from formal logical axioms to industrial standards for manufacturing.
But all this is only to say that knowledge is subject to criticism – or, insofar as all knowledge embodies some kind of insight, subject to insight into insight. Conversely, criticism consists (in this context) of the application of what intelligence as gathered from the past to the present, and indeed to the future, to the hypothetical, to the counterfactual, and to any number of other domains of insight.
The scope of criticism demonstrates that, like insight, it is not an in-built function of intelligence so much as a spontaneous effect of the extraordinarily flexible relationship between the various aspects of intelligence as a whole. In that respect it is like gravity, of course: there is no ‘program’ or structure within a rock that causes it to fall (the mathematical construct of ‘gravitons’ notwithstanding); it is simply the result of its property as a piece of matter. But insofar as criticism is simply a corollary of intelligence then, unlike computer programs, there are no limits to the ‘data’ to which the structures out of which natural intelligence is composed can be applied. On the contrary, there seem to be no limits to intelligence’s ability to apply to one another all manner of concepts, relations, systems and so on, in any content and in any context, no matter how remotely connected they may be connected. Hence the universality of criticism, corresponding to the universality of insight. In fact, criticism’s only limit appears to be the objective mutuality to the structures in question. There is no need for that this process to be adaptively significant or functionally connected in some way to the intelligence in question’s current state or concerns.
Before analysing criticism as such, I should emphasise that it is by no means its only corollary. All of intelligence’s expressions rely on criticism just as much as on insight: intuition, understanding, explanation, humour, religion, planning, values, meaning, technology, means-end relationships, art, sympathy, science, love, AI research… On the other hand, the notion of criticism as insight into insight seems to play an especially pivotal role in clarifying the nature of intelligence, as it is central to (among other things) the methods intelligence uses to interpret, evaluate, qualify, judge and develop its own capacity for insight, and so to enrich and mature all the other structures out of which it is composed. Indeed, as I shall show later, the relationship between insight and criticism foreshadows the whole genesis of natural intelligence. Hence the need for AI to encompass criticism as well as insight, and hence the following analysis.
If insight is the natural corollary of intelligence and criticism is the application of insight to itself, clearly the relationship between insight and criticism is both direct and directly expressive of intelligence’s proper character. If insight is awareness of the structures informing a given phenomenon – the internal forces and relationships that determine its empirical, functional and otherwise external appearances – then having an insight consists of intelligence recognising the key structures through which the entity into which it has gained this insight is constructed, acts, responds, develops, and so on. Criticism in turn requires only the continuing articulation of these insights with other insights it has had, as this will force the intelligence in question to elaborate their mutual implications. However, this wholly natural yet wholly unpremeditated and unpredetermined ‘method’ is quite enough to ensure that the various forms of knowledge and action as natural intelligence’s disposal discover their various individual and collective errors and omissions, conflicts and contradictions, recognise new scope for their own application, and also construct new forms of knowledge and action that it can now apply to things and events.
Insofar as criticism can be thought of in these terms, it must be emphasised that applying preceding experience to each new experience is by no means simply a matter of relating past and present data to one another. Even in the case of narrowly empirical and functional experience, the past experience that informs new experience must first transformed into a strictly rational form; only then can it be applied to the latest ‘data’.
So the possibility on which all computational models rest — that one can deal with things and events in terms of the functional or empirical ‘data’ they produce — does not imply that this is the most appropriate way to deal with them, or that even the empirical and functional aspects of objects can be properly understood in purely empirical and functional terms. It was after all technically convenient to deal with the data with which the heavens presented by means of a geocentric model, but no amount of precision was able to turn it into structural model of the universe as it actually exists. So for natural intelligence, the functional and the empirical is not only always complemented by but in fact only make sense when supported by an equal element of rational insight and criticism. This is the case even in the earliest stage in the development of natural intelligence, although it is not explicitly recognised as such; in later stages, however, not only are these rational insights and criticisms recognised but the development of such forms of rationalism actually define the stages themselves.
The significance of rational insight and criticism is perhaps best illustrated by what happens when the insights and criticisms in question turns out to be wrong, as in the case of pre-Copernican cosmologies. As Ptolemy demonstrated, it is perfectly possible to regard the sun, moon and planets as a system of epicycles and eccentrics, resulting in a cosmology that is not only highly productive but also (from an empirical standpoint on planet Earth) really very accurate. But as the histories of cosmology and every other science illustrate rather fully, the accuracy of functional and empirical results provide relatively little support for rational conclusions.
As far as computational AI is concerned, the rational element seems either to be injected by the researcher or allowed to remain implicit in the equations. In neither case are the results likely to be very constructive, given that most AI researchers seem to subscribe to a narrowly computational model, without any explicit element of rationalism either in their systems or in their own analysis. As for wholly implicit forms of rationalism (assuming that that is not, as far as scientific rationality is concerned, a contradiction in terms), these certainly cannot be expected to make any constructive contribution until they are brought to the surface and subjected to rigorous examination.
The dynamism introduced into experience by criticism is evident from how natural intelligence handles a simple repetition of experience, in which I am required to deal with the same situation for a second time. This shows not only that I have learned some of the facts of the situation (which may be conceived of in computational terms) but also that the situation is transformed in its very nature. At the very least I now encounter the repeated situation as repeated, which allows me to know that I can deal with it at least partly by anticipation and experience, rather than either wandering into it blindly again, or even comparing it with previous empirical and functional experience. In other words, the minimal effect of bringing my previous experience is that my method for dealing with it becomes more methodical. Furthermore, my existential attitude to the situation is changed by my recognition of its repetition, be it to make me more confident, more self-conscious, more anxious or simply a little bit bored. These do not appear to be attributes of any computational system, any more than Gödel’s well known example of intuitively seeing the truth of an unprovable proposition – a problem that Turing addressed and to which he added some order, but never demonstrated could be formalised or mechanised. And yet, as he put it, ‘It is intended that when these are really well arranged the validity of the intuitive steps which are required cannot seriously be doubted’. At least as importantly, they form an integral element of cognition and everyday reasoning, almost completely regardless of whether or not they could ever be given an acceptable computational analysis or interpretation.
Nor is criticism limited to applying insights from direct experience. Any insight (and also any act of criticism) will have its consequences, including not only direct results but also a broader range of effects on any related structures of knowledge and action, including innumerable meanings and significances, implications for and contradictions to other insights and criticisms, confirmations and challenges to values, and so on, to the outcome of which the mechanisms of insight and criticism can then be applied. Hence a natural intelligence’s immediate awareness of the plausibility of its results, the validity of its methods, the validity of the values and unspoken assumptions they rest on, and so on. In other words, criticism not only follows naturally and immediately from insight but also potentially brings to bear the full range of knowledge and experience at that intelligence’s disposal.
Our capacity for criticism and self-criticism also undermines a further favourite assumption of much AI research: that the structure of cognition is in some important sense modular. Superficially the very structured character of human knowledge and action intuitive support for the idea of modules. But whether construed as a primitive fact of cognition or, somewhat more plausibly, as the product of considerable development, the detailed model on which the theories of modularity rests seem to be completely at odds with the most ordinary facts of criticism. I think it is fair to ignore Fodor’s exclusion of ‘central’ cognitive functioning from his account – a postulate that saves modularity from many obvious pitfalls – on the grounds that it also seems to be ignored by its enthusiasts. But once that postulate is abandoned, then the very domain-specificity, rigidity, opacity and mutual ignorance that he ascribes to modules makes almost any degree of either insight or criticism unthinkable.
All this is to describe the situation very simplistically. For example, it is notoriously difficult to translate this ‘potential’ availability of criticisms into actual criticisms, not least because, even though the idea of mutually impenetrably and cognitively opaque modules is not appropriate here, human activity is sufficiently heavily demarcated on both social and psychic levels to hamper their mutual influence and support. But, taken either to their own logical conclusions or in conjunction with other concepts and insights with which the intelligence in question is acquainted, these structures will effectively provide their own mutual criticism – which is to say, their subjection to a kind of cognitive commentary by virtue of their relationship to other structures. Hence the natural role of criticism (and explanation and all the rest) in natural intelligence.
2.3. Insight, criticism and computational AI
It is not difficult to extend this kind of argument into other areas. For example, the possibility of being surprised is presumably another corollary of insight. Turing himself claimed that he was often surprised by machines, but that is not really the point; it was after all he who was surprised, not the machine. I doubt that a machine is ever circumspect either, or sceptical, or any of a wide range of other things intelligent beings take for granted as their lot.
It is hard to find anything in contemporary AI research that corresponds to surprise or to either insight or, a fortiori, criticism. Even in the most promising areas of research (such as neural nets) the real purpose of this research seems to be to gradually ‘train’ systems in empirical and functional skills that can precisely and reliably generate a desired output or course of action from a given array of inputs. However, much more importantly, it is clear that no system of computations could possibly be intelligent as defined here. For insofar as intelligence relies on insight and criticism, the very stability of mathematical equations militates against their use to operate intelligently. For the terms of an equation are necessarily stable in the face of al transformations. If a = b + c, it is outside the reach of mathematics to say, ‘Yes, but that’s only because you are assuming that a is [definition], whereas a is really…’ Of course, once the definition of a has been agreed, normal mathematical services can be resumed.
I do not mean to deny the intelligence of mathematics: that would be merely perverse. However, I do mean to deny that the intelligence in mathematics lies in executing the formulae. Rather, it lies in the insights and criticism whereby formulae are arrived at, in the recognition by mathematicians of where they are going, in the mathematician’s awareness and criticism of the assumptions and axioms from which they proceed, as so on. But by the same token, no mathematical system can result in the system that implements it doubting and overthrowing the very terms on which the system is founded without immediately bringing the system in question to a grinding halt.
2.4. The structural and the existential
To conclude this account of the core functions of intelligence, I must amplify two terms I have already applied to intelligence without explanation: ‘structural’ and ‘existential’. These terms must be addressed in order to understand what exactly it is that functional and empirical notions lack.
I can start to summarise this distinction by referring back to Block’s well known definition of psychologism:
Let psychologism be the doctrine that whether behavior is intelligent behavior depends on the character of the internal information processing that produces it. More specifically, I mean psychologism to involve the doctrine that two systems could have actual and potential behavior typical of familiar intelligent beings, that the two systems could be exactly alike in their actual and potential behavior, and in their behavioral dispositions and capacities and counterfactual behavioral properties… – the two systems could be alike in all these ways, yet there could be a difference in the information processing that mediates their stimuli and responses that determines that one is not at all intelligent while the other is fully intelligent.
Leaving aside the assumption that intelligence can be understood as a form of information processing – or indeed whether two such systems would be completely indistinguishable at every level – the significance of Block’s definition for present purposes is the implication that intelligence is not to be explained at the functional or empirical levels at which behavioural explanations operate, and at which, as I have said, computationalism necessarily operates. Rather, a new level of explanation is called for, and this new level it that of the structural and existential.
The structural aspects of knowledge and activity relate to whatever elements of activity and knowledge that need to be understood go beyond empirical data and functional effects. This includes implication, logically derived knowledge, classification, syllogistic reasoning and formally structured action (as a result of, for example, a bureaucratic procedure), and so on. It is structural aspects of knowledge and activity that gives us analogy, symbols, and so on.
The existential aspects of knowledge and activity relate to
Both these terms will be elaborated as the argument proceeds. However, it is important to distinguish the structural and the existential from the empirical and functional immediately.
Functional-empirical awareness is concrete, and as such embodies knowledge at its most immediately compelling. However, while it lacks any grasp of the concomitant structural and existential dimensions of concrete existence, it still lacks the richness that a clear apprehension that concrete knowledge gains from an explicit awareness of the abstract nature and conditions of that existence. As such it is compelling only for as long as one does not ask what that existence means, signifies, implies or otherwise amounts to from a rational point of view. This is a wholly satisfactory state of affairs while one is concerned solely with gathering data, but as soon as one starts to ask what facts those data support, how those facts inform my grasp of a given situation, how that information constitutes ‘proper’ knowledge, whether such knowledge can be translated into wisdom or, ultimately, where the truth lies in all this, it becomes increasingly obvious that empirical and functional awareness as such, is incapable of fulfilling intelligence’s real demands.
Or rather, any established awareness at any of these levels can be translated into computational form: that is why mathematics is so indispensable to any advance nevertheless knowledge. But translating established awareness into regular mathematical form is a secondary effect by comparison with the truly intelligent step, which is to establish that insight in the first place. That step necessarily cannot be defined by any existing mathematical procedure (or any other ‘definite method’), since it is precisely the absence of any suitable procedure or method that is the problem. That in turn is why philosophy is also essential to the advance of knowledge, and why philosophy is so seldom concerned with functional or empirical matters. It is, on the other hand, constantly circulating around questions of method and concept – which is to say, questions of structure, logic and existence. Once a proper conception and method can be established, philosophy can hand the question over to the scientists for mathematisation – but not before.
What then would a viable test of intelligence be like, even within Turing’s limited sphere of conversation? First and foremost, it must test not the ability to resemble intelligence under carefully controlled conditions, but rather what Block described (for strictly verbal intelligence) as ‘the capacity to produce a sensible sequence of verbal responses to a sequence of verbal stimuli, whatever they may be’. However, this definition is not meant here in quite the way Block intends. For the significance of that final ‘whatever they may be’ is not only that it renders the system immune to any purely quantitative advance in information processing capability but also that it allows the conversation with the candidate system to develop in any direction, including the requirement for the system to formulate original insights, criticisms and other ‘meta-conversational’ elements – capabilities whose significance will become clearer shortly.
The reason why such elements are indispensable to an intelligent conversation is easily stated. A conversation that continued on no other basis and with no other topic than the one with which it began would be easily exhausted, or at least reduced to repetition. So would any relationship that was based on a finite or predefined range of shared interests and concerns. That is why the many public exchanges in which we each engage each day, with bus drivers and shop assistants and teachers and others, as so readily exhausted. Conversely, it is only by virtue of our ability to occasionally raise our social intercourse to a new plane (which is to say, to generate these meta-relationship and meta-conversation elements) that an indefinitely extended relationship (or conversation) can be intelligibly maintained.
3. The structure of intelligence
One very basic problem with neural nets exposes the fallacy underlying one of the naturalistic assumptions on which AI has often relied. This is the assumption that the development of intelligence is based on learning. The following is an early statement of the concept of learning that dominated early AI research, based on successive layers of feedback:
Feedback is a method of controlling a system by reinserting into it the results of its past performance. If these results are merely used as numerical data for the criticism of the system and its regulation, we have the simple feedback of the control engineers. If, however, the information which proceeds backwards is able to change the general method and pattern of performance, we have a process which may well be called learning.
Other approaches have been proposed by other precursors and exponents of AI. Note that although this definition creates at least semi-qualitative changes in the competence of the system in question, there is no hint – and no obvious need – for the system itself to recognise or in any understand this development. As a result, just as insight is not related directly to either the empirical or the functional, so there is no direct link between learning as such and intelligence as such. After all, every reptile and amphibian and many an insect is capable of learning, but I doubt that anyone would seriously propose that a salamander or a beetle is intelligent. Conversely, it may well be that if you managed to build a true artificial intelligence it would be good at learning, but it’s very unlikely that, if you built a device whose principle talent was learning, it would be – or could ever become – intelligent. Clever, perhaps – but intelligent, no.
Clever depends on empirical, functional knowledge rather than structural, existential insight.
Of course, from one point of view the interactive quality of neural nets represents a marked change in strategy from either inductive, data-driven or deductive, rule-driven approaches. Inductive methods fall short of insight because they are based on pragmatic adjustment that never attempts to go beneath the surfaces of things, so identifying the structures that are responsible for the data with which they are presented. This cannot be said of neural nets, which are based on intermediate processing taking place behind the immediate relationship between the system and the world. As for deductive approaches, these are indeed concerned with internal structures, but impose them on the problem rather than seeking them out as a natural intelligence might do, namely by acting on the entity it wants to find out about, and abstracting the relevant principles. Again, this cannot be said of neural nets, which gradually construct rule-like methods of actions, decision-making, as so on, yet are always open to further modification in the light of experience.
Nonetheless, even neural nets also fall far short of insight properly so called. Although their flexible, quasi-dynamic and interactive nature is likely to lead to great improvements in machine learning of empirical and functional knowledge, there is nothing in it that could lead to either rational, structural or existential knowledge. Nor is this situation likely to be resolved by, as it were, plugging neural nets into one another, and so perhaps constructing a device getting to know how neural nets get to know things. The real problem lies in a quite different direction.
A similar problem arises when defining or studying intelligence in terms of not of learning but in terms of memory or perception – two more favourite topics in AI circles. Just as learning allows us to learn from empirical and functional contingencies, so perception allows us to sense such ‘data’ and memory restores such knowledge to present experience after it has passed out of reality. It is true that, in already intelligent beings, both incorporate more or less rational insights of various kinds. But neither memory and perception expresses those insights in any explicit way or in any form that could possibly be extracted from them without both further work and a radical change of form – which is to say, without rationalisation into insights.
Despite these clear differences, in the absence of any widely agreed definition of intelligence, it is naturally difficult to say anything about its structure. Cognition is not like learning or perception or memory, but in what sense is it so very different? Is it a difference in structure, and even if it is, is it really such a profound difference that it matters in this context? It’s hard to say. This fact is compounded by a second issue: that it is by no means self-evident that there needs to be any structural resemblance between artificial and natural intelligence. After all, it is very unlikely that an artificial heart or eye would be very much like its natural counterpart; why should an artificial intelligence be any different?
The answer lies in the peculiar properties of intelligence itself. In other cases, the function performed by a particular structure could often be performed by another structure just a well. An artificial heart made of steel and plastic and built like a rather elaborate central hearing pump might well do a perfectly good job. After all, the heart of a fish is no less a ‘natural’ heart than its mammalian counterpart just because it is structurally different. What is more, many a human endeavour (notably flight) has been led astray by the belief that success will favour those who most slavishly copy nature’s structural solutions when trying to solve the functional problems it poses. But as it turns out, it is much harder for human beings to fly with bird-like wings than by other means. So why is it necessary (or even helpful) to know what the structure of a natural intelligence is?
But when comparing artificial and natural hearts, it is clear that the latter’s structure is only an expedient basis for performing its function – powering the circulation of the blood. Conversely, if we chose to define the result at which we are aiming by the function or the result we are trying to produce, we might not bother trying to build a heart at all. Other arrangements, some involving no central organ at all, are quite conceivable. After all, we are really trying to build whatever it is that will make the heart’s contribution to our physiology, not a replica of our anatomy.
But cannot the same be said about human intelligence? After all, it is clear that that intelligence relies on the human brain, but it is by no means certain that an artificial intelligence would have to replicate the brain. Turing was wise enough to reject this approach, eventually coming to prefer to simulate the logic of human activity (of certain limited kinds) rather than replicating the brain. After all, the structure of intelligence to which I refer here is no more obliged to follow the structure of the brain than the brain itself follows the logic of the chemical reactions on which it is based in turn. As Turing put it:
We could produce fairly accurate electrical models to copy the behaviour of nerves, but there seems very little point in doing so. It would be rather like putting a lot of work into cars which walked on legs rather than continuing to use wheels.
What needs to be modelled (if that is the right approach) is the structure of the various forms of intelligent activity, some of which are outlined in this essay. These structures are to be grasped in terms of logic and mathematics, the various forms of knowledge and action at our disposal, and more generally the methods we use to ensure that the content and context of our activity and experience is coherent, consistent, complete and correct.
As far as this kind of structure is concerned, it is true that any adequate implementation would do – human brain, chimpanzee brain, parrot brain perhaps – or, as with the circulation, perhaps not a central nervous system of any kind. Anything that is capable of organising rational, empirical, functional knowledge and action would do. But as for rational, empirical, functional knowledge and action as such, these do require a specific kind of structure. As far as intelligence is concerned, intelligent structure is exactly as essential as function. But as with everything else connected with intelligence, the reasons for this are a little upside down.
The possession of a specific structure will always determine a great deal about the forms of activity of which the structure in question is capable. Conversely, every form of functioning is predicated on the possession of a particular structure. Nevertheless, the possession of a specific structure does not necessarily imply that that structure in question possesses a concrete content or that it is bound to any concrete context. That would be true of any non-intelligent organism (which could even be defined in terms of the compulsions and constraints its structure imposes on the content and context of its functioning) and it seems to be true for existing forms of AI. Yet to possess a restricted or predetermined concrete content or context would be to violate one of the cardinal rules of natural intelligence, namely that it be independent of any particular content or context. That is after all why intelligence is capable of, on the one hand, object permanence, autonomy and universality. For just as all these kinds of activity are by definition independent of any particular content – that is only another way of defining what object permanence, autonomy and universality all mean – so they require an abstract structure through which to be implemented.
Hence the fact that artificial intelligence, unlike artificial flight or an artificial heart, is bound to a particular structure: not in order to impose specific concrete forms of activity or knowledge that somehow constitute intelligence, but precisely in order to avoid being trapped by any particular content or context, which would thereby negate its claim to intelligence. So what makes a structure ‘abstract’? Abstract is not a synonym of ‘vague’, ‘ethereal’ or ‘other-worldly’: logic and mathematics may be archetypal abstract structures, but so are bureaucratic procedures, and so is money (or at least, so they are in a strictly formal organisational and economic system such as industrial capitalism). Rather, an abstract structure is one that is independent of any actual content or context, which is only to say that it persists through all changes in content and context. Hence the relationship not only between abstractness and the special rational quality of our functional and empirical knowledge and activity, which can, as I have already hinted, be summarised as both existential (permanent, autonomous, universal) and structural (explicit, logical, self-regulated).
Of course, precisely because it is lacking in any definite content or context, it is tempting to take some of intelligence’s most powerful capabilities for granted, and to proceed by encompassing its various concrete expressions one by one, as though intelligence were a kind of shopping list of features. This is perhaps why AI research is so prone to building advanced mathematical capabilities into their systems, as though natural intelligence’s very possession of such capabilities were not the very problem; and why so much research begins by selecting some thing that looks rather clever, without any real attempt to justify this choice by reference to a more general model of intelligence as such. The abstract structure of intelligence so infuses our actions and experience that we take it for granted, and prefer to look for something a little more concrete to define intelligence, such as rather narrow linguistic abilities or the ability to fake intelligence by passing the Turing test. Yet our achievements surpass anything AI research addresses by many, many orders of magnitude. How else can we describe our collective reconstruction of the entire planet, or the vast sweeps of history, of the immense social systems and the wealth of cultural and technological accomplishments? Beside all this – all of which follows from the most ordinary actions of natural intelligence – language (as it is treated by AI research) is a very meagre result.
Not that it would be easy to start by building an artificial intelligence that was capable of creating the kind of vast social and historical structures of the kind natural intelligence has produced. Fortunately this is not necessary (although it is necessary that any true artificial intelligence be capable of participating in and contributing to such achievements), as many simpler forms of intelligent structure exist, on which a would-be artificial intelligence could be tested. As I have just suggested, our ability to grasp an object’s permanence is only the complement of the various forms of logic and mathematics to which we are all explicitly or implicitly committed (AI researchers more than most). Conversely, it is only because we possess structures of activity such as logic and mathematics that we are capable of the functional abstractness inherent in permanence, autonomy and universality.
It may seem strange to say that existing AI systems are incapable of logic or mathematics. Such abstract ‘knowledge’ is surely what computers do best – the hard part is making them use these powers to accomplish anything practical or concrete. But in fact the logic and mathematics of which computers are capable are only simulations of logic and mathematics as performed by natural intelligence, and, as a simulation, are no more convincing as realisations of natural logic or mathematics than the simulation of Saturn referred to earlier. Anyone who understands not only how a computer’s processors perform logical and mathematical operations but also how a child who develops the ‘same’ operations knows immediately not only how different these processes are but also how unintelligent a computer’s actions are. Not only does it generate innumerable new insights into and criticisms of the problem with which it started, but it also progressively transforms the very structures of activity from which the natural intelligence in question started out.
On the other hand, unless its command of logic and mathematics is actively constructed by an artificial intelligence system, it can hardly be credited to the system itself, as opposed to being injected by the researcher. What is worse, if such skills are inherent in the system from the start – primitives rather than accomplishments – then its ability to act logically or mathematically is not open to being reflected on by the system or in any way under its own control. Impressive though its abilities often are, in functional and empirical terms, such a system can hardly be said to be acting intelligently just be cause it knows how to handle propositions or the law of the excluded middle by virtue of a built-in sub-system. After all, it would be carrying out these feats for exactly the same reason that an internal combustion engine can drive my car or an ant can lift fifty times its own body weight: because it is built into it mechanically, rather than because it explicitly knows how and applies its abilities rationally. Likewise, knowing that we will soon have a race of computers that can defeat any chess grandmaster is only as impressive as knowing that any car can go faster than any human being – it has an inescapable ring of the ‘So what?’ about it.
Of course, there have been attempts to inculcate true logico-mathematical abilities into machines, as opposed to building them in. None seems to have succeeded, not least because none has followed the methods of natural intelligence. Exactly how natural intelligence arrives at logic or mathematics is a very long and involved story, but the process has been largely teased out. As with any higher-level structure of activity at intelligence’s disposal, they are synthesised from a host of lower level structures, abstracting from them their essential regularities and relationships. By this means natural intelligence constructs higher level structures of activity that are as independent of any or all of them taken in isolation as it relies on them for its embodiment and realisation. Just as a map is assembled and synthesised from innumerable local triangulations, yet once it has been drawn puts every particular locality firmly in its place, so the higher level structures of intelligence arise from nowhere but the coordination and mutual synthesis of particular concrete structures of activity, yet are wholly independent both of any of them and of any particular content or context to which they may ever have been applied. Hence the emergence of the ultimate abstractions (such as logic and mathematics themselves) out of the small child’s the most concrete and pragmatic scrabbling and fumblings. Hence also their ability to break free from and dominate all practical and concrete activity, for they provide such activity with totally permanent, autonomous and universal forms, to which all are subject and none aloof – even as the activity in question is being formulated.
So intelligence demands abstract structures that are absolutely vital to its functions, starting with its capacities for insight and criticism and including absolutely all other forms of knowledge and action. It is only by virtue of such abstractness that the objects of its knowledge and action can likewise continue to exist and remain themselves (at a very high level) more or less independently of any particular change in concrete content or context. Hence the permanence of an object is both possible and intelligible to its subject only as a property of the most abstract kind. Likewise for its autonomy and universality.
How then would an abstract structure work? Being independent of (though never indifferent to) both any particular content or context and any particular concrete or practical action, it would operate entirely endogenously – on its own terms, in its own right and for its own sake. This may seem to be rather an empty definition, not to mention contrary to the uniquely intense engagement intelligence has with its world. But then an abstraction is bound to seem empty and remote, if by that you mean ‘lacking in immediate concreteness’ or ‘lacking in direct practical application’. But it is this very lack of fixed or pre-determined concreteness and practicality that gives intelligence its indisputable power. We see it around us all the time: there are clearly no limits to the forms of organisation, product, tool and system intelligence is capable of producing, and there is equally little doubt about the alacrity with which intelligence abandons them when they become obsolete (an equally necessary condition for natural intelligence).
Exactly how the formation or operation of an abstract structure is possible is well outside of this scope of this paper. Piaget himself describes it in terms of two complementary processes whereby abstract and concrete forms of knowledge and activity are respectively brought into existence. That is, cognitive development proceeds only partly by abstracting from content and context, by virtue of a process Piagetians call reflective abstraction, which culminates in logic and mathematics. There is also the complementary abstraction of content and context, which Piagetians call empirical abstraction, which results in knowledge and control of objects that is isolated from any particular object, such as ‘hot’ and ‘cold’, ‘heavy’ and ‘light’, and so on. The higher stages of intelligence then consist very largely of simultaneously distinguishing and synthesising these differing cognitive streams, eventually developing into ideas like ‘scientific method’, ‘rationality’, ‘computation’ and so on.
Intelligence’s abstractness expresses itself in every facet of its knowledge and action. One of the most intensively studied properties to which it gives rise is ‘reversibility’. We can undo what we have done, we can nullify it with its contrary, we can counterbalance it, we can take countermeasures, and so on. To be able to reverse in this manner – or rather, to be able to recognise a situation as reversible, and so use reversibility as a method for managing things and events – is quite beyond any non-intelligent structure, and apparently beyond any non-human intelligence. To return briefly to the example of the repetition of previous experiences: as far as non-intelligent organisms are concerned, by contrast, it is more a case of ‘one damned thing after another’. Instead of either appreciating that they were the author of the present situation, and so might be able to undo it too, or recognising that knowing how their present predicament was brought about gives them a powerful insight into how to get out of it again (ie, by some kind of reversal), they act as though they had no clue – which, apart from basic empirical and functional learning, they haven’t. But although both of the latter make a powerful contribution to our response to a repeated experience, neither empirical nor functional learning allows us to recognise or take advantage of the fact that this is indeed an extension of past events or subject to known relationships. Given reversibility, however, the situation appears as the creation of a known pattern of actions, and if not always reversible by simply retracing one’s footsteps, it can be certainly treated as the symptom of a larger pattern, of which both the current situation, the one that immediately preceded it and its potentially reversed successor are elements.
Reversibility is not a primitive feature of intelligence. It starts to emerge as natural intelligence approaches the end of the first major stage in its development, which Piaget termed ‘pre-operational’. (In fact it is the reversibility of a form of activity that makes it an ‘operation’ in Piaget’s terms.) On the other hand, although a non-primitive, derived feature of intelligence, it is not limited to the generalisation of the experiences which gave rise to it. Rather, it emerges through the process of abstraction, which is broadly the same as the process of insight already referred to. In addition, far from being a special capability (that one could, perhaps, program into or ‘teach’ an artificial system by computational means), reversibility is the product of intelligence merely being itself. It observes the results of its own activity, recognises them as the results of it own activity, attends to the patterns it observes in this activity, and gradually stabilises and mobilises these observations into definite and independent objects (a ‘relationship’, a ‘procedure’, a ‘sequence’, a ‘class’, and so on). By this means intelligence internalises the structure of these patterns and puts them at its own disposal in the form of the structure of activity needed to reconstruct that kind of object. The term ‘reversibility’ refers to a common aspect of such objectifications (in principle if not in practice): that they show how what is done can be undone, that what is included can also be excluded, that an ascending sequence is also a descending sequence, and so on. This explicit objectification of the structure in question allows that intelligence to see it as a whole, for its own sake, and so to see that one can subtract what has been added, go back the way one has come, compensate for a force or an action here by applying another force or action there, and so on.
The relationship between reversibility, insight and abstraction is, I hope, obvious. To have an insight is to abstract a pattern that allows me to understand the situation in a new sense, be that ‘sense’ a matter of qualities, meaning or dynamics. Admittedly this insight initially arises only in relation to this particular situation, and so risks being fused, and then buried, with its particular content and context. Nevertheless, the more explicit it is made, the more it is articulated and elaborated and its relationship to the situation at hand is worked out, the more readily it can be grasped independently of that situation. In other words, an insight is translated into an abstraction – a distinct object in its own right. Likewise for reversibility: once the insight starts to take on a more abstract form, it becomes an explicit instrument for grasping and comprehending activity and experience, and for performing exactly the feat to which I referred above: turning a disconnected assemblage of disparate things and events into elements of a larger pattern.
Hence another key aspect of the contradictory relationship between reversibility and computationalism. Firstly, the original insight on which the reversal is based is no more susceptible to explanation in computational terms than any other insight. Secondly, once a new form of reversibility is available or applied to a particular situation, it alters the terms on which any preceding train of ‘computations’ was based. After all, any injection of reversibility implies a change to how the situation is understood – a qualitative shift rather than a merely quantitative addition to the information given. This is not simply a question of new empirical data or new functional content: the very structure, and so the logic, of the situation is transformed.
How this squares with the computational assumption (based on the Church-Turing thesis) that a computation consists of a finite set of exact instructions is not at all clear. Indeed, whether it bears any relation at all to that assumption is questionable. Like any insight, recognising the reversibility of a situation both adds to and tells us more about the meaning of the original ‘instructions’. In other words, both their finiteness and their exactness are called into question. What is more, it is precisely this ability to call such the original situation into question – the unstated assumptions and the stated goals, the implicit axioms and the prescribed methods, the desirability of a given result, and so on – that characterises our response to instructions as intelligent.
But surely a reversal of this kind is entirely within the capabilities of computation? In a way it is – any computational device could be given a set of rules for reversals of various kinds. But in another sense – and this sense should be a guiding theme in any AI programme – it is not. A device that is given a solution or has it built in has not come by that solution intelligently and cannot use it intelligently. It merely possesses it, and its ability to deploy it is not an expression of any intelligence it may happen to possess. In that respect it is not very different from the way in which my body ‘understands’ gravity: through a system of bones, connective tissue, muscles, tendons, nervous systems, reflexes and so on, it does a very good job of managing what would otherwise be my body’s rather unhelpful tendency to collapse into a soggy heap. But as to whether my body understands gravity in any more cognitive or rational sense, and so would be capable of appreciating either gravity as such or applying any such insight constructively, the answer is surely that the human body was here a long time before either Newton or Einstein was born, but it still hasn’t learned to handle gravity any better by virtue of our strictly intelligent understanding of the theory of gravity. The same could certainly be said of any artificial system that claimed to be intelligent by virtue of its implementation of a set of algorithms and other structures that were indistinguishable from natural intelligence: its inability of stand back from those structures, and reflect on, articulate them and subject them to insight and criticism, or even to appreciate their criticism by another intelligence, precludes any thought of regarding them as intelligent in their own right.
3.2. Structure and computation
So the essential point about the structure of intelligence is that intelligence is intelligent only by virtue of its structural properties. Unlike an organ or a physical function such a flight, it is crucial that the structure of natural intelligence be replicated as well as its functions. Unlike a stomach or the ability to walk, the relationship between intelligent activity and its structure is direct and internal: you cannot have one without the other. However, the reason for this is precisely the opposite of the reason why one would normally try to replicate the natural structure of a given function: it is not because, in recreating nature’s original, the natural structure imposes the natural function. That approach has proved to be neither helpful nor valid for other forms of artifice, and there is no reason to think it would work for intelligence. Indeed, it would be less effective for intelligence than for any other natural system. For if intelligence is characterised by its abstractness, then it is hard to see how we could build an artificial intelligence without also imposing an artificial content and context on it. But this is not only what natural intelligence achieves, but also what must be achieved if the intelligence in question is to be at all intelligent.
As for creating an artificial counterpart of a naturally intelligent structure, it is difficult to see how one could replace an inherently abstract structure with another, different abstract structure. Are structures not identical in proportion to their abstractness? Could there be another system of integers other than the one we have? True, there are many alternatives to classical formal logic, but it cannot really be said that they are different logics in this sense. Either they are only elaborations of special points, which could in principle be accommodated to a wider general logic, or they claim that there is some concrete feature of reasoning existing logics do not take into account. So it is possible to conclude that a functionally equivalent structure could not exist. However, this is not because the existing natural structure confers any particular concrete features or practical abilities on intelligence that it is so essential; rather, it is because it does not force intelligence into any particular concrete forms or make intelligence adopt any particular practical solutions of approach.
As far as artificial intelligence is concerned, the crucial issue is whether the most typical structural arrangements used by AI practitioners are capable, even in principle, of supporting intelligence. Insofar as they are bound to any particular content or context, the answer is no; and insofar as they have freed themselves from either, the answer is yes. But it is up the proponents of AI to show which is actually the case. Insofar as the structure in question depends on the computation of data, there is no possibility that a true intelligence could ever result.
3.3. The vagaries of ‘information’
4. The development of natural intelligence
One curious aside in Turing’s famous 1950 paper that he does not seem to have been pursued far is his suggestion that it might be more fruitful to build an artificial child than try to go straight to an adult mind. Given the nature of natural intelligence as fundamentally a self-developing structure that emerges from the interaction of much simpler, pre-intelligent organism with its environment, thereby transforming the one into a subject and the other into a world of objects, there is certainly a good deal of merit in this suggestion. Nor could much be derived the model of insight as a fundamental form of intelligent activity were it not for the fact that each new insight is a developmental step, not only in the sense that insight tells us about the present situation its peculiar content and context but also in the sense that the intelligence’s entire developmental trajectory can be though of as a succession of global abstractions of its current forms of activity onto a new plane, more or less as sequence of a quasi-insights. Thus the successive stages in the development of intelligence are not additions or extensions but genuine transformations and enrichments of what went before, not by any external force but by intelligence’s reflection on itself. That such a sequence and such a simple process could fuel development all the way from the newborn baby to the wisest age and the most advanced social system, all based on the same basic structure, tends to undermine the predilection many AI theorists seem to have to reduce the determinism they rightly feel a scientific method obliges them to adopt to a more or less crude mechanism. In terms Alan Turing might recognise, this is the equivalent not of building intelligent machinery but of intelligent machinery building itself.
However, whatever merit there may be in the idea of growing an intelligence from seed, as it were, it is quite certain that it could not be modelled on Turing’s own image of childhood. He offered two quite different options: either a kind of semi-tabula rasa – ‘as simple as possible consistently with the general principles’ – or, at the opposite extreme, something with ‘a complete system of logical inference “built in”’, and it is not obvious that contemporary AI researchers have managed to get beyond this polarisation. As so often, Turing is very prescient about the options the AI community would tend to take for granted, and as so often, quite wrong about what sort of psychology could possibly inform the creation of such a homunculus. For if there is one thing we can be quite certain of as a result of seven or so decades of research into child development, a child is neither a blank slate nor blessed with innate cognitive capacities of any significance. Nor do these capacities emerge as a result of any process the AI community has yet researched.
Hence the final, especially telling aspect of natural intelligence that is generally ignored by AI researchers yet: the way natural intelligence emerges and matures. For it is primarily through its own actions and according to its own criteria and judgements that intelligence advances (or, occasionally, falls back). The development of intelligence is moreover not a merely quantitative process, but a sequence of truly qualitatively different stages.
However, it does not seem to matter from which specific reflexes sensorimotor development starts out; in fact development appears to be more or less normal even where there are huge deficits in the neonate’s abilities. Furthermore, although the neonate may be somewhat predisposed in various empirical and functional directions – for example, they seem to prefer face-like arrangements, to favour ranges of sound in which human voices fall, and so on – there seems to be no bias towards any particular ways of structuring activity. Genuine insight is certainly not innate.
What does seem to matter to human neonates is their ability to develop through their spontaneous activity in the face of experience, and so their capacity for mutual development and overall abstraction from any previous structure. So the essential point is that the development of intelligence is not only a process of self-development – a feature shared by, say, neural networks – but also that what is actually developed is not a more refined form of the structures from which the pre-intelligent infant started but a qualitatively new system of abstractions that are in no way anticipated by its precursors, in no way programmed (at any level) into the neonate, and yet which are wholly universal and invariant (assuming that development is allowed to proceed unhindered).
Permanence is a feature that is attributed to the object, yet can be grasped at all only because of the activity and development of the subject. That is why Piaget refers to this (and other rational properties of the object) being constructed rather than invented (as an idealist would say), imposed (as a vulgar Darwinian would claim) or discovered (as an empiricist would argue). This is the normal mode of development for natural cognition.
The paradoxical solution to this highly paradoxical state of affairs is that the only structures capable of supporting intelligence of any kind are abstract, and that there could be only one such structure, since all structures that were capable of organising intelligent activity and yet were shorn of their content and context would in fact be identical. In that respect intelligence is of course quite different from any of its physical, chemical and biological predecessors, all of which rely of elaborating different forms of very concrete structure to achieve differences in activity. To hunt like of tiger and harvest like an ant by means of biological structures means that the structures in question must be very different; but for human beings, hunting and harvesting are merely different expressions of a single capacity for intelligence activity in general.
I would not want to suggest that anything magical is happening here. It is true that all pre-intelligent forms of material are concrete while intelligence alone is abstract; but at the same time, there is a sequence of progressively greater abstraction in material activity, from the physical to the chemical and the biological. Indeed, the abstractness of intelligence is only the culmination of the progressive abstraction of activity from any particular commitments or situation that characterises the progression of matter from the physical to the chemical to the biological, and within the biological realm, the triumph of (quasi-abstract) adaptability over (decidedly concrete) adaptedness.
In short, the reason why there is only one possible structure of intelligence is not because only human beings are intelligent, so intelligence relies on the structure of a human being, but rather because all intelligences emerge from the same (strictly non-teleological) progression whereby all attachment to and determination by content or context is abandoned. So it does not matter where the intelligence in question starts from – it could have been a primate, a whale, a parrot – and, perhaps, a computer: what makes a given structure intelligent is not its point of departure but its point of arrival.
Generally speaking, there are three ‘moments’ in natural cognitive development: objectification, abstraction and internalisation. Taken together, these three moments constitute the process of reflection, on which all of intelligence’s natural developmental processes rely, and of which criticism is a simple expresses.
For example, if various different forms of activity are connected to one another internally, without the mediation of shared data or parameters passed, then a thing or event that is registered by one of these forms may well affect another, be transmitted to it in some sense, without either the former form of activity being ‘designed’ to communicate in this way or the latter ever encountering the original thing or event directly. The latter’s experience, grasp and comprehension of the ‘stimulus’ in question depends solely on the version and aspects of it that it receives from its neighbour. Conversely, when the form of activity that is directly affected by the original ‘stimulus’ loses track of it, this loss does not affect the second form of activity directly either. It may well continue to ‘respond’ to it and to act as though it is still there long after it has disappeared. What is more, so long as the indirectly affected form of activity continues to act as though the original ‘stimulus’ were still there, its actions may well feed back some version of the original thing or event to the originally affected form of activity.
Naturally, this pattern will not persist for very long when only a few local, weakly established forms of activity are involved and innumerable new experiences are thronging in to replace their predecessor. But that is only to say that the permanence of this purely internal structuring of activity will be proportionate to the range, depth, organisation and experience of the forms of activity involved. Furthermore, in proportion to the forms in question establish similar connections relating to other things and events and the relationships between them, they will tend to be reproduced internally without further external ‘stimulation’. Ultimately there is no reason why the originals should not be comprehended as existing in principle even though they are not in practice grasped by any of the particular forms involved.
Hence the significance of the internal organisation of the forms of activity for insight, over and above the their external functionality, power, range, competence, experience, and so on. For what the persistence of the original thing or event at the level of the organisation of these forms tells the intelligence in question is that, quite independently of either the particular kinds of activity through which it was grasped or the attributes it revealed when they were grasped, it also simply exists.
Starting from the earliest forms of intelligence (in infancy), the concepts, relationships, systems and other structures to which the development of intelligence leads and on which all further development is based all consist of the structures of activity – our own activity – internalised through repeated experience, through mutual differentiation and integration, and through final stabilisation as enduring insights into how to make sense of this or that content or context. Conversely, the ability to apply ‘computational’ methods is the result, not the basis, of successfully coming to terms with thing and events.
4.4. Summary of reflection
The ‘principles’ in terms of which a trained neural net operates are never abstracted from either the net’s ‘experience’ or from the inputs and outputs with which the net is associated. So although the ‘intelligence’ of a neural net is apparently flexible, it does not amount to insight.
This is already evident as intelligence is coming into existence in the first place. Taking the Piagetian analysis of sensorimotor development, the only factors that are needed to transform the sensorimotor reflexes with which the human neonate is born into the intelligence with which even the youngest child is equipped are the sensorimotor reflexes themselves. There is no master program controlling local modules, no central processes managing I/O from [peripherals. There isn’t even a program for development itself. It is true that the sensorimotor reflexes from which intelligence emerges need to have certain attributes, but none of them involve any positive drive or direction towards intelligence as such. All they need is a completely generic capacity to adapt themselves to the activity of their neighbours, including external events in the infant’s environment, somatic events such as changes in state, and the other sensorimotor reflexes that individual infant possesses. Indeed, were our innate sensorimotor reflexes prone to any kind of predisposition to develop in one way rather than another, this would be quite enough to prevent it from achieving the openness, universality and existential character that defines intelligence in the first place.
This can be illustrated by means of a practical example. An infant does not learn to feed by having a ‘program’ that tells it how to coordinate its hunger with sucking, the stilling of its body and the presence of food. Instead, the sensorimotor reflexes that control sucking and stilling will gradually adapt themselves to anything with which they are regularly associated. Of course, their caretakers ensure that, as a matter of fact, signs of hunger and the presence of food are indeed regularly coordinated with opportunities to suck and still, so it is no surprise that infants quickly learn to do both in the right circumstances. Indeed, it is entirely by virtue of this social relationship that the infant is in a position to acquire new a (and highly functional) skill, namely feeding, even though it has no previous propensity to do any such thing. And as intelligence proceeds through higher and higher levels, the same principle applies: it is intelligence’s own operations (individual and social) that ‘bootstrap’ it upwards.
But why is this process of self-development so critical to intelligence? Part of the answer has already been given – that if the development of intelligence were in any way predetermined, it would also be prejudiced, which is the very antithesis of intelligence. In this respect at least, approaches to AI such as neural networks seem to pass the test, in principle at least.
Nevertheless, if this argument is correct, it has a critical corollary that even neural nets meet far less convincingly. This is that, if intelligence is not to develop in terms defined even partly externally – by some form or programming, predisposition, or whatever – then it must develop in intimate connection with the objects and the world. So not only must intelligence develop itself, but it must do so in direct relation to a world of things and events. In other words, it cannot be spoon-fed ‘data’ about the world: whatever it ‘knows’ about the world it must know from its own activity. There is, after all, no alternative that would not divert its development in much the same way as a putative ‘programme’.
So the peculiar way in which human babies develop into intelligent beings is not merely a contingent fact about human beings, which the AI community can safely ignore; on the contrary, as with so many of the apparently incidental features of natural intelligence, it is absolutely critical to its nature as intelligence. From the point of view of AI, then, the issue is why an artificial intelligence should be expected to develop in any other way.
This scepticism about neural networks is mainly based on the assumption that the crucial developmental process that accounts for the genuinely progressive and qualitative nature of developmental stages in intelligence is one of reflecting abstraction, leading to higher level structures. Although neural networks are good at constant, non-deterministic adjustment, this does not include the possibility that the network in question could have any insight into what it is itself doing or, a fortiori, into its own nature, and so abstracts from the empirical and functional details of its activity to the principles on which that activity could or should be based. No reflecting abstraction, no intelligence.
It should be noted that reflexivity in not a capacity with which intelligence comes equipped in advance. Rather, it follows from the very abstractness of the sensorimotor reflexes from which the development of natural intelligence sprang in the first place, and the on-going process of abstraction from content and context on which all subsequent cognitive development is based.
Structural and functional overlaps.
4.5. Notes on the development of intelligence
Even where it was capable of operating on symbols of any kind, and not just mathematics, the requirement for a ‘definite method’ would preclude any real form of self-development, even though Turing and others have noted the awesome power that would be possessed by a machine that was capable of a genuinely self-development.
A very powerful example of the completely opposed approaches to development taken by artificial and natural intelligence, and the radical consequences of this difference, is the formation of formal logic and mathematics.
As research into natural intelligence shows quite unequivocally, formal logic or mathematics are anything but primitive to intelligence. What is more, it follows rather than precedes the intelligence in question’s progressive mastery of all manner of concrete contents and contexts. This contrast arises from another fundamental difference between natural and artificial intelligence: that their ability of each to master abstract and concrete problems is completely opposite to the other’s. In existing research, an artificial intelligence’s ability to master formal skills such as chess playing is vast and easy to implement, but the ability of the average AI to walk around a carelessly arranged room is quickly overwhelmed by the dizzying complexities of the situation. For natural intelligence, by contrast, the smallest toddler get around the room very well, thank you very much, but all but the simplest of abstractions is beyond it. Indeed, strictly formal logic and mathematics are the among natural intelligence’s highest and latest accomplishments.
This massive contradiction is the product of the fundamentally opposed ways in which natural and artificial intelligence’s develop.
The sophistication of the insights and concepts at intelligence’s disposal develops in parallel with intelligence itself. Piagetian research reveals a progressive sequence, starting from tracking static perceptual and behavioural properties, through basic rational properties such as the continuing existence of things even though I am not immediate aware of them (object ‘permanence’), relationships between things (series, classes, and so on), the equivalence of things that can be related by analogy, part-whole relationships and metaphor, systems of relationships between relationships, and so on. Likewise, the methods intelligence uses to organise and apply these insights also mature, from a small child’s attempts to simply literally bend the world to its will to the massive forces and relations of production of an industrial society, or the elaborate, mutually supporting theories, hypotheses and data on which the entire paraphernalia of scientific research is based.
There is, finally, a central fact about how natural intelligence develops itself, which seems to be completely lost on AI research. This is the fact that the development of intelligence is not only composed of intelligence’s own native structures but is actively accomplished by intelligence itself. All the forms of structure and function described above operate and come into existence through intelligence’s own activity. Taking the case of the formation of feeding in the sensorimotor infant, the various elements referred to – the infant’s sensorimotor reflexes, the somatic environment and the empirical environment that is managed by the infant’s caretakers – do not all lay an equal part. The child’s mother can put her baby to her breast as often as she pleases, but if the child does not suckle then feeding will neither take place right then nor be constructed as a new form of spontaneous activity over the longer term. Likewise for the subject as a whole: as Piaget argued, it exists only to the extent that it is assembled out of the infant’s native sensorimotor reflexes, and only to the extent that it is actively assembled by the interaction of those sensorimotor reflexes.
More generally, all the structures and functioning of intelligence come into existence and operate through the intelligence in question’s actions, unintentional though those actions and their outcome often necessarily are. However material a thing or event may be and however forcefully it imposes itself on the developing intelligence, it exists for that intelligence only to the extent and in the sense that that intelligence makes sense of it. That is, after all, the difference between a ‘thing’ and an object – the intelligibility of the latter. And likewise for the world: it exists as a world only to the extent and in the sense that the individual intelligence at its hub learns to navigate it, and to the extent that that intelligence comes to recognise that the objects out of which its world is composed are shared with other intelligences.
A typical strategy employed by AI is to divide process from data. This by definition separates the content of any artificial intelligence (as defined by the data on which it works) from its form (as defined by the processes through which it works).
In short, intelligence almost certainly cannot be built, but it may be possible to create the appropriate seeds and nurture them. Other intelligent beings (human beings, for example) may create circumstances favouring its development, but only the intelligence in question can bring itself into existence, however inadvertent that process may be.
Looking at current approaches to AI, it is again very hard to see much that resembles true intelligence. Clearly neither a rule-based not a data-driven approach will do here, since both assume a predefined structure, even if there can be some ‘learning’ around the sides. Nor will neural nets do: although they develop by virtue of their own activity (although the role of the trainer should not be neglected, or compared with that of the human infant’s caretakers), neural nets do not proceed by a process of objectification. Their performance and competence may be progressively refined, but the method whereby these improvements come about is not specifically intelligent.
In particular, there is no point at which the net is able to detect what it is itself doing (unless it is trained to – at which point we re-enter the vicious circle) in the manner of a child or a society reflecting on its own activity, detecting the underlying relationships, systems, principles and higher level structures through which it is acting, and actively using this insight to refine, control and evaluate its subsequent activity. For example, a neural net is unlikely to ask itself ‘Is this logical?’ or ‘Am I being consistent?’ without first being trained to see this as an important criterion of or method for success. So it is out of the question that a neural net should pass through a series of qualitatively higher stages of intelligence, or, in the absence of any self-recognition, intelligence of any kind, for this is the essential method whereby intelligence first bootstraps itself into existence and then develops itself to maturity.
5. So is artificial intelligence possible?
Not only is a true artificial intelligence possible, it is all but inevitable. The basis for this claim is the view (which is shall summarise briefly below) that the relationship between natural intelligence and technology is such that .
5.1. The relationship between intelligence and technology
If the Church-Turing thesis does not support the uses to which it is put by the AI community, then is there any basis for a true artificial intelligence? The answer remains a decided Yes, not only in the sense that we may not achieve real artificial intelligence but we will almost certainly be able to construct very clever robot. Despite my doubts about contemporary AI, I have no doubt that a true artificial intelligence, in every way the equal of natural intelligence, will one day be among us. In other words, I am quite sure that a ‘strong’ AI will one day be created; the only doubt I have is whether it will be based on computation.
Technology (with culture) is the normal method for creating rational objects – increasingly permanent, autonomous and universal. In fact all objects can be regarded in a technological light. But technology proper – the creation of objective means to rational ends –
5.2. Rules for building a true artificial intelligence
- Understand what is to be achieved:
ú A natural intelligence is a person. The basis of personhood is personality – the individual as a complete, rounded totality, rather than a concatenation of technical functions and skills.
ú A natural intelligence is a self-regulating being. It knows what it is doing, and it is only by virtue of that knowledge that it can be said to act intelligently.
ú A natural intelligence is embodied. This does not mean just that it possesses a body with structures and functions through which it applies its intelligence but also that it is through its embodied relationship to the world that that intelligence comes into existence in the first place and subsequently develops.
ú The functions and skills we typically identify as intelligent are only superficial expressions of a single internal, more or less coherent, consistent, complete and correct structure, whose basic ‘components’ are a subject, objects and a world.
ú On the individual plane, the core function of intelligence is cognition. Cognition is structural and existential as well as functional and empirical. It is not a separate skill or capability but the synthesis of many other structures of activity. On the other hand, once it exists, cognitive structure imposes itself through rules of coherence, consistency, completeness and correctness on all other forms of activity: imperfectly but both deliberately and with increasing recognition and exploitation of the role such criteria play in making the world make sense.
ú On the supra-individual plane, intelligence builds, works and develops through cultural and technological artefacts that embody not data but meaning, significance and value.
ú ‘Data processing’ is a superficial conceptual abstraction from these realities that assumes that intelligence is defined by empirical and functional performance. The real basis of intelligence is structural logic and existential awareness, commitment and involvement.
- Understand the starting point:
ú The starting point of intelligence is a collection of overlapping sensorimotor reflexes. However, these overlaps are not characterised by any special properties or propensities to resolve themselves. They are not inherently ‘self-organising’ in any active sense, but their mutual structural and functional overlaps and lack of developmental commitments and constraints ensure that they do end up organising themselves into a more or less unified totality.
ú The basic structures from which the development of any natural intelligence proceeds consist of sensorimotor reflexes. The basic structure of an sensorimotor reflex is as its name suggests: it links a sensory input to a motor output. Sensorimotor reflexes are embodied – that is, they exist in a real body, with sensory and motor affectors, channels and effectors.
ú These reflexes are not connected to either a specific empirical or functional context or context. Hence the reflexive nature of the sensorimotor reflex, which, like a knee-jerk reflex, has a form but that form is not connected to either any functional need or to any particular empirical stimulant.
- Understand the process:
ú The sole motivator for cognitive development is contradiction: there is no inherent desire or drive for knowledge, performance, effectiveness or improvement, important though these factors may be in supporting and guiding development.
ú Development consists of making sense of things. Making sense of things consists of resolving contradictions within and between actions, things and events.
ú Development consists of resolving the structural and functional overlaps between sensorimotor reflexes. For example, a visual tracking reflex that is following a light may be disrupted by an auditory attention reflex that centres the head (which is assumed to be the location for both eyes and ears, in a roughly human configuration) on a sound. This happens because both reflexes use the control of head movement. Resolving the conflict consists of articulating the overlap and so allowing each reflex to operate simultaneously, either with or without a head movement, as circumstances dictate.
ú Development consists of abstraction – on things (empirical abstraction) and on action itself (reflecting abstraction).
ú There is no boundary between process and data.
- Understand the role of the caretaker:
ú Every natural intelligence creates itself. You cannot build one, you can only grow it. The development of intelligence requires opportunity, care, nurture, guidance, not programming.
ú The newborn intelligence-to-be is more or less helpless. This is crucial to the development of intelligence, which is only so free to develop and so flexible in its development because of the vacuity of its starting point.
ú As a result, a natural intelligence is able to develop other because other intelligences are there to support it. However, not only does this primitive sociality ensure that it participates in and contributes to interpersonal and social relationships, but the structure of that support provides at least as much of the food for its development as its content. It is easiest to develop into a rational being when one is surrounded by and participates in rationality.
ú The role of the caretaker is only partly to instruct or create in his or her own image. Much more it is to ensure that the new intelligence creates itself in its own image – whatever that turns out to be.
The reader will have anticipated that I have no very convincing arguments of a positive nature to support my views.
Alan Turing, ‘Computing machinery and intelligence’
Sweet is the lore which Nature brings;
Our meddling intellect
Misshapes the beauteous form of things:-
We murder to dissect.
Wordsworth, The Tables Turned.
So intelligence is not a bundle of more or less tightly coupled modular capabilities. That is simply a conceptual convenience that happens to suit and mimic the AI community’s preferred tool – the computer program. Nor is intelligence like an atomic pile going super-critical or the layers of an onion – two more of Turing’s preferred analogies. Of course, such models suit any methodology that operates on the basis of either reductionism or ‘divide and conquer’. But intelligence is a unity in the same sense that life is a unity: you cannot make sense of being alive by adding bone structure to the function of the lungs, by pealing away successive layers of structure, or by expecting that concatenating isolated chemical processes will lead to a sudden emergence of something as radically new as life. Rather, you can make sense of such detail only when you have grasped the nature of life as a whole. Of course, in practice the two approaches go hand in hand, but there is certainly no question of deciding in advance that the whole is no more than the sum of its parts. That is not to turn the whole question of intelligence into a vicious circle. As I have already observed rather too often, it is not as though we lacked a convincing model of what natural intelligence is like. Piaget and his successors have been refining our understanding of intelligence for more than seven decades. Their models embody vast empirical research and theoretical analysis of exceptional rigour. The problem is that the AI community has assumed the validity of certain models that suit their methods, even though there is little to substantiate them in terms of how human beings really act.
So back this essay’s epigraph. Historically speaking at least, Chomsky’s contribution to our understanding of cognition, or at least of language, is now wholly unassailable, and I would not think of criticising that part of Jerry Fodor’s bold assertion. But Turing? Three things militate against crediting him with any such contribution. The first is his behaviourism, which assumes that intelligence is intelligible in terms of its empirical or functional expressions. For example, when Turing proposed translating the question, ‘Can machines think?’ into ‘Are there imaginable digital computers which would do well in the imitation game’ (ie, the Turing test), he described this as ‘the more accurate form of the question’. For reasons are now I hope clear, this translation is quite false. Nor should one regard the paradox of speaking of Turing’s contribution in the same breath as Chomsky as merely an amusing irony, given the fatal blow Chomsky struck against behaviourism: their positions on human nature in general and psychology in particular are completely opposed on precisely this point. The second is the inability of any computational system to explain cognition, as I have argued above. And the third is Turing’s own explicit account of what he thought his analysis explained (quoted above), namely specifically unintelligent forms of activity, in which the marvellous creative individuals human beings can be were quite openly replaced by thoughtless drudges. If we are looking for replacements for thoughtless drudgery (which can always be defined in functional or empirical terms), then the computer seems to be an excellent candidate – for exactly the same reasons why computation can play no part in any explanation of truly intelligent activity.
I should emphasise (again) that the rational properties of objects are by no means peripheral or secondary, especially not for intelligence. Insight and criticism are remarkable capacities, but even they pale into insignificance when placed next to the wider roles played (in the right conditions) by object permanence, autonomy and universality and the objectivity (in both subjective and objective senses) for which they provide. Indeed, objectivity is so central to intelligence that history and consciousness, technology and culture would be impossible without it.
This paper is designed to make two points. This first is very general, namely that no AI programme makes very much sense without a clear conception of natural intelligence. It simply is not good enough to argue that ‘AI’ is an ‘approach’, a ‘technique’, a ‘discipline’, and so on. On the contrary, it is quite overtly a theory of intelligence, and as such explicitly claims to provide an objective explanation of the objective structure, function and (occasionally) genesis of intelligence in all its forms, natural and artificial. This point is quite independent of the particular model of intelligence presented here: the question of exactly what such an approach, technique or discipline is for, if not to create an artefact that will be truly intelligent. If the Wright brothers had failed, would anybody have accepted that they were merely researching an ‘approach’, a ‘technique’ or a ‘discipline’ of ‘artificial flight’ that didn’t involve – or even really aim at – any actual flying. No matter what interesting things we might subsequently achieve in the areas of ticketing software, departure lounges and undrinkable coffee, ‘flying machines’ that did not actually fly would be a ludicrous oxymoron, and so is an ‘artificial intelligence’ that has no claims to being intelligent.
The second purpose of this paper is to show just how far any existing AI programme is from constructing anything like a real intelligence. No artificial intelligence exhibits insight or criticism or is a subject in the above sense, no artificial intelligence constructs objects, and no artificial intelligence inhabits anything resembling a world. What is more, no existing AI programme is designed to produce anything that possesses any of these attributes except in the most superficial sense. So inevitably, no existing AI programme (as that phrase is currently used) will ever produce an actual intelligence.
I hope that it is also clear what it is that creating a true artificial intelligence does not require. Most especially it doesn’t need telling how to be intelligent and it doesn’t need to be provided with a set of prior modules or data or even training (at least, not in the manner of a neural net). After all, a human child can be provided with training and models (skills, for example) too, but they would not make the child intelligent in the sense that teaching me grammar, vocabulary and so on would make me a speaker of French.
As the evidence from cognitive development demonstrates abundantly, both the competence and the aptitude that define intelligence follow from the ordinary processes of development. We classify and play games, have symbol systems and can relate to others socially because these things flows directly from the further differentiation and integration of the basic reflexes with which all intelligent beings are born. Everything seems to flow from first principles – although in this case, the principles in question are drawn from action, not from data. Not that there seems to be any particular set of reflexes from which you must start this process. Rather, if the evidence from cognitive development in handicapped children or the contrasting pictures of apes and children cognition are to be believed, it is a combination of complete openness and global capacity that seems to be crucial. It is a far cry from any current AI programme.
So, as it currently stands, AI is a lost cause. And it will not find its place in the science of intelligence proper until it is prepared to define what exactly it means by this elusive term – and moreover appreciates that its core concepts will have to be drawn from the study of natural intelligence – the very antithesis of how it operates at the moment.
Berry, J.W. and Dasen, P.R., (1974). Culture and Cognition: Readings in Cross-Cultural Psychology. London: Methuen.
Bloch, M. and Parry, J. (eds) (1989). Money and the Morality of Exchange. Cambridge: Cambridge University Press.
Block, N.J. (1981). Psychologism and behaviourism. Philosophical Review 90: 5–43.
Bringsjord, S. (1998). Chess is too easy. MIT Technology Review, March/April: 23-28.
Bringsjord, S. and Zenzen, M. (1997). Cognition is not computation: The argument from irreversibility. Synthese. 113: 285-320.
Copeland, B.J. (2002). The Church-Turing thesis. Stanford Encyclopedia of Philosophy. Stanford: Stanford University.
Dennett, D.C. (1996). Kinds of Mind, New York: Basic Books.
Fodor, J.A. (1983). The Modularity of Mind. Cambridge, Massachusetts: MIT Press.
Foss, B.M. (1969). Determinants of Infant Behaviour, 4. London: Methuen.
Frede, M., and Striker, G. (1996). Rationality in Greek Thought. Oxford : Oxford University Press.
Hallpike, C.R. (1979). The Foundations of Primitive Thought. Oxford: Oxford University Press.
Harnad, S. (1982). Neoconstructivism: A unifying constraint for the cognitive sciences. In Language, Mind and Brain. Simon, T. and Scholes, R. (eds), pp.1–11. Hillsdale, NJ: Erlbaum.
Hodges, A. (1992). Alan Turing: The Enigma. London: Random House.
Hodges, A. (1997). Turing. A Natural Philosopher. London: Orion.
Hopper, A. (2002). The confusion of software with intelligence. Prometheus Research Group: www.prometheus.org.uk.
Inhelder, B. and Piaget, J. (1958). The Growth of Logical Thinking from Childhood to Adolescence. London: Routledge and Kegan Paul.
Inhelder, B. and Piaget, J. (1964). The Early Growth of Logic in the Child: Classification and Seriation. London: Routledge and Kegan Paul.
Kamenka, E. (1989). Bureaucracy. Oxford: Basil Blackwell.
Karmiloff-Smith, A. (1992). Beyond Modularity. A Developmental Perspective on Cognitive Science. London: MIT Press.
Kopp, C.B. and Shaperman, J. (1973). Cognitive development in the absence of object manipulation during infancy. Abbreviated version published in Developmental Psychology, 9, p.430ff.
Luria, A.R. (1976). Cognitive Development: Its Cultural and Social Foundations. Cambridge, Mass.: Harvard University Press.
Merleau-Ponty, M. (1973). Consciousness and the Acquisition of Language. Evanston, Northwestern University Press.
Parker, S.T., and McKinney, M.L. (1999). Origins of Intelligence: The Evolution of Cognitive Development in Monkeys, Apes and Humans. Baltimore: Johns Hopkins University Press.
Pepperberg, I.M. (1999). Pepperberg, I.M. (1999). The Alex Studies. Cognitive and Communicate Abilities of Grey Parrots. Cambridge, Mass.: Harvard University Press.
Piaget, J. (1951). Play, Dreams and Imitation in Childhood. London: Routledge and Kegan Paul.
Piaget, J. (1953a). The Origins of Intelligence in Children. London: Routledge and Kegan Paul.
Piaget, J. (1953b). Logic and Psychology. Manchester: Manchester University Press.
Piaget, J. (1972). Psychology and Epistemology: Towards a Theory of Knowledge. Harmondsworth: Penguin.
Piaget, J. (2001). Studies in Reflecting Abstraction. Hove: Psychology Press.
Piaget, J., and Inhelder, B. (1967). The Child’s Conception of Space. New York: W.W. Norton.
Richards, R.J. (1987). Darwin and the Emergence of Evolutionary Theories of Mind and Behaviour. Chicago: University of Chicago Press.
Robinson, R.J. (2004). The History of Human Reason. Prometheus Research Group: www.prometheus.org.uk.
Robinson, R.J. (in preparation). The Birth of Reason. Prometheus Research Group: www.prometheus.org.uk.
Saygin, A.P., Cicekli, I., and Akman, V. (2000). Turing test: 50 years later. Minds and Machines, 10: 463–518.
Smith, L. (1993). Necessary Knowledge: Piagetian Perspectives on Constructivism. Hove, Lawrence Erlbaum.
Spufford, P. (1988). Money and its Use in Medieval Europe. Cambridge: Cambridge University Press.
Turing, A.M. (1937). On computable numbers, with an application to the Entscheidungsproblem. Proc. Lond. Math. Soc. Series 2, 42: 230-265.
Turing, A.M. (1939). Systems of logic based on ordinals. Proc. Lond. Math. Soc. Series 2, 45: 161-228.
Turing, A.M. (1948). Intelligent Machinery. National Physical Laboratory Report. In Meltzer, B., Michie, D. (eds) 1969. Machine Intelligence 5. Edinburgh: Edinburgh University Press.
Turing, A.M. (1950). Computing machinery and intelligence. Mind. 59: 236, 433-460.
Vauclair, J. (1996). Animal Cognition: An Introduction to Modern Comparative Psychology. London: Harvard University Press.
Wiener, N. (1954). The Human Use of Human Beings. Cybernetics and Society. New York: Doubleday.
 Readers may object that, by referring to computers rather than computation or Turing Machines, I trivialise the whole computational methodology. This is partly correct, but as only a little of the argument that follows relies on a detailed knowledge of computationalism as such, referring to computers is a helpful shorthand. But just as importantly, my reading of the AI literature has always given me the very strong impression that, so powerful is the impetus created by computers as such (which are surely the most extraordinary tools humanity has ever invented) that it is just as much this beguiling contraption as computationalism proper that defines and drives AI. On the other hand, in many quarters both inside and outside AI the term ‘computation’ seems to have gone the same way as ‘information’ or ‘adaptation’ – that is, it has degenerated into technobabble cum shibboleth, whose main purpose is as much to declare one’s membership of the tribe as to indicate a serious intellectual stance.
 For a history of the Turing test and its would-be successors, see Saygin, Cicekli, and Akman (2000).
 Turing (1950).
 Saygin, Cicekli, and Akman (2000).
 Turing (1950: 442). Turing seems to have been making a general point rather than a precise prediction, and suggested other targets at other times (eg, Hodges, 1992, 348-349).
 On the latter slightly surreal proposition, see Robinson (2004).
 Hopper (2002).
 Saygin, Cicekli, and Akman (2000: 484).
 Turing (1948: 9).
 Quoted in Copeland (2002: ‘Some Key Remarks by Turing’).
 Quoted in Copeland (2002: ‘Some Key Remarks by Turing’). Emphasis added. See Hodges (1992: 484 and passim) for other examples.
 Turing, A.M. (1950: 436).
 Hodges (1992: 96–97).
 Turing (1950: 435).
 The Loebner Prize offers a reward to any machine that can pass the Turing test.
 Turing (1950).
 Samples are reprinted in Saygin, Cicekli, and Akman (2000).
 Significant elements of Turing’s model were anticipated by Emil Post (Hodges, 1992: 125).
 Copeland (2002: ‘Misunderstandings of the Thesis’). Copeland then proceeds to criticise a number of the most eminent supporters of computation, including Dennett, Paul and Patricia Churchland, Smolensky and Newell, for misconstruing the Church-Turing thesis in just this way.
 For the range, operation and development of many of intelligence’s other qualities, see Robinson (2004).
 M. Frede in Frede and Striker (1996: 157–173).
 Richards (1987).
 On non-human insight, see Vauclair (1996); on primates in particular, see Parker and McKinney (1999). See also Pepperberg (1999).
 Eg, Luria (1976); Bovet in Berry and Dasen (1974: 311–334); Hallpike (1979: 190–191).
 Contrast Harnad’s view that, although cognition need not be computational in itself, its study must result in the derivation of computable input-output relationships, on the grounds that this is essential to scientific method (Harnad, 1982).
 Piaget and Inhelder (1967).
 Eg, Hallpike (1979); Parker and McKinney (1999).
 Turing (1937).
 D. Frede in M. Frede and G. Striker (1996: 29-58).
 Piaget (1953a).
 Robinson (in preparation).
 Dennett (1996: 43).
 Turing (1950).
 Piaget and Inhelder (1967)
 Robinson (2004: Chapter 6).
 Piaget (1972).
 Turing (1939). According to Hodge (1997: 29, 34ff), Turing eventually came to doubt the link between intuitions of this kind and uncomputable operations. However, his revised view, that anything of which the human mind was capable was computable, does not affect the general point being made here. He came to the conclusion that ‘machines of sufficient complexity would have the capacity evolving into behaviour that had never been explicitly programmed‘– a familiar assertion from AI enthusiasts but, even if true (and complexity breeding unexpected results is hardly a radical idea), still not an argument for even the most complex machine eventually becoming intelligent.
 Fodor (1983).
 Karmiloff-Smith (1992).
 Fodor (1983).
 Block (1981: 5).
 Block (1981: 18).
 Wiener (1954: 61).
 On the relationship between perception and rationalisation, see Merleau-Ponty (1973); Frederic Bartlett’s work on memory, summarised in many textbooks, initiated research on how rationalisation organises memory. On rationalisation in general, see Robinson (2004).
 Quoted in Hodges (1992: 404).
 Robinson (in press).
 On pre-formal forms of money and bureaucracy see Bloch and Parry (1989) or Spufford (1988), and Kamenka (1989) respectively. On the general emergence of formal systems in history, see Robinson (2004).
 Bringsjord suggests replacing the Turing test with the question: ‘Can a machine tell a story?’ (Bringsjord, 1998). This would require genuine insight, in this case into ‘feeling what it’s like to be these characters’. However, his criteria of success remain equally wedded to fooling the onlooker. It would be interesting to know whether any biologist would accept the ability of a machine to look and behave like an animal as a valid test of whether it was alive. For a still more compelling aesthetic test, see Benjamin Zephaniah’s ‘Protest Poets’. Interesting, Turing claimed that both the ability to play chess and teaching a machine English should be considered equally valid and important tests (Turing, 1950).
 Inhelder and Piaget (1964, 1958).
 See Robinson (in press).
 Piaget (2001).
 Reversibility is one of the most general of all Piaget’s themes. For a recent summary, see Piaget (2001: 153:167). For the centrality of reversibility to rational action, see Smith (1993).
 Note that the real patterns of our activity and the sense we make of them are not necessarily the same: intelligence is not infallible. As the entire history of ideology demonstrates, it is extremely common, indeed unavoidable in the early stages, for intelligence’s objectification of its own activity to be incorrect, and for the errors to persist for centuries. That is how it is possible for highly rational beings to deliberately burn thousands of innocent people to death for witchcraft – not to mention murdering millions of Armenians, Jews, Slavs, gipsies, Tutsis and others for their membership of a particular ‘race’, not to mention as many millions of Russian and Chinese peasants for their class affiliation and hundreds of thousands of innocent bystanders who could be bracketed out of existence as ‘collateral damage’.
 Bringsjord and Zenzen (1997) argue that ‘computation is reversible; cognition isn’t: ergo, cognition isn’t computation’ – apparently exactly the contrary of the case I shall make here. However, Bringsjord and Zenzen are looking at a quite different aspect of reversibility, and their general argument is quite compatible with my own.
 But see Hodges (1992: 377ff).
 Turing (1950: 457). Interestingly, Turing is known (Hodges, 1992: 480) to have attended Piaget’s lectures of child development to the Manchester Department of Philosophy in 1952 (published as Piaget 1953b). This does not appear to have influenced his approach.
 Robinson (in press).
 Turing (1950: 442).
 Gouin-Décarie in Foss (1969). For some of the developmental consequences of this irreducible reflex nature, see Kopp and Shaperman (1973).