In the fictional dialogue entitled Impressions of Theophrastus Such (first edition, 1879), George Eliot’s first-person narrator envisions the development of machines that can think, affect the physical world, and reproduce themselves. Humans suffer as a result, devolving into passivity and ultimately becoming extinct:

Under such uncomfortable circumstances our race will have diminished with the diminishing call on their energies, and by the time that the self-repairing and reproducing machines arise, all but a few of the rare inventors, calculators, and speculators will have become pale, pulpy, and cretinous from fatty or other degeneration, and behold around them a scanty hydrocephalous offspring. As to the breed of the ingenious and intellectual, their nervous systems will at last have been overwrought in following the molecular revelations of the immensely more powerful unconscious race, and they will naturally, as the less energetic combinations of movement, subside like the flame of a candle in the sunlight. Thus the feebler race, whose corporeal adjustments happened to be accompanied with a maniacal consciousness which imagined itself moving its mover, will have vanished, as all less adapted existences do before the fittest—i.e., the existence composed of the most persistent groups of movements and the most capable of incorporating new groups in harmonious relation. Who—if our consciousness is, as I have been given to understand, a mere stumbling of our organisms on their way to unconscious perfection—who shall say that those fittest existences will not be found along the track of what we call inorganic combinations, which will carry on the most elaborate processes as mutely and painlessly as we are now told that the minerals are metamorphosing themselves continually in the dark laboratory of the earth’s crust? Thus this planet may be filled with beings who will be blind and deaf as the inmost rock, yet will execute changes as delicate and complicated as those of human language and all the intricate web of what we call its effects, without sensitive impression, without sensitive impulse: there may be, let us say, mute orations, mute rhapsodies, mute discussions, and no consciousness there even to enjoy the silence.

In On Liberty (1859), John Stuart Mill had not forecast such a future as explicitly as Eliot would do, but he used it as a thought-experiment to demonstrate that the point of life is to develop one’s own capacities, not to accomplish any practical ends. A life in which important matters are handled by other minds–or by machines–is a life devoid of value:

He who lets the world, or his own portion of it, choose his plan of life for him, has no need of any other faculty than the ape-like one of imitation. He who chooses his plan for himself, employs all his faculties. He must use observation to see, reasoning and judgment to foresee, activity to gather materials for decision, discrimination to decide, and when he has decided, firmness and self-control to hold to his deliberate decision. And these qualities he requires and exercises exactly in proportion as the part of his conduct which he determines according to his own judgment and feelings is a large one. It is possible that he might be guided in some good path, and kept out of harm’s way, without any of these things. But what will be his comparative worth as a human being? It really is of importance, not only what men do, but also what manner of men they are that do it. Among the works of man, which human life is rightly employed in perfecting and beautifying, the first in importance surely is man himself. Supposing it were possible to get houses built, corn grown, battles fought, causes tried, and even churches erected and prayers said, by machinery—by automatons in human form—it would be a considerable loss to exchange for these automatons even the men and women who at present inhabit the more civilised parts of the world, and who assuredly are but starved specimens of what nature can and will produce. Human nature is not a machine to be built after a model, and set to do exactly the work prescribed for it, but a tree, which requires to grow and develop itself on all sides, according to the tendency of the inward forces which make it a living thing.

The possibility that AI will render us extinct remains speculative, 150 years after Eliot posited it. But there is an urgent, present threat that AI tools will “guide” us along “some good path” and thereby block “the free development of individuality,” which “is one of the leading essentials of well-being.”

See also: the difference between human and artificial intelligence: relationships; artificial intelligence and problems of collective action; what I would advise students about ChatGPT; the human coordination involved in AI; the design choice to make ChatGPT sound like a human etc. I owe the reference to Eliot to Harry Law.

The Association for Computing Machinery (ACM) has 110,000 members. As artificial intelligence rapidly acquires users and uses, some ACM members see an analogy to nuclear physics in the 1940s. Their profession is responsible for technological developments that can do considerable good but that also pose grave dangers. Like physicists in the era of Einstein and Oppenheimer, computer scientists have developed ideas that are now in the hands of governments and companies that they cannot control.

The ACM’s Technology Policy Council has published a brief by David Leslie and Francesca Rossi with the following problem statement: “The rapid commercialization of generative AI (GenAI) poses multiple large-scale risks to individuals, society, and the planet that require a rapid, internationally coordinated response to mitigate.”

Considering that this brief is only three pages long (plus notes), I think it offers a good statement of the issue. It is vague about solutions, but that may be inevitable for this type of document. The question is what should happen next.

One rule-of-thumb is that legislatures won’t act on demands (let alone friendly suggestions) unless someone asks them to adopt specific legislation. In general, legislators lack the time, expertise, and degrees of freedom necessary to develop responses to the huge range of issues that come before them.

This passage from the brief is an example of a first step, but it won’t generate legislation without a lot more elaboration:

Policymakers confronting this range of risks face complex challenges. AI law and policy thus should incorporate end-to-end governance approaches that address risks comprehensively and “by design.” Specifically, they must address how to govern the multiphase character of GenAI systems and the foundation models used to construct them. For instance, liability and accountability for lawfully acquiring and using initial training data should be a focus of regulations tailored to the FM training phase.

The last quoted sentence begins to move in the right direction, but which policymakers should change which laws about which kinds of liability for whom?

The brief repeatedly calls on “policymakers” to act. I am guessing the authors mean governmental policymakers: legislators, regulators, and judges. Indeed, governmental action is warranted. But governments are best seen as complex assemblages of institutions and actors that are in the midst of other social processes, not as the prime movers. For instance, each legislator is influenced by a different set of constituents, donors, movements, and information. If a whole legislature manages to pass a law (which requires coordination), the new legislation will affect constituents, but only to a limited extent. And the degree to which the law is effective will depend on the behavior of many other actors inside of government who are responsible for implementation and enforcement and who have interests of their own.

This means that “the government” is not a potential target for demands: specific governmental actors are. And they are not always the most promising targets, because sometimes they are highly constrained by other parties.

In turn, the ACM is a complex entity–reputed to be quite decentralized and democratic. If I were an ACM member, I would ask: What should policymakers do about AI? But that would only be one question. I would also ask: What should the ACM do to influence various policymakers and other leaders, institutions, and the public? What should my committee or subgroup within ACM do to influence the ACM? And: which groups should I be part of?

In advocating a role for the ACM, it would be worth canvassing its assets: 110,000 expert members who are employed in industry, academia, and governments; 76 years of work so far; structures for studying issues and taking action. It would also be worth canvassing deficits. For instance, the ACM may not have deep expertise on some matters, such as politics, culture, social ethics, and economics. And it may lack credibility with the diverse grassroots constituencies and interest-groups that should be considered and consulted. Thus an additional question is: Who should be working on the social impact of AI, and how should these activists be configured?

I welcome the brief by David Leslie and Francesca Rossi and wouldn’t expect a three-page document to accomplish more than it does. But I hope it is just a start.

See also: can AI help governments and corporations identify political opponents?; the design choice to make ChatGPT sound like a human; what I would advise students about ChatGPT; the major shift in climate strategy (also about governments as midstream actors).