Cybernetics or Control and Communication in the Animal and the Machine
Norbert Wiener
Second Edition · MIT Press · 1961
被誉为信息时代的奠基之作,维纳在此书中创立了控制论(Cybernetics)学科,
将通信、控制与统计力学统一在同一个理论框架下,
深刻影响了人工智能、计算神经科学、自动化和社会学的发展。
Foreword
Doug Hill
Doug Hill
Foreword
Doug Hill
Doug Hill
Foreword
Doug Hill
Doug Hill
It tells you something about Norbert Wiener's qualifications as a prophet that Cybernetics has been republished twice since it first appeared in 1948, and that on each of those occasions his thoughts and predictions about the technologies he helped create have seemed more ingenious, more prescient, and, in some respects, more troubling. The New York Times acknowledged as much in 1965 when it turned over the front page of its prestigious book review section to Cybernetics' first reissue--a rare editorial honor indeed for a book that was, at the time, seventeen years old.1 (The decision to grant cover status may have been influenced by the fact that a month earlier Wiener had won, posthumously, a National Book Award for God & Golem, Inc.) The Times' reviewer, John Pfeiffer, called Cybernetics "one of the finest intellectual adventure stories ever written." He also noted, however, that upon the book's initial publication, Wiener's warnings about the potential dangers of computer technology had created "something of a scandal." Three years after Hiroshima and Nagasaki, Americans were worried that scientists might not be able to contain the powers they'd unleashed. Wiener didn't reassure them.
His introduction concludes by noting his distress that those who contributed to the development of cybernetics, including himself, "stand in a moral position which is, to say the least, not very comfortable." The techniques emerging from their ideas, he said, could be used for evil as well as good, and it was obvious he expected they would be. True, he said, many of his colleagues held out hope that cybernetics could lead to "a better understanding of man and society." Speaking for himself, Wiener said, "I am compelled to say that it is a very slight hope." This latest reissue of Cybernetics likewise makes its appearance at a time when people are wondering whether we have our technologies firmly in control. And it's safe to say Wiener would not be surprised that those anxieties arise from just the sorts of technological power he wrote about in 1948.
The likely impact of automation on employment is an example. Over the past few years, a plethora of articles, books, studies, and conferences have warned, often alarmingly, that advances in artificial intelligence and robotics pose a significant threat to a significant number of jobs. Some of these postulate that half or more of all jobs could soon be replaced.
"This will be the fastest transition humankind has experienced," said Kai-Fu Lee, an AI expert who has held executive positions at Google, Microsoft, and Apple, "and we're not ready for it."2 Recent reports of thriving employment don't disprove automation's impact; the decisive factor is how many of those jobs offer meaningful employment at a living wage, especially for unskilled workers.3 Two MIT economists, Erik Brynjolfsson and Andrew McAfee, say we're witnessing the birth of the "Second Machine Age." In an interview with the Harvard Business Review, McAfee noted that digital technologies are doing for human brainpower what the steam engine and related technologies did
for human muscle power during the Industrial Revolution (aka the First Machine Age).4 If that doesn't sound familiar, it will after reading Cybernetics, where Wiener uses exactly the same comparison. Wiener notes in the book that he'd attempted to alert the labor unions of the threats posed by automation to their membership. With a few exceptions, he said, he found union leadership "totally unprepared to enter into the larger political, technical, sociological, and economic questions which concern the very existence of labor." Today we seem more willing to talk about those questions.
Whether we're willing to do something about them is another question. The potentially ruinous impact of communication technologies on democracy is another issue that Wiener anticipated with uncanny accuracy. The most fundamental conviction of cybernetic theory is that it all comes down to messages (i.e., information) sent and responded to (i.e., feedback).
The functionality of a machine, organism, or society depends on the quality of those messages. Faithful exchanges of information allow the machine, organism, or society to maintain a state of homeostasis. Information corrupted by various forms of noise leads to the opposite.
In Cybernetics, Wiener calls the mass media of his day the single greatest anti-homeostatic influence in modern society. One wishes that the techno-utopians of Silicon Valley had heeded his point that genuine community extends only so far as the transmission of effective--meaning homeostatic--information. As the scale, scope, and speed of information technologies have increased, so has the potential for corruption.
Certainly Mark Zuckerberg failed to appreciate that Facebook's "global community" of two billion users would inevitably produce countless messages that were antithetical to homeostasis, and thus to genuine community.
That the routine operation of computer technologies can lead to disaster was a point Wiener stressed repeatedly. Thinking machines are relentlessly literal-minded, he said. They mechanically perform the tasks we ask them to, even though their interpretation of those tasks may be diametrically opposed to what we had in mind.
So it is that Facebook's algorithms have posted videos of murders as efficiently as they've posted videos of marriages. Speed is another routine feature of automation that Wiener frequently warned could thwart our intentions. As he put it in a 1960 essay for Science magazine, "By the very slowness of our human actions, our effective control of our machines may be nullified.
By the time we are able to react to information conveyed by our senses and stop the car we are driving, it may already have run head-on into a wall."5 He was using the car image metaphorically, of course; the fatal accidents involving Tesla's and Uber's automated vehicles were still many decades in the future. Still, during those decades we've become ever more dependent on any number of complex computer systems that aren't as stable as we like to think. The failure of a relay switch in a power station in Niagara Falls, New York, on November 9, 1965, triggered a series of mechanical and electrical events that blacked out power to the entire northeastern United States and significant portions of Canada; now we're warned that Russian hackers may be as interested in disrupting America's power grid as they have been in disrupting its elections.6 In 2010, computerized trading programs were blamed for a "flash crash" on Wall Street, when the Dow Jones Industrial Average plunged more than 900 points in a matter of minutes.
in minutes, and unfortunately billions of dollars of damage can occur in the meantime." Stocks rebounded quickly on that occasion, but a 2018 report from J.P. Morgan warned that a more extensive crash spurred by electronic trading algorithms could lead to social chaos.7 Given the profits they engender, it's unlikely we'll stop using electronic trading programs anytime soon, which points to the driver of technological folly that most enraged Wiener: greed, for both money and power. He regularly railed against the "hucksters" in commerce and "gadget worshippers" in science whose cupidity leads irrevocably, he believed, to "no homeostasis whatever." Readers will find piquant examples of Wiener's disdain for the captains of capitalist industry in Cybernetics; here's one from his follow-up book, The Human Use of Human Beings: So long as anything remained of the rich endowment of nature with which we started, our national hero has been the exploiter who has done the most to turn this endowment into ready cash.
In our theories of free enterprise, we have exalted him as if he had been the creator of the riches which he has stolen and squandered. We have lived for the day of our prosperity, and we have hoped that some benevolent heaven would forgive our excesses and make life possible for our impoverished grandchildren. This is what is known as the fifth freedom.
Under these conditions, the new resources given by invention have naturally been turned to the even more rapid exploitation of the resources of our soil. ... This game has another name than that of war. It is also called beggar-your-neighbor.8
Where did this willingness to so vehemently attack the powers that be come from? The same place, I suspect, that accounted for the magnificent originality of Wiener's science, and also for the notorious difficulties of his personality. That place was his childhood.
Wiener showed signs of genius almost from infancy. According to Dark Hero of the Information Age, the superb biography by Flo Conway and Jim Siegelman, he was reading by the age of three and reciting in Greek and Latin by the age of five. As a child, he was consuming books of philosophy and literature that would enrich his writing as an adult.
By the time he was ten, he was studying chemistry, geometry, physics, botany, and zoology; by the age of eleven, he'd entered college. He earned his undergraduate degree in mathematics from Tufts at fourteen, spent a year studying philosophy at Cornell, and earned his doctorate with a thesis on mathematical logic from Harvard at seventeen.9 Wiener joined the faculty of MIT at the age of 23 and remained there until his death in 1964. Like many, if not most, prodigies, Wiener emerged from childhood bearing psychological scars, many inflicted by his father.
Unable to find a public or private school that could keep up with his son, Leo Wiener, an instructor in Slavic languages and literature at Harvard, began teaching him at home. The course of study was expansive in the subject areas it covered but brutal in its execution. In the first volume of his autobiography, Ex-Prodigy: My Childhood and Youth (like Cybernetics, a surprise best seller), Wiener recalls a regimen of "systematic belittling." Mistakes were greeted by explosions of temper, his father progressing rapidly from demanding teacher to "avenger of blood." This explains the title Wiener had originally chosen for his memoir: "The Bent Twig."10 His academic precocity made Wiener a minor celebrity--a newspaper profile on the occasion of his entry into Tufts called him "The Most Remarkable Boy in the World"11--but did him no favors socially.
permanent self-identification as an outsider. Wiener suffered, professionally as well as personally, from his uniqueness, but he also used it. As he put it in the second volume of his autobiography, I Am a Mathematician, "I knew very well that I was competitive beyond the run of younger mathematicians, and I knew equally that this was not a pretty attitude.
However, it was not an attitude which I was free to assume or to reject. I was quite aware that I was an out among ins and that I would get no shred of recognition that I did not force. If I was not to be welcomed, well then, let me be too dangerous to be ignored."12 Cybernetics shows frequent evidence of Wiener's substantial ego, but just as often shows his willingness to share credit with his collaborators and other workers in the field.
Claude Shannon, whose reputation as the father of information theory would come to overshadow Wiener's, receives several mentions. (Shannon earned his masters and doctorate degrees at MIT and, according to Conway and Siegelman, spent considerable time talking to Wiener about cybernetics there.) The text is also peppered with examples of Wiener's literary and philosophical chops; it's not often we encounter pages of mathematical formulae interspersed with name-dropping references to the likes of Balzac and Hume, Kipling and Carroll. It's likely that the breadth of Wiener's erudition and ambition contributed to his eclipse by Shannon. As the subtitle of Cybernetics shows, Wiener set out to explain how information is the lingua franca of both animal and machine, a mission that consciously involved exploring, as he put it, "the boundary regions of science." Thus, cybernetics as Wiener conceived it is physically embodied--understanding the workings of the human brain and nervous system was a central goal, though at one point Wiener studied signal transmission in the nerve fibers
of cats--yet consciously holistic and theoretically unlimited. As the historian Steve J. Heims put it, cybernetics "leads to a more unified view of the world, including its human inhabitants, where everything is connected to everything else."13 Shannon's information theory, by contrast, is both highly abstract and intensely practical, resolutely focused on moving electronic messages from point A to point B.
For him, wrote the literary scholar N. Katherine Hayles, information is emphatically disembodied, "a probability function with no dimensions, no materiality, and no necessary connection with meaning."14 With the advent of digital technology, Shannon's more circumscribed approach paradoxically lent itself more readily to a host of real-life applications--they're at work in the smartphones, fitness trackers, and countless other devices we use today. So it is that Shannon has typically received credit for jump-starting the computer revolution while Wiener's legacy remained relatively obscure.
Many scientists believe that is changing. In the burgeoning age of artificial intelligence, the line between human and machine is becoming increasingly blurred. Cybernetics was far ahead of its time in anticipating how fruitful those parallels could be.15 There were personal as well as theoretical reasons for Wiener's fading.
His books made him a nationally recognized figure, but he used that fame to take principled public stands against the use of science--his science--to develop new ways to wage war, exploit labor, and destroy the environment. In these concerns he was, again, far ahead of his time, as well as uncommonly outspoken. In an article for the Bulletin of Atomic Scientists, he said that "the degradation of the position of the scientist as an independent worker and thinker to that of a morally irresponsible stooge in a science-factory has proceeded even more rapidly
and devastatingly than I had expected."16 Putting his money, and his career, where his mouth was, Wiener refused to take on research funded by the military, which effectively excluded him from much of the cutting-edge work in computers at a crucial moment in their history. (I can imagine Wiener smiling with approval of the Google engineers who insisted in 2018 that the company withdraw from its commitments to develop advanced AI capabilities for the Pentagon.17) Wiener also declined offers from corporations who sought his advice on automating their factories. "Those who suffer from a power complex," he wrote in The Human Use of Human Beings, "find the mechanization of man a simple way to realize their ambitions."18 The impact of Wiener's ideas was also undermined by his own demons. According to Conway and Siegelman, he suffered from periods of severe depression during which he would lose control of his emotions, breaking into sobbing or shouting fits at the slightest provocation.
More than once he threatened suicide. Wiener's colleagues at MIT, although mostly unaware of those episodes, nonetheless at times found him imperious, temperamental, maddeningly absent-minded, and neurotically self-absorbed. Perhaps most important, Conway and Siegelman report that in 1951 Wiener abruptly abandoned leadership of a new neurophysiology laboratory at MIT--one that held immense promise for furthering the growth of cybernetics--after his wife convinced him, falsely, that more than one of his partners in the venture had seduced his nineteen-year-old daughter.
Wiener's fears that his colleagues were attempting to steal credit for cybernetics made him more likely to believe the story.19 Reviewers of Ex-Prodigy remarked that, despite his many successes, Wiener's life story seemed imbued with sadness. His writings reveal a deep skepticism about human nature. Earlier,
I mentioned Wiener's comment that he used his outsider status to fuel his creative drive. Steve Heims noted a similar sentiment in a journal article Wiener cowrote, criticizing Rudyard Kipling for his "fear-based conformity." In a comment that could easily have reflected Wiener's view of himself, the piece added, "The marginal man is always relatively the more civilized human being."20 Throughout his life, Wiener harbored another conviction that in my view was equally important in fueling his genius, and that was his recognition of uncertainty. It's remarkable that at the age of ten he wrote a paper entitled "The Theory of Ignorance," in which he proclaimed "the impossibility of man's being certain of anything." He mentions the paper in the second volume of his autobiography.
"Even at that time," he wrote, "I was struck by the impossibility of originating a perfectly tight theory with the aid of so loose a mechanism as the human mind." A key moment on his path to cybernetics came when he discovered the statistical mechanics theories of his scientific hero, Josiah Willard Gibbs, whose applications of probability demonstrated, Wiener said, that chance is part of the "warp and weft" of physics. This in turn affirmed his belief that it would be a mistake to view the world as purely rational.21 That Wiener's warnings about technology haven't been heeded in the long run isn't surprising; technological advance and enthusiasm for technological advance have been drowning out the warnings of skeptics for centuries. The surprise was that Cybernetics and his other books and articles received as much initial attention as they did.
Nonetheless, the attitudes of the public do seem to have shifted in his direction. After years of enamored embrace of Google, Facebook, Amazon, YouTube, and any number of other internet attractions, a host of problems--loss of
privacy, tech addiction, and subverted elections among them--has, for many, provoked serious doubts. And it's not just the users who are having second thoughts.22 In several of his works, including this one, Wiener mentions Goethe's "The Sorcerer's Apprentice," "The Fisherman and the Genie" from One Thousand and One Nights, and W. Jacobs's short story, "The Monkey's Paw." In one way or another, the protagonists of all three tales end up asking themselves how the magic they thought would deliver them could have gone so horribly wrong.
A series of scandals, beginning with revelations of social media's contributions to the election of Donald Trump, has caused many of Silicon Valley's would-be wizards to ask themselves, belatedly, the same question. Doug Hill is the author of Not So Fast: Thinking Twice About Technology.
When I wrote the first edition of Cybernetics some thirteen years ago, I did it under some serious handicaps which had the effect of piling up unfortunate typographical errors, together with a few errors of content. Now I believe the time has come to reconsider cybernetics, not merely as a program to be carried out at some period in the future, but as an existing science. I have therefore taken this opportunity to put the necessary corrections at the disposal of my readers and, at the same time, to present an amplification of the present status of the subject and of the new related modes of thought which have come into being since its first publication.
If a new scientific subject has real vitality, the center of interest in it must and should shift in the course of years. When I first wrote Cybernetics, the chief obstacles which I found in making my point were that the notions of statistical information and control theory were novel and perhaps even shocking to the established attitudes of the time. At present, they have become so familiar as a tool of the communication engineers and of the designers of automatic controls that the chief danger against which I must guard is that the book may seem trite and commonplace.
design and in biology has come to be well established. The role of information and the technique of measuring and transmitting information constitute a whole discipline for the engineer, for the physiologist, for the psychologist, and for the sociologist. The automata which the first edition of this book barely forecast have come into their own, and the related social dangers against which I warned, not only in this book, but also in its small popular companion The Human Use of Human Beings,1 have risen well above the horizon.
Thus it behooves the cyberneticist to move on to new fields and to transfer a large part of his attention to ideas which have arisen in the developments of the last decade. The simple linear feedbacks, the study of which was so important in awakening scientists to the role of cybernetic study, now are seen to be far less simple and far less linear than they appeared at first view. Indeed, in the early days of electric circuit theory, the mathematical resources for systematic treatment of circuit networks did not go beyond linear juxtapositions of resistances, capacities, and inductances.
This meant that the entire subject could be adequately described in terms of the harmonic analysis of the messages transmitted, and of the impedances, admittances, and voltage ratios of the circuits through which the messages were passed. Long before the publication of Cybernetics, it came to be realized that the study of non-linear circuits (such as we find in many amplifiers, in voltage limiters, in rectifiers, and the like) did not fit easily into this frame. Nevertheless, for want of a better methodology, many attempts were made to extend the linear notions of the older electrical engineering well beyond the point where the newer types of apparatus could be naturally expressed in terms of these.
When I came to M.I.T. around 1920, the general mode of putting the questions concerning non-linear apparatus was to look for a direct extension of the notion of impedance which would cover linear as well as non-linear systems. The result was that the study of non-linear electrical engineering was getting into a state comparable with that of the last stages of the Ptolemaic system of astronomy, in which epicycle was piled on epicycle, correction upon correction, until a vast patchwork structure ultimately broke down under its own weight. Just as the Copernican system arose out of the wreck of the overstrained Ptolemaic system, with a simple and natural heliocentric description of the motions of the heavenly bodies instead of the complicated and unperspicuous Ptolemaic geocentric system, so the study of non-linear structures and systems, whether electric or mechanical, whether natural or artificial, has heeded a fresh and independent point of commencement.
I have tried to initiate a new approach in my book Nonlinear Problems in Random Theory.2 It turns out that the overwhelming importance of a trigonometric analysis in the treatment of linear phenomena does not persist when we come to consider non-linear phenomena. There is a clear-cut mathematical reason for this. Electrical circuit phenomena, like many other physical phenomena, are characterized by an invariance with respect to a shift of origin in time.
A physical experiment which will have arrived at a certain stage by 2 o'clock if we started at noon, will have arrived at the same stage at 2:15 if we started at 12:15. Thus the laws of physics concern invariants of the translation group in time. The trigonometric functions sin nt and cos nt show certain important invariants with respect to the same translation group.
will go into the function e iω (t +τ ) = e iωτ $e^{i\omega t}$ of the same form under the translation which we obtain by adding τ to t. As a consequence, a cos n (t + τ ) + b sin n (t + τ ) = ( a cos nτ + b sin nτ ) cos nt + ( b cos nτ − a sin nτ ) sin nt = a1 cos nt + b1 sin nt In other words, the families of functions A$e^{i\omega t}$ and A cos ωt + B sin ωt are invariant under translation. Now there are other families of functions which are invariant under translations.
If I consider the so-called random walk in which the movement of a particle under any time interval has a distribution dependent only on the length of that time interval and independent of everything that has happened up to its initiation, the ensemble of random walks will also go into itself under the time translation. In other words, the mere translational invariance of the trigonometric curves is a property shared by other sets of functions as well. The property which is characteristic of the trigonometric functions in addition to these invariants is that A$e^{i\omega t}$ + B$e^{i\omega t}$ = ( A + B ) $e^{i\omega t}$ so that these functions form an extremely simple linear set.
combination of two. It is this specific property which creates the value of harmonic analysis in the treatment of the linear properties of electric circuits. The functions $e^{i\omega t}$ are characters of the translation group and yield a linear representation of this group.
When, however, we deal with combinations of functions other than addition with constant coefficients--when for example we multiply two functions by one another--the simple trigonometric functions no longer show this elementary group property. On the other hand, the random functions such as appear in the random walk do have certain properties very suitable for the discussion of their non-linear combinations. It is scarcely desirable for me to go into the detail of this work here, for it is mathematically rather complicated, and it is covered in my book Nonlinear Problems in Random Theory.
The material in that book has already been put to considerable use in the discussion of specific non-linear problems, but much remains to be done in carrying out the program laid down there. What it amounts to in practice is that an appropriate test input for the study of non-linear systems is rather of the character of the Brownian motion than a set of trigonometric functions. This Brownian motion function in the case of electric circuits can be generated physically by the shot effect.
This shot effect is a phenomenon of irregularity in electrical currents which arises from the fact that such currents are carried not as a continuous stream of electricity but as a sequence of indivisible and equal electrons. Thus electric currents are subject to statistical irregularities which are themselves of a certain uniform character and which can be amplified up to the point at which they constitute an appreciable random noise.
As I shall show in Chapter IX, this theory of random noise can be put into practical use not merely for the analysis of electrical circuits and other non-linear processes but for their synthesis as well.3 The device which is used is the reduction of the output of a non-linear instrument with random input to a well-defined series of certain orthonormal functions which are closely related to the Hermite polynomials. The problem of the analysis of a non-linear circuit consists in the determination of the coefficients of these polynomials in certain parameters of the input by a process of averaging. The description of this process is rather simple.
In addition to the black box which represents an as yet unanalyzed non-linear system, I have certain bodies of known structure which I shall call white boxes representing the various terms in the expansion desired.4 I put the same random noise into the black box and into a given white box. The coefficient of the white box in the development of the black box is given as an average of the product of their outputs. While this average is taken over the entire ensemble of shot-effect inputs, there is a theorem which allows us to replace this average in all but a set of cases of probability by an average taken over time.
To obtain this average, we need to have at our disposal a multiplying instrument by which we can get the product of the outputs of the black and the white box, as well as an averaging instrument, which we can base on the fact that the potential across a condenser is proportional to the quantity of electricity held in the condenser and hence to the time integral of the current flowing through it. Not only is it possible to determine the coefficients of each white box constituting an additive part of the equivalent representation of the black box one by one, but it is also possible to determine these quantities simultaneously. It is even possible by
the use of appropriate feedback devices to make each one of the white boxes automatically adjust itself to the level corresponding to its coefficient in the development of the black box. In this manner we are able to construct a multiple white box which, when it is properly connected to a black box and is subjected to the same random input, will automatically form itself into an operational equivalent of the black box even though its internal structure may be vastly different. These operations of analysis, synthesis, and automatic self-adjustment of white boxes into the likeness of black boxes can be carried out by other methods which have been described by Professor Amar Bose5 and by Professor Gabor.6 In all of them there is a use of some process of working in, or learning, by choosing appropriate inputs for the black and white boxes and comparing them; and in many of these processes, including the method of Professor Gabor, multiplication devices play an important role.
While there are many approaches to the problem of multiplying two functions electrically, this task is not technically easy. On the one hand, a good multiplier must work over a large range of amplitudes. On the other hand, it must be so nearly instantaneous in its operation that it will be accurate up to high frequencies.
Gabor claims for his multiplier a frequency range running to about 1,000 cycles. In his inaugural dissertation for the chair of Professor of Electrical Engineering at the Imperial College of Science and Technology of the University of London, he does not state explicitly the amplitude range over which his method of multiplication is valid nor the degree of accuracy to be obtained. I am awaiting very eagerly an explicit statement of these properties so that we can give a good evaluation of the multiplier for use in other pieces of apparatus dependent on it.
All of these devices in which an apparatus assumes a specific structure or function on the basis of past experience lead to a very interesting new attitude both in engineering and in biology. In engineering, devices of similar character can be used not only to play games and perform other purposive acts but to do so with a continual improvement of performance on the basis of past experience. I shall discuss some of these possibilities in Chapter IX of this book.
Biologically, we have at least an analogue to what is perhaps the central phenomenon of life. For heredity to be possible and for cells to multiply, it is necessary that the heredity-carrying components of a cell--the so-called genes--be able to construct other similar heredity-carrying structures in their own image. It is, therefore, very exciting for us to be in possession of a means by which engineering structures can produce other structures with a function similar to their own.
I shall devote Chapter X to this, and in particular shall discuss how oscillating systems of a given frequency can reduce other oscillating systems to the same frequency. It is often stated that the production of any specific kind of molecule in the image of existing ones has an analogy to the use of templates in engineering whereby we can use a functional element of a machine as the pattern on which another similar element is made. The image of the template is a static one, and there must be some process by which one gene molecule manufactures another.
我将把第十章用于讨论这个问题,特别将讨论给定频率的振荡系统如何能够降低其频率而不改变其基本结构。
I give the tentative suggestion that frequencies, let us say the frequencies of molecular spectra, may be the pattern elements which carry the identity of biological substances; and the self-organization of genes may be a manifestation of the self-organization of frequencies which I shall discuss later. I have already spoken of learning machines in a general way. I shall devote a chapter to a more detailed discussion of these
machines and potentialities and some of the problems of their use. Here I wish to make a few comments of a general nature. As will be seen in Chapter I, the notion of learning machines is as old as cybernetics itself.
学习机器的潜力和一些使用它们的问题,在这里我只想做一些一般性的评论。这些机器代表了自动化的前沿。
In the anti-aircraft predictors which I described, the linear characteristics of the predictor which is used at any given time depend on a long-time acquaintance with the statistics of the ensemble of time series which we desire to predict. While a knowledge of these characteristics can be worked out mathematically in accordance with the principles which I have given there, it is perfectly possible to devise a computer which will work up these statistics and develop the short-time characteristics of the predictor on the basis of an experience which is already observed by the same machine as is used for prediction and which is worked up automatically. This can go far beyond the purely linear predictor.
In various papers by Kallianpur, Masani, Akutowicz, and myself,7 we have developed a theory of non-linear prediction which can at least conceivably be mechanized in a similar manner with the use of long-time observations to give the statistical basis for short-time prediction. The theory of linear prediction and of non-linear prediction both involve some criteria of the goodness of fit of the prediction. The simplest criterion, although by no means the only usable one, is that of minimizing the mean square of the error.
This is used in a particular form in connection with the functionals of the Brownian motion which I employ for the construction of non-linear apparatus, inasmuch as the various terms of my development have certain orthogonality properties. These ensure that the partial sum of a finite number of these terms is the best simulation of the apparatus to be imitated, which can be made by the employment of these terms if the mean square criterion of error is to be maintained. The work of Gabor also
depends upon mean square criterion of error, but in a more general way, applicable to time series obtained by experience. The notion of learning machines can be extended far beyond its employment for predictors, filters, and other similar apparatus. It is particularly important for the study and construction of machines which play a competitive game like checkers.
Here the vital work has been done by Samuel8 and Watanabe9 at the laboratories of the International Business Machines Corporation. As in the case of filters and predictors, certain functions of the time series are developed in terms of which a much larger class of functions can be expanded. These functions can have numerical evaluations of the significant quantities on which the successful playing of a game depends.
For example, they comprise the number of pieces on both sides, the total command of these pieces, their mobility, and so forth. At the beginning of the employment of the machine, these various considerations are given tentative weightings, and the machine chooses that admissible move for which the total weighting will have a maximum value. Up to this point, the machine has worked with a rigid program and has not been a learning machine.
However, at times the machine assumes a different task. It tries to expand that function which is 1 for won games, 0 for lost games, and perhaps 21 for drawn games in terms of the various functions expressing the considerations of which the machine is able to take cognizance. In this way, it redetermines the weightings of these considerations so as to be able to play a more sophisticated game.
I shall discuss some of the properties of these machines in Chapter IX, but here I must point out that they have been sufficiently successful for the machine to be able to defeat its programmer in from 10 to 20 hours of learning and working in. I also wish to mention in that chapter some of the
work that has been done on similar machines devised for proving geometrical theorems and for simulating, to a limited extent, the logic of induction. All of this work is a part of the theory and practice of the programming of programming, which has been extensively studied in the Electronic Systems Laboratory of the Massachusetts Institute of Technology. Here it has been found cut that unless some such learning device is employed, the programming of a rigidly patterned machine is itself a very difficult task and that there is an urgent need for devices to program this programming.
Now that the concept of learning machines is applicable to those machines which we have made ourselves, it is also relevant to those living machines which we call animals, so that we have the possibility of throwing a new light on biological cybernetics. Here I wish to single out, among a variety of current investigations, a book by the Stanley-Jones on the Kybernetics (notice the spelling) of living systems.10 In this book they devote a great deal of attention to those feedbacks which maintain the working level of the nervous system as well as those other feedbacks which respond to special stimuli. Since the combination of the level of the system with the particular responses is to a considerable extent multiplicative, it is also non-linear and involves considerations of the sort we have already brought out.
This field of activity is very much alive at present, and I expect it to become much more alive in the near future. The methods of memory machines and of machines that multiply themselves which I have so far given are largely, although not entirely, those which depend on apparatus of a high degree of specificity, or of what I may call blueprint apparatus. The physiological aspects of the same process must conform more to the peculiar techniques of living organisms in which blueprints are
replaced by a less specific process, but one in which the system organizes itself. Chapter X of this book is devoted to a sample of a self-organizing process, namely, that by which narrow, highly specific frequencies are formed in brain waves. It is, therefore, largely the physiological counterpart of the previous chapter, in which I am discussing similar processes on more of a blueprint basis.
This existence of sharp frequencies in brain waves and the theories which I gave to explain how they are originated, what they can do, and what medical use may be made of them represent in my mind an important and new break-through in physiology. Similar ideas can be used in many other places in physiology and can make a real contribution to the study of the fundamentals of life phenomena. In this field, what I am giving is more a program than work already achieved, but it is a program for which I have great hopes.
It has not been my intention, either in the first edition or in the present one, to make this book a compendium of all that has been done in cybernetics. Neither my interests nor my abilities lie that way. My intention is to express and to amplify my ideas on this subject, and to display some of the ideas and philosophical reflections which led me in the beginning to enter upon this field, and which have continued to interest me in its development.
Thus it is an intensely personal book, devoting much space to those developments in which I myself have been interested, and relatively little to those in which I have not worked myself. I have had valuable help from many quarters in revising this book. I must acknowledge in particular the cooperation of Miss Constance D.
Lee of the Electrical Engineering Department of M.I.T., and Dr. Gordon Raisbeck of
在撰写新章节时,特别是在第九章和第十章的计算中,我得到了许多人的协助,包括John S. Barlow博士(马萨诸塞综合医院)的无价贡献。
Preface to the Second Edition
xlvii
the Bell Telephone Laboratories. Also, in the writing down of my new chapters, and particularly in the computations of Chapter X, in which I have considered the case of self-organizing systems which manifest themselves in the study of the electroencephalogram, I wish to mention the aid which I received from my students, John C. Kotelly and Charles E.
在撰写新章节时,特别是在新章节的计算中,我得到了许多人的协助。
Robinson, and especially the contribution of Dr. John S. Barlow of the Massachusetts General Hospital.
Robinson,特别是马萨诸塞综合医院的John S. Barlow博士的贡献。索引由James W. Davis完成。
The indexing was done by James W. Davis. Without the meticulous care and devotion of all of these I would not have had either the courage or the accuracy to turn out a new and corrected edition.
Norbert Wiener Cambridge, Massachusetts, March, 1961
诺伯特·维纳,马萨诸塞州剑桥市,1961年3月。
Part I Original Edition (1948)
I. Newtonian and Bergsonian Time
I. Newtonian and Bergsonian Time
I. Newtonian and Bergsonian Time
There is a little hymn or song familiar to every German child. It goes: "Weisst du, wieviel Sternlein stehen An dem blauen Himmelszelt? Weisst du, wieviel Wolken gehen Weithin über alle Welt?
有一首每个德国儿童都熟悉的小赞美诗或歌曲。歌词是:"Weisst du, wieviel Sternlein stehen An dem blauen Himmelszelt? Weisst du, wieviel Wolklein gehen Weithin uber alle Welt?"(你知道蓝色天穹中挂着多少颗小星星吗?你知道在全世界飘过多少朵小云彩吗?)
Gott, der Herr, hat sie gezählet Dass ihm auch nicht eines fehlet An der ganzen, grossen Zahl." W.
"Gott, der Herr, hat sie gezehlet Dass ihm auch nicht eines fehlet An der ganzen, grossen Zahl."(上帝,主,已经数过它们,使它们在全部巨大的数量中一个也不缺少。)
In English this says: "Knowest thou how many stars stand in the blue tent of heaven? Knowest thou how many clouds pass far over the whole world? The Lord God hath counted them, that not one of the whole great number be lacking." This little song is an interesting theme for the philosopher and the historian of science, in that it puts side by side two sciences which have the one similarity of dealing with the heavens above us, but which in almost every other respect offer an extreme contrast.
Astronomy is the oldest of the sciences, while meteorology is among the youngest to begin to deserve the name. The more familiar astronomical phenomena can
be predicted for many centuries, while a precise prediction of tomorrow's weather is generally not easy and in many places very crude indeed. To go back to the poem, the answer to the first question is that, within limits, we do know how many stars there are. In the first place, apart from minor uncertainties concerning some of the double and variable stars, a star is a definite object, eminently suitable for counting and cataloguing; and if a human Durchmusterung of the stars--as we call these catalogues--stops short for stars less intense than a certain magnitude, there is nothing too repugnant to us in the idea of a divine Durchmusterung going much further.
On the other hand, if you were to ask the meteorologist to give you a similar Durchmusterung of the clouds, he might laugh in your face, or he might patiently explain that in all the language of meteorology there is no such thing as a cloud, defined as an object with a quasi-permanent identity; and that if there were, he neither possesses the facilities to count them, nor is he in fact interested in counting them. A topologically inclined meteorologist might perhaps define a cloud as a connected region of space in which the density of the part of the water content in the solid or liquid state exceeds a certain amount, but this definition would not be of the slightest value to anyone, and would at most represent an extremely transitory state. What really concerns the meteorologist is some such statistical statement as, "Boston: January 17, 1950: Sky 38% overcast: Cirrocumulus." There is of course a branch of astronomy which deals with what may be called cosmic meteorology: the study of galaxies and nebulae and star clusters and their statistics, as pursued for example by Chandrasekhar, but this is a very young branch of astronomy, younger than meteorology itself, and is something
outside the tradition of classical astronomy. This tradition, apart from its purely classificatory, Durchmusterung aspects, was originally concerned rather with the solar system than with the world of the fixed stars. It is the astronomy of the solar system which is that chiefly associated with the names of Copernicus, Kepler, Galileo, and Newton, and which was the wet nurse of modern physics.
It is indeed an ideally simple science. Even before the existence of any adequate dynamical theory, even as far back as the Babylonians, it was realized that eclipses occurred in regular predictable cycles, extending backward and forward over time. It was realized that time itself could better be measured by the motion of the stars in their courses than in any other way.
The pattern for all events in the solar system was the revolution of a wheel or a series of wheels, whether in the form of the Ptolemaic theory of epicycles or the Copernican theory of orbits, and in any such theory the future after a fashion repeats the past. The music of the spheres is a palindrome, and the book of astronomy reads the same backward as forward. There is no difference save of initial positions and directions between the motion of an orrery turned forward and one run in reverse.
Finally, when all this was reduced by Newton to a formal set of postulates and a closed mechanics, the fundamental laws of this mechanics were unaltered by the transformation of the time variable t into its negative. Thus if we were to take a motion picture of the planets, speeded up to show a perceptible picture of activity, and were to run the film backward, it would still be a possible picture of planets conforming to the Newtonian mechanics. On the other hand, if we were to take a motion-picture photograph of the turbulence of the clouds in a thunderhead and reverse it, it would
look altogether wrong. We should see downdrafts where we expect updrafts, turbulence growing coarser in texture, lightning preceding instead of following the changes of cloud which usually precede it, and so on indefinitely. What is the difference between the astronomical and the meteorological situation which brings about all these differences, and in particular the difference between the apparent reversibility of astronomical time and the apparent irreversibility of meteorological time?
In the first place, the meteorological system is one involving a vast number of approximately equal particles, some of them very closely coupled to one another, while the astronomical system of the solar universe contains only a relatively small number of particles, greatly diverse in size and coupled with one another in a sufficiently loose way that the second-order coupling effects do not change the general aspect of the picture we observe, and the very high order coupling effects are completely negligible. The planets move under conditions more favorable to the isolation of a certain limited set of forces than those of any physical experiment we can set up in the laboratory. Compared with the distances between them, the planets, and even the sun, are very nearly points.
Compared with the elastic and plastic deformations they suffer, the planets are either very nearly rigid bodies, or, where they are not, their internal forces are at any rate of a relatively slight, significance where the relative motion of their centers is concerned. The space in which they move is almost perfectly free from impeding matter; and in their mutual attraction, their masses may be considered to lie very nearly at their centers and to be constant. The departure of the law of gravity from the inverse square law is most minute.
and the computation of their future and past positions, while not easy in detail, is easy and precise in principle. On the other hand, in meteorology, the number of particles concerned is so enormous that an accurate record of their initial positions and velocities is utterly impossible; and if this record were actually made and their future positions and velocities computed, we should have nothing but an impenetrable mass of figures which would need a radical reinterpretation before it could be of any service to us. The terms "cloud," "temperature," "turbulence," etc., are all terms referring not to one single physical situation but to a distribution of possible situations of which only one actual case is realized.
If all the readings of all the meteorological stations on earth were simultaneously taken, they would not give a billionth part of the data necessary to characterize the actual state of the atmosphere from a Newtonian point of view. They would only give certain constants consistent with an infinity of different atmospheres, and at most, together with certain a priori assumptions, capable of giving, as a probability distribution, a measure over the set of possible atmospheres. Using the Newtonian laws, or any other system of causal laws whatever, all that we can predict at any future time is a probability distribution of the constants of the system, and even this predictability fades out with the increase of time.
Now, even in a Newtonian system, in which time is perfectly reversible, questions of probability and prediction lead to answers asymmetrical as between past and future, because the questions to which they are answers are asymmetrical. If I set up a physical experiment, I bring the system I am considering from the past into the present in such a way that I fix certain quantities and have a reasonable right to assume that certain other quantities have known statistical distributions. I then observe
the statistical distribution of results after a given time. This is not a process which I can reverse. In order to do so, it would be necessary to pick out a fair distribution of systems which, without intervention on our part, would end up within certain statistical limits, and find out what the antecedent conditions were a given time ago.
However, for a system starting from an unknown position to end up in any tightly defined statistical range is so rare an occurrence that we may regard it as a miracle, and we cannot base our experimental technique on awaiting and counting miracles. In short, we are directed in time, and our relation to the future is different from our relation to the past. All our questions are conditioned by this asymmetry, and all our answers to these questions are equally conditioned by it.
A very interesting astronomical question concerning the direction of time comes up in connection with the time of astrophysics, in which we are observing remote heavenly bodies in a single observation, and in which there seems to be no unidirectionalness in the nature of our experiment. Why then does the unidirectional thermodynamics which is based on experimental terrestrial observations stand us in such good stead in astrophysics? The answer is interesting and not too obvious.
Our observations of the stars are through the agency of light, of rays or particles emerging from the observed object and perceived by us. We can perceive incoming light, but cannot perceive outgoing light, or at least the perception of outgoing light is not achieved by an experiment as simple and direct as that of incoming light. In the perception of incoming light, we end up with the eye or a photographic plate.
We condition these for the reception of images by putting them in a state of insulation for some time past: we dark-condition the eye to avoid after-images, and we wrap our plates in black paper to prevent halation. It is clear
that only such an eye and only such plates are any use to us: if we were given to pre-images, we might as well be blind; and if we had to put our plates in black paper after we use them and develop them before using, photography would be a very difficult art indeed. This being the case, we can see those stars radiating to us and to the whole world; while if there are any stars whose evolution is in the reverse direction, they will attract radiation from the whole heavens, and even this attraction from us will not be perceptible in any way, in view of the fact that we already know our own past but not our future. Thus the part of the universe which we see must have its past-future relations, as far as the emission of radiation is concerned, concordant with our own.
The very fact that we see a star means that its thermodynamics is like our own. Indeed, it is a very interesting intellectual experiment to make the fantasy of an intelligent being whose time should run the other way to our own. To such a being, all communication with us would be impossible.
Any signal he might send would reach us with a logical stream of consequents from his point of view, antecedents from ours. These antecedents would already be in our experience, and would have served to us as the natural explanation of his signal, without presupposing an intelligent being to have sent it. If he drew us a square, we should see the remains of his figure as its precursors, and it would seem to be the curious crystallization--always perfectly explainable--of these remains.
Its meaning would seem to be as fortuitous as the faces we read into mountains and cliffs. The drawing of the square would appear to us as a catastrophe--sudden indeed, but explainable by natural laws--by which that square would cease to exist. Our counterpart would have exactly similar ideas
concerning us. Within any world with which we can communicate, the direction of time is uniform. To return to the contrast between Newtonian astronomy and meteorology: most sciences lie in an intermediate position, but most are rather nearer to meteorology than to astronomy.
与我们相关的。在任何我们可以与之通信的世界中,时间的方向是统一的。
Even astronomy, as we have seen, contains a cosmic meteorology. It contains as well that extremely interesting field studied by Sir George Darwin, and known as the theory of tidal evolution. We have said that we can treat the relative movements of the sun and the planets as the movements of rigid bodies, but this is not quite the case.
即使天文学,正如我们已经看到的,也包含一种宇宙气象学。它还包含那个关于潮汐演化的极其有趣的领域。
The earth, for example, is nearly surrounded by oceans. The water nearer the moon than the center of the earth is more strongly attracted to the moon than the solid part of the earth, and the water on the other side is less strongly attracted. This relatively slight effect pulls the water into two hills, one under the moon and one opposite to the moon.
地球,例如,几乎被海洋包围着。比地球中心更靠近月亮的海洋水受到更大的引力吸引,形成一个潮汐隆起。
In a perfectly liquid sphere, these hills could follow the moon around the earth with no great dispersal of energy, and consequently would remain almost precisely under the moon and opposite to the moon. They would consequently have a pull on the moon which would not greatly influence the angular position of the moon in the heavens. However, the tidal wave they produce on the earth gets tangled up and delayed on coasts and in shallow seas such as the Bering Sea and the Irish Sea.
It consequently lags behind the position of the moon, and the forces producing this are largely turbulent, dissipative forces, of a character much like the forces met in meteorology, and need a statistical treatment. Indeed, oceanography may be called the meteorology of the hydrosphere rather than of the atmosphere. These frictional forces drag the moon back in its course about the earth and accelerate the rotation of the earth forward.
tend to bring the lengths of the month and of the day ever closer to one another. Indeed, the day of the moon is the month, and the moon always presents nearly the same face to the earth. It has been suggested that this is the result of an ancient tidal evolution, when the moon contained some liquid or gas or plastic material which could give under the earth's attraction, and in so giving could dissipate large amounts of energy.
This phenomenon of tidal evolution is not confined to the earth and the moon but may be observed to some degree throughout all gravitating systems. In ages past it has seriously modified the face of the solar system, though in anything like historic times this modification is slight compared with the "rigid-body" motion of the planets of the solar system. Thus even gravitational astronomy involves frictional processes that run down.
There is not a single science which conforms precisely to the strict Newtonian pattern. The biological sciences certainly have their full share of one-way phenomena. Birth is not the exact reverse of death, nor is anabolism--the building up of tissues--the exact reverse of catabolism--their breaking down.
没有一门科学完全符合严格的牛顿模式。生物科学当然不符合。活体系统本质上是不可逆的、演化的。
The division of cells does not follow a pattern symmetrical in time, nor does the union of the germ cells to form the fertilized ovum. The individual is an arrow pointed through time in one way, and the race is equally directed from the past into the future. The record of paleontology indicates a definite long-time trend, interrupted and complicated though it might be, from the simple to the complex.
By the middle of the last century this trend had become apparent to all scientists with an honestly open mind, and it is no accident that the problem of discovering its mechanisms was carried ahead through the same great step by two men working at about the same time: Charles Darwin
到上个世纪中叶,这种趋势已经对任何诚实开放的科学家变得明显,这是对科学思想的一次重大修正。
Chapter I
and Alfred Wallace. This step was the realization that a mere fortuitous variation of the individuals of a species might be carved into the form of a more or less one-directional or few-directional progress for each line by the varying degrees of viability of the several variations, either from the point of view of the individual or of the race. A mutant dog without legs will certainly starve, while a long thin lizard that has developed the mechanism of crawling on its ribs may have a better chance for survival if it has clean lines and is freed from the impeding projections of limbs.
An aquatic animal, whether fish, lizard, or mammal, will swim better with a fusiform shape, powerful body muscles, and a posterior appendage which will catch the water; and if it is dependent for its food on the pursuit of swift prey, its chances of survival may depend on its assuming this form. Darwinian evolution is thus a mechanism by which a more or less fortuitous variability is combined into a rather definite pattern. Darwin's principle still holds today, though we have a much better knowledge of the mechanism on which it depends.
The work of Mendel has given us a far more precise and discontinuous view of heredity than that held by Darwin, while the notion of mutation, from the time of de Vries on, has completely altered our conception of the statistical basis of mutation. We have studied the fine anatomy of the chromosome and have localized the gene on it. The list of modern geneticists is long and distinguished.
Several of these, such as Haldane, have made the statistical study of Mendelianism an effective tool for the study of evolution. We have already spoken of the tidal evolution of Sir George Darwin, Charles Darwin's son. Neither the connection of the idea of the son with that of the father nor the choice of the name "evolution" is fortuitous.
the origin of species, we have a mechanism by means of which a fortuitous variability, that of the random motions of the waves in a tidal sea and of the molecules of the water, is converted by a dynamical process into a pattern of development which reads in one direction. The theory of tidal evolution is quite definitely an astronomical application of the elder Darwin. The third of the dynasty of Darwins, Sir Charles, is one of the authorities on modern quantum mechanics.
This fact may be fortuitous, but it nevertheless represents an even further invasion of Newtonian ideas by ideas of statistics. The succession of names Maxwell-Boltzmann-Gibbs represents a progressive reduction of thermodynamics to statistical mechanics: that is, a reduction of the phenomena concerning heat and temperature to phenomena in which a Newtonian mechanics is applied to a situation in which we deal not with a single dynamical system but with a statistical distribution of dynamical systems; and in which our conclusions concern not all such systems but an overwhelming majority of them. About the year 1900, it became apparent that there was something seriously wrong with thermodynamics, particularly where it concerned radiation.
The ether showed much less power to absorb radiations of high frequency--as shown by the law of Planck--than any existing mechanization of radiation theory had allowed. Planck gave a quasi-atomic theory of radiation--the quantum theory--which accounted satisfactorily enough for these phenomena, but which was at odds with the whole remainder of physics; and Niels Bohr followed this up with a similarly ad hoc theory of the atom. Thus Newton and Planck-Bohr formed, respectively, the thesis and antithesis of a Hegelian antinomy.
replaced by a statistical theory very similar to that of Newton and Gibbs for large-scale phenomena, but in which the complete collection of data for the present and the past is not sufficient to predict the future more than statistically. It is thus not too much to say that not only the Newtonian astronomy but even the Newtonian physics has become a picture of the average results of a statistical situation, and hence an account of an evolutionary process. This transition from a Newtonian, reversible time to a Gibbsian, irreversible time has had its philosophical echoes.
被一种非常类似于牛顿和吉布斯用于大尺度现象的统计理论所取代,但在其中决定论被概率所取代。
Bergson emphasized the difference between the reversible time of physics, in which nothing new happens, and the irreversible time of evolution and biology, in which there is always something new. The realization that the Newtonian physics was not the proper frame for biology was perhaps the central point in the old controversy between vitalism and mechanism; although this was complicated by the desire to conserve in some form or other at least the shadows of the soul and of God against the inroads of materialism. In the end, as we have seen, the vitalist proved too much.
Instead of building a wall between the claims of life and those of physics, the wall has been erected to surround so wide a compass that both matter and life find themselves inside it. It is true that the matter of the newer physics is not the matter of Newton, but it is something quite as remote from the anthropomorphizing desires of the vitalists. The chance of the quantum theoretician is not the ethical freedom of the Augustinian, and Tyche is as relentless a mistress as Ananke.
The thought of every age is reflected in its technique. The civil engineers of ancient days were land surveyors, astronomers, and navigators; those of the seventeenth and early eighteenth centuries were clockmakers and grinders of lenses. As in ancient
times, the craftsmen made their tools in the image of the heavens. A watch is nothing but a pocket orrery, moving by necessity as do the celestial spheres; and if friction and the dissipation of energy play a role in it, they are effects to be overcome, so that the resulting motion of the hands may be as periodic and regular as possible. The chief technical result of this engineering after the model of Huyghens and Newton was the age of navigation, in which for the first time it was possible to compute longitudes with a respectable precision, and to convert the commerce of the great oceans from a thing of chance and adventure to a regular understood business.
It is the engineering of the mercantilists. To the merchant succeeded the manufacturer, and to the chronometer, the steam engine. From the Newcomen engine almost to the present time, the central field of engineering has been the study of prime movers.
Heat has been converted into usable energy of rotation and translation, and the physics of Newton has been supplemented by that of Rumford, Carnot, and Joule. Thermodynamics makes its appearance, a science in which time is eminently irreversible; and although the earlier stages of this science seem to represent a region of thought almost without contact with the Newtonian dynamics, the theory of the conservation of energy and the later statistical explanation of the Carnot principle or second law of thermodynamics or principle of the degradation of energy--that principle which makes the maximum efficiency obtainable by a steam engine depend on the working temperatures of the boiler and the condenser--all these have fused thermodynamics and the Newtonian dynamics into the statistical and the non-statistical aspects of the same science. If the seventeenth and early eighteenth centuries are the age of clocks, and the later eighteenth and the nineteenth centuries
constitute the age of steam engines, the present time is the age of communication and control. There is in electrical engineering a split which is known in Germany as the split between the technique of strong currents and the technique of weak currents, and which we know as the distinction between power and communication engineering. It is this split which separates the age just past from that in which we are now living.
构成了蒸汽机时代,而当前时代则是通信和控制的时代。在电气工程中有一个分支叫做通信工程。
Actually, communication engineering can deal with currents of any size whatever and with the movement of engines powerful enough to swing massive gun turrets; what distinguishes it from power engineering is that its main interest is not economy of energy but the accurate reproduction of a signal. This signal may be the tap of a key, to be reproduced as the tap of a telegraph receiver at the other end; or it may be a sound transmitted and received through the apparatus of a telephone; or it may be the turn of a ship's wheel, received as the angular position of the rudder. Thus communication engineering began with Gauss, Wheatstone, and the first telegraphers.
It received its first reasonably scientific treatment at the hands of Lord Kelvin, after the failure of the first transatlantic cable in the middle of the last century; and from the eighties on, it was perhaps Heaviside who did the most to bring it into a modern shape. The discovery of radar and its use in the Second World War, together with the exigencies of the control of anti-aircraft fire, have brought to the field a large number of well-trained mathematicians and physicists. The wonders of the automatic computing machine belong to the same realm of ideas, which was certainly never so actively pursued in the past as it is at the present day.
At every stage of technique since Daedalus or Hero of Alexandria, the ability of the artificer to produce a working simulacrum of a living organism has always intrigued people. This desire to
produce and to study automata has always been expressed in terms of the living technique of the age. In the days of magic, we have the bizarre and sinister concept of the Golem, that figure of clay into which the Rabbi of Prague breathed life with the blasphemy of the Ineffable Name of God. In the time of Newton, the automaton becomes the clockwork music box, with the little effigies pirouetting stiffly on top.
制造和研究自动机的能力总是以该时代的活技术的术语来表达。在魔法时代,自动机是神秘力量的产品。
In the nineteenth century, the automaton is a glorified heat engine, burning some combustible fuel instead of the glycogen of the human muscles. Finally, the present automaton opens doors by means of photocells, or points guns to the place at which a radar beam picks up an airplane, or computes the solution of a differential equation. Neither the Greek nor the magical automaton lies along the main lines of the direction of development of the modern machine, nor do they seem to have had much of an influence on serious philosophic thought.
在十九世纪,自动机是一种被美化的热机,燃烧一些可燃燃料而不是动物体内的糖原。
It is far different with the clockwork automaton. This idea has played a very genuine and important role in the early history of modern philosophy, although we are rather prone to ignore it. To begin with, Descartes considers the lower animals as automata.
This is done to avoid questioning the orthodox Christian attitude that animals have no souls to be saved or damned. Just how these living automata function is something that Descartes, so far as I know, never discusses. However, the important allied question of the mode of coupling of the human soul, both in sensation and in will, with its material environment is one which Descartes does discuss, although in a very unsatisfactory manner.
He places this coupling in the one median part of the brain known to him, the pineal gland. As to the nature of his coupling--whether or not it represents a direct action of mind on matter and of matter on mind--he is none too clear.
He probably does regard it as a direct action in both ways, but he attributes the validity of human experience in its action on the outside world to the goodness and honesty of God. The role attributed to God in this matter is unstable. Either God is entirely passive, in which case it is hard to see how Descartes' explanation really explains anything, or He is an active participant, in which case it is hard to see how the guarantee given by His honesty can be anything but an active participation in the act of sensation.
Thus the causal chain of material phenomena is paralleled by a causal chain starting with the act of God, by which He produces in us the experiences corresponding to a given material situation. Once this is assumed, it is entirely natural to attribute the correspondence between our will and the effects it seems to produce in the external world to a similar divine intervention. This is the path followed by the Occasionalists, Geulincx and Malebranche: In Spinoza, who is in many ways the continuator of this school, the doctrine of Occasionalism assumes the more reasonable form of asserting that the correspondence between mind and matter is that of two self-contained attributes of God; but Spinoza is not dynamically minded, and gives little or no attention to the mechanism of this correspondence.
This is the situation from which Leibniz starts, but Leibniz is as dynamically minded as Spinoza is geometrically minded. First, he replaces the pair of corresponding elements, mind and matter, by a continuum of corresponding elements: the monads. While these are conceived after the pattern of the soul, they include many instances which do not rise to the degree of self-consciousness of full souls, and which form part of that world which Descartes would have attributed to matter.
the creation or from minus infinity in time to the indefinitely remote future; but closed though they are, they correspond one to the other through the pre-established harmony of God. Leibniz compares them to clocks which have so been wound up as to keep time together from the creation for all eternity. Unlike humanly made clocks, they do not drift into asynchronism; but this is due to the miraculously perfect workmanship of the Creator.
Thus Leibniz considers a world of automata, which, as is natural in a disciple of Huyghens, he constructs after the model of clockwork. Though the monads reflect one another, the reflection does not consist in a transfer of the causal chain from one to another. They are actually as self-contained as, or rather more self-contained than, the passively dancing figures on top of a music box.
They have no real influence on the outside world, nor are they effectively influenced by it. As he says, they have no windows. The apparent organization of the world we see is something between a figment and a miracle.
The monad is a Newtonian solar system writ small. In the nineteenth century, the automata which are humanly constructed and those other natural automata, the animals and plants of the materialist, are studied from a very different aspect. The conservation and the degradation of energy are the ruling principles of the day.
在十九世纪,由人类构建的自动机是基于蒸汽机模型的。同样,活体有机体也被类比为热机。
The living organism is above all a heat engine, burning glucose or glycogen or starch, fats, and proteins into carbon dioxide, water, and urea. It is the metabolic balance which is the center of attention; and if the low working temperatures of animal muscle attract attention as opposed to the high working temperatures of a heat engine of similar efficiency, this fact is pushed into a corner and glibly explained by a contrast between the chemical energy of the living organism and the
thermal energy of the heat engine. All the fundamental notions are those associated with energy, and the chief of these is that of potential. The engineering of the body is a branch of power-engineering.
Even today, this is the predominating point of view of the more classically minded, conservative physiologists; and the whole trend of thought of such biophysicists as Rashevsky and his school bears witness to its continued potency. Today we are coming to realize that the body is very far from a conservative system, and that its component parts work in an environment where the available power is much less limited than we have taken it to be. The electronic tube has shown us that a system with an outside source of energy, almost all of which is wasted, may be a very effective agency for performing desired operations, especially if it is worked at a low energy level.
即使在今天,这仍然是更古典倾向的保守生理学家的主导观点;而且这种观点绝非没有价值。
We are beginning to see that such important elements as the neurons, the atoms of the nervous complex of our body, do their work under much the same conditions as vacuum tubes, with their relatively small power supplied from outside by the circulation, and that the bookkeeping which is most essential to describe their function is not one of energy. In short, the newer study of automata, whether in the metal or in the flesh, is a branch of communication engineering, and its cardinal notions are those of message, amount of disturbance or "noise"--a term taken over from the telephone engineer--quantity of information, coding technique, and so on. In such a theory, we deal with automata effectively coupled to the external world, not merely by their energy flow, their metabolism, but also by a flow of impressions, of incoming messages, and of the actions of outgoing messages.
other receptors for light; radar systems, receiving their own short Hertzian waves; hydrogen-ion-potential recorders, which may be said to taste; thermometers; pressure gauges of various sorts; microphones; and so on. The effectors may be electrical motors or solenoids or heating coils or other instruments of very diverse sorts. Between the receptor or sense organ and the effector stands an intermediate set of elements, whose function is to recombine the incoming impressions into such form as to produce a desired type of response in the effectors.
The information fed into this central control system will very often contain information concerning the functioning of the effectors themselves. These correspond among other things to the kinesthetic organs and other proprioceptors of the human system, for we too have organs which record the position of a joint or the rate of contraction of a muscle, etc. Moreover, the information received by the automaton need not be used at once but may be delayed or stored so as to become available at some future time.
This is the analogue of memory. Finally, as long as the automaton is running, its very rules of operation are susceptible to some change on the basis of the data which have passed through its receptors in the past, and this is not unlike the process of learning. The machines of which we are now speaking are not the dream of the sensationalist nor the hope of some future time.
They already exist as thermostats, automatic gyrocompass ship-steering systems, self-propelled missiles--especially such as seek their target--anti-aircraft fire-control systems, automatically controlled oil-cracking stills, ultra-rapid computing machines, and the like. They had begun to be used long before the war--indeed, the very old steam-engine governor belongs among them--but the great mechanization of the Second World War brought them into their own, and the need of handling the
extremely dangerous energy of the atom will probably bring them to a still higher point of development. Scarcely a month passes but a new book appears on these so-called control mechanisms, or servomechanisms, and the present age is as truly the age of servomechanisms as the nineteenth century was the age of the steam engine or the eighteenth century the age of the clock. To sum up: the many automata of the present age are coupled to the outside world both for the reception of impressions and for the performance of actions.
掌握原子极端危险的能量的前景将把它们推向更高的发展阶段。几乎每个月都有关于这个领域新进展的报道。
They contain sense organs, effectors, and the equivalent of a nervous system to integrate the transfer of information from the one to the other. They lend themselves very well to description in physiological terms. It is scarcely a miracle that they can be subsumed under one theory with the mechanisms of physiology.
The relation of these mechanisms to time demands careful study. It is clear, of course, that the relation input-output is a consecutive one in time and involves a definite past-future order. What is perhaps not so clear is that the theory of the sensitive automata is a statistical one.
这些机制与时间的关系需要仔细研究。当然,输入-输出关系对于理解任何系统的时间行为都是基础。
We are scarcely ever interested in the performance of a communication-engineering machine for a single input. To function adequately, it must give a satisfactory performance for a whole class of inputs, and this means a statistically satisfactory performance for the class of input which it is statistically expected to receive. Thus its theory belongs to the Gibbsian statistical mechanics rather than to the classical Newtonian mechanics.
We shall study this in much more detail in the chapter devoted to the theory of communication. Thus the modern automaton exists in the same sort of Bergsonian time as the living organism; and hence there is no reason in Bergson's considerations why the essential mode of functioning
of the living organism should not be the same as that of the automaton of this type. Vitalism has won to the extent that even mechanisms correspond to the time-structure of vitalism; but as we have said, this victory is a complete defeat, for from every point of view which has the slightest relation to morality or religion, the new mechanics is fully as mechanistic as the old. Whether we should call the new point of view materialistic is largely a question of words: the ascendancy of matter characterizes a phase of nineteenth-century physics far more than the present age, and "materialism" has come to be but little more than a loose synonym for "mechanism." In fact, the whole mechanist-vitalist controversy has been relegated to the limbo of badly posed questions.
There is a large class of phenomena in which what is observed is a numerical quantity, or a sequence of numerical quantities, distributed in time. The temperature as recorded by a continuous recording thermometer, or the closing quotations of a stock in the stock market, taken day by day, or the complete set of meteorological data published from day to day by the Weather Bureau are all time series, continuous or discrete, simple or multiple. These time series are relatively slowly changing, and are well suited to a treatment employing hand computation or ordinary numerical tools such as slide rules and computing machines.
Their study belongs to the more conventional parts of statistical theory. What is not generally realized is that the rapidly changing sequences of voltages in a telephone line or a television circuit or a piece of radar apparatus belong just as truly to the field of statistics and time series, although the apparatus by means of which they are combined and modified must in general be very rapid in its action, and in fact must be able to put out results pari passu with the very rapid alterations of input. These pieces of apparatus--telephone receivers, wave filters, automatic sound-coding devices like the Vocoder of the Bell Telephone
Laboratories, frequency-modulating networks and their corresponding receivers--are all in essence quick-acting arithmetical devices, corresponding to the whole apparatus of computing machines and schedules, and the staff of computers, of the statistical laboratory. The ingenuity needed in their use has been built into them in advance, just as it has into the automatic range finders and gun pointers of an anti-aircraft fire-control system, and for the same reasons. The chain of operation has to work too fast to admit of any human links.
One and all, time series and the apparatus to deal with them, whether in the computing laboratory or in the telephone circuit, have to deal with the recording, preservation, transmission, and use of information. What is this information, and how is it measured? One of the simplest, most unitary forms of information is the recording of a choice between two equally probable simple alternatives, one or the other of which is bound to happen--a choice, for example, between heads and tails in the tossing of a coin.
We shall call a single choice of this sort a decision. If then we ask for the amount of information in the perfectly precise measurement of a quantity known to lie between A and B, which may with uniform a priori probability lie anywhere in this range, we shall see that if we put A = 0 and B = 1, and represent the quantity in the binary scale by the infinite binary number. a1 a2 a3 ... an ..., where a1, a2, ..., each has the value 0 or 1, then the number of choices made and the consequent amount of information is infinite. Here .a1a2 a3 an =
uniformly distributed error lying over a range of length · b1 b2 ... bn ..., where bk is the first digit not equal to 0, we shall see that all the decisions from a1 to ak−1, and possibly to ak, are significant, while all the later decisions are not. The number of decisions made is certainly not far from − log 2 . b1 b2 bn
均匀分布的误差,分布在长度为 b₁, b₂, ..., bₙ 的范围内……
(3.02)
and we shall take this quantity as the precise formula for the amount of information and its definition. We may conceive this in the following way: we know a priori that a variable lies between 0 and 1, and a posteriori that it lies on the interval (a, b) inside (0, 1). Then the amount of information we have from our a posteriori knowledge is − log 2
我们将把这个量作为信息量的精确公式。这是香农信息论的基本定义:信息是可能性的对数度量。
measure of ( a, b ) measure of ( 0, 1)
(3.03)
However, let us now consider a case where our a priori knowledge is that the probability that a certain quantity should lie between x and x + dx is f1(x) dx, and the a posteriori probability is f2(x) dx. How much new information does our a posteriori probability give us? This problem is essentially that of attaching a width to the regions under the curves y = f1(x) and y = f2(x).
It will be noted that we are here assuming the variable to have a fundamental equipartition; that is, our results will not in general be the same if we replace x by x3 or any other function of x. Since f1(x) is a probability density, we shall have ∞
so that the average logarithm of the breadth of the region under f1(x) may be considered as some sort of average of the height of the logarithm of the reciprocal of f1(x). Thus a reasonable
因此,在f₁(x)下,区域宽度的对数平均值可以计算出来,并作为信息量的度量。
Chapter III
measure1 of the amount of information associated with the curve f1(x) is ∞
∫ [log f ( x)] f ( x) dx 2
−∞
(3.05)
The quantity we here define as amount of information is the negative of the quantity usually defined as entropy in similar situations. The definition here given is not the one given by R. Fisher for statistical problems, although it is a statistical definition; and can be used to replace Fisher's definition in the technique of statistics.
∫ [log f ( x)] f ( x) dx = b − a log b − a = log b − a 2
−∞
(3.06)
Using this to compare the information that a point is in the region (0, 1) with the information that it is in the region (a, b), we obtain for the measure of the difference log 2
The definition which we have given for the amount of information is applicable when the variable x is replaced by a variable ranging over two or more dimensions. In the two-dimensional case, f(x, y) is a function such that ∞
= ∫ dxφ ( x ) log 2 φ ( x ) + ∫ dyψ ( y ) log 2 ψ ( y )
and the amount of information from independent sources is additive. An interesting problem is that of determining the information gained by fixing one or more variables in a problem. For example, let us suppose that a variable u lies between x and x + dx with the probability exp ( − x2 2a ) dx
variable v lies between the same two limits with a probability
exp ( − x2 2b ) dx
2π b . How much information do we gain con-
cerning u if we know that u + v = w? In this case, it is clear that u = w − v, where w is fixed. We assume the a priori distributions of u and v to be independent.
关于u,如果我们知道u + v = w,我们能知道什么?在这种情况下,显然u = w − v,其中v是已知分布的随机变量。
Then the a posteriori distribution of u is proportional to ( w − x )2 x2 2 a + b exp − exp − = c1 exp − ( x − c2 ) 2a 2 2ab b
where c1 and c2 are constants. They both disappear in the formula for the gain in information given by the fixing of w. The excess of information concerning x when we know w to be that which we have in advance is
Note that this expression (Eq. 3.091) is positive, and that it is independent of w. It is one-half the logarithm of the ratio of the sum of the mean squares of u and v to the mean square of v. If v has only a small range of variation, the amount of information concerning u which a knowledge of u + v gives is large, and it becomes infinite as b goes to 0.
We can consider this result in the following light: let us treat u as a message and v as a noise. Then the information carried by a precise message in the absence of a noise is infinite. In the presence of a noise, however, this amount of information is finite, and it approaches 0 very rapidly as the noise increases in intensity.
We have said that amount of information, being the negative logarithm of a quantity which we may consider as a probability, is essentially a negative entropy. It is interesting to show that, on the average, it has the properties we associate with an entropy. Let ϕ(x) and ψ(x) be two probability densities; then [ϕ(x) + ψ(x)]/2 is also a probability density.
In other words, the overlap of the regions under ϕ(x) and ψ(x) reduces the maximum information belonging to ϕ(x) + ψ(x). On the other hand, if ϕ(x) is a probability density vanishing outside (a, b), ∞
is a minimum when ϕ(x) = 1/(b − a) over (a, b) and is zero elsewhere. This follows from the fact that the logarithm curve is convex upward. It will be seen that the processes which lose information are, as we should expect, closely analogous to the processes which gain entropy.
They consist in the fusion of regions of probability which were originally distinct. For example, if we replace the distribution of a certain variable by the distribution of a function of that variable which takes the same value for different arguments, or if in a function of several variables we allow some of them to range unimpeded over their natural range of variability, we lose information. No operation on a message can gain information on the average.
它们包括最初可区分的概率区域在获得额外信息后的融合。这个过程是通信中信息丢失的基础。
Here we have a precise application of the second law of thermodynamics in communication engineering. Conversely, the greater specification of an ambiguous situation, on the average, will, as we have seen, generally gain information and never lose it. An interesting case is when we have a probability distribution with n-fold density f(x1, ..., xn) over the variables (x1, ..., xn), and where we have m dependent variables y1, ..., ym.
How much information do we get by fixing these m variables? First let them be fixed between the limits y1*, y1* + dy1*; ...; ym*, ym +
通过固定这m个变量,我们能获得多少信息?首先让它们固定为精确值,然后考虑由此产生的信息量。
Chapter III
Let us take as a new set of variables x1, x2, ..., xn−m, y1, y2, ..., ym. Then over the new set of variables, our distribution function will be proportional to f(x1, ..., xn) over the region R given by y1 * y1 y1 * + dy1 *, , ym * ym ym * + dym * and 0 outside. Thus the amount of information obtained by the specification of the y's will be dx ∫ dx f ( x , , x ) log f ( x , , x ) ∫ 1
Closely related to this problem is the generalization of that which we discussed in Eq. 3.13; in the case just discussed, how much information do we have concerning the variables x1, ..., xn−m alone? Here the a priori probability density of these variables is ∞
与此问题密切相关的是我们在前一节中讨论的问题的推广。这里我们考虑的是更一般的约束和变量类型。
∫ dx −∞
n − m +1
∞
∫ dxn f ( x1 , , xn ) −∞
(3.14)
and the un-normalized probability density after fixing the y*'s is y1 *, , y m * n − m +1 , , xn
而在固定y*后的非归一化概率密度为 f(x₁, …, x_{n−m}, y₁*, …, y_m*)
∑ J x
−1
f ( x1 , , xn )
(3.141)
Time Series, Information, and Communication
where the ∑ is taken over all sets of points (xn−m+1, ..., xn) corresponding to a given set of y*'s. On this basis, we may easily write down the solution to our problem, though it will be a bit lengthy. If we take the set x1, ..., xn−m) to be a generalized message, the set (xn−m+1, ..., xm) to be a generalized noise, and the y*'s to be a generalized corrupted message, we see that we have given the solution of a generalization of the problem of Expression 3.141.
We have thus at least a formal solution of a generalization of the message-noise problem which we have already stated. A set of observations depends in an arbitrary way on a set of messages and noises with a known combined distribution. We wish to ascertain how much information these observations give concerning the messages alone.
这样,我们至少得到了消息-噪声问题推广的一个形式解。该推广考虑了更复杂的噪声结构和多维消息。
This is a central problem of communication engineering. It enables us to evaluate different systems, such as amplitude modulation or frequency modulation or phase modulation, as far as their efficiency in transmitting information is concerned. This is a technical problem and not suitable to a detailed discussion here; however, certain remarks are in order.
这是通信工程的一个核心问题。它使我们能够评估在给定噪声水平下通过给定信道可以可靠传输的最大信息量。
In the first place, it can be shown that with the definition of information given here, with a random "static" on the ether equidistributed in frequency as far as power is concerned, and with a message restricted to a definite frequency range and a definite power output for this range, no means of transmission of information is more efficient than amplitude modulation, although other means may be as efficient. On the other hand, the information transmitted by this means is not necessarily in the form most suitable for reception by the ear or by any other given receptor. Here the specific characteristics of the ear and of other receptors must be considered by employing a theory very similar to the one just developed.
首先,可以证明,根据所给出的信息定义,当信源和噪声均为高斯分布时,信道容量具有一个简单表达式。
In general, the efficient
一般而言,高效的
Chapter III
use of amplitude modulation or any other form of modulation must be supplemented by the use of decoding devices adequate to transforming the received information into a form suitable for reception by human receptors or use by mechanical receptors. Similarly, the original message must be coded for the greatest compression in transmission. This problem has been attacked, at least in part, in the design of the "Vocoder" system by the Bell Telephone Laboratories, and the relevant general theory has been presented in a very satisfactory form by Dr.
Shannon of those laboratories. So much for the definition and technique of measuring information. We shall now discuss the way in which information may be presented in a form homogeneous in time.
Let it be noted that most of the telephone and other communication devices are actually not attached to a particular origin in time. There is indeed one operation which seems to contradict this, but which really does not. This is the operation of modulation.
This, in its simplest form, converts a message f(t) into one of the form f(t) sin (at + b). If, however, we regard the factor sin (at + b) as an extra message which is put into the apparatus, it will be seen that the situation will come under our general statement. The extra message, which we call the carrier, adds nothing to the rate at which the system is carrying information.
All the information it contains is conveyed in an arbitrarily short interval of time, and thereafter nothing new is said. A message homogeneous in time, or, as the statisticians call it, a time series which is in statistical equilibrium, is thus a single function or a set of functions of the time, which forms one of an ensemble of such sets with a well-defined probability distribution, not altered by the change of t to t + τ throughout. That is, the transformation group consisting of the operators Tλ which
It follows from this that if Φ[f(t)] is a "functional" of f(t)--that is, a number depending upon the whole history of f(t)--and if the average of f(t) over the whole ensemble is finite, we are in a position to use the Birkhoff ergodic theorem quoted in the previous chapter, and to come to the conclusion that, except for a set of values of f(t) of zero probability, the time-average of Φ[f(t)], or in symbols, lim A→∞
exists. There is even more here than this. We have stated in the previous chapter another theorem of ergodic character, due to von Neumann, which states that, except for a set of elements of zero probability, any element belonging to a system which goes into itself under a group of measure-preserving transformations such as Eq. 3.15 belongs to a sub-set (which may be the whole set) which goes into itself under the same transformation, which has a measure defined over itself and also invariant under the transformation, and which has the further property that any portion of this sub-set with measure preserved under the group of transformations either has the maximum measure of the sub-set, or measure 0.
If we discard all elements except those of such a sub-set, and use its appropriate measure, we shall find that the time average (Eq. 3.16) is in almost all cases the average of Φ[f(t)] over all the space of functions f(t); the so-called phase average. Thus in the case of such an ensemble of functions f(t), except in a set of
cases of zero probability, we can deduce the average of any statistical parameter of the ensemble--indeed we can simultaneously deduce any countable set of such parameters of the ensemble--from the record of any one of the component time series, by using a time average instead of a phase average. Moreover, we need to know only the past of almost any one time series of the class. In other words, given the entire history up to the present of a time series known to belong to an ensemble in statistical equilibrium, we can compute with probable error zero the entire set of statistical parameters of an ensemble in statistical equilibrium to which that time series belongs.
Up to here, we have formulated this for single time series; it is equally true, however, for multiple time series in which we have several quantities varying simultaneously, rather than a single varying quantity. We are now in a position to discuss various problems belonging to time series. We shall confine our attention to those cases where the entire past of a time series can be given in terms of a countable set of quantities.
到目前为止,我们仅针对单时间序列推导了这些结果;它们同样适用于多变量时间序列。
For example, for quite a wide class of functions f(t) (−∞ < t < ∞), we have fully determined f when we know the set of quantities 0
例如,对于相当广泛的一类函数f(t)(−∞ < t < ∞),我们有完全确定的傅里叶变换,因此f(t)可以被表示为不同频率的正弦波分量的叠加。
an = ∫ e t t n f (t ) dt −∞
(n = 0, 1, 2,)
(3.17)
Now let A be some function of the values of t in the future, that is, for arguments greater than 0. Then we can determine the simultaneous distribution of (a0, a1, ..., an, A) from the past of almost any single time series if the set of f's is taken in its narrowest possible sense. In particular, if a0, ..., an are all given, we may determine the distribution of A.
Here we appeal to the known theorem of Nikodym on conditional probabilities. The same theorem will assure us that this distribution, under very general circumstances, will tend to a limit as n → ∞ and this limit will give
这里我们应用了著名的尼科迪姆条件概率定理。同一理论也将使我们能够确定预测的精度。
Time Series, Information, and Communication
us all the knowledge there is concerning the distribution of any future quantity. We may similarly determine the simultaneous distribution of values of any set of future quantities, or any set of quantities depending both on the past and the future, when the past is known. If then we have given any adequate interpretation to the "best value" of any of these statistical parameters or sets of statistical parameters--in the sense, perhaps, of a mean or a median or a mode--we can compute it from the known distribution, and obtain a prediction to meet any desired criterion of goodness of prediction.
We can compute the merit of the prediction, using any desired statistical basis of this merit--mean square error or maximum error or mean absolute error, and so on. We can compute the amount of information concerning any statistical parameter or set of statistical parameters, which fixing of the past will give us. We can even compute the whole amount of information which a knowledge of the past will give us of the whole future beyond a certain point; although when this point is the present, we shall in general know the latter from the past, and our knowledge of the present will contain an infinite amount of information.
我们可以使用任何所需的统计准则——均方误差、平均绝对误差或其他——来计算预测的优劣。
Another interesting situation is that of a multiple time series, in which we know precisely only the pasts of some of the components. The distribution of any quantity involving more than these pasts can be studied by means very similar to those already suggested. In particular, we may wish to know the distribution of a value of another component, or a set of values of other components, at some point of time, past, present, or future.
The general problem of the wave filter belongs to this class. We have a message, together with a noise, combined in some way into a corrupted message, of which we know the past. We also know the statistical joint distribution of the message and the noise as time
series. We ask for the distribution of the values of the message at some given time, past, present, and future. We then ask for an operator on the past of the corrupted message which will best give this true message, in some given statistical sense.
我们要求给定某个未来时刻的消息值的分布。这不仅给出了点预测,还给出了预测的不确定性度量。
We may ask for a statistical estimate of some measure of the error of our knowledge of the message. Finally, we may ask for the amount of information which we possess concerning the message. There is one ensemble of time series which is particularly simple and central.
我们可以要求对我们消息知识误差的某个度量进行统计估计。这就是预测区间估计的问题。
This is the ensemble associated with the Brownian motion. The Brownian motion is the motion of a particle in a gas, impelled by the random impacts of the other particles in a state of thermal agitation. The theory has been developed by many writers, among them Einstein, Smoluchowski, Perrin, and the author.2 Unless we go down in the time scale to intervals so small that the individual impacts of the particles on one another are discernible, the motion shows a curious kind of undifferentiability.
这就是与布朗运动相关的系综。布朗运动是粒子在流体中受到分子碰撞而产生的无规则运动。
The mean square motion in a given direction over a given time is proportional to the length of that time, and the motions over successive times are completely uncorrelated. This conforms closely to the physical observations. If we normalize the scale of the Brownian motion to fit the time scale, and consider only one coordinate x of the motion, and if we let x(t) equal 0 for t = 0, then the probability that if 0 t1 t 2 t n the particles lie between x1 and x1 + dx1 at time t1, ..., between xn and xn + dxn at time tn, is x 2 ( x − x1 )2 ( x − xn−1 )2 exp − 1 − 2 − − n 2 (t n − t n −1 ) 2t1 2 (t 2 − t1 ) dx dx
corresponding to the different possible Brownian motions depend on a parameter α lying between 0 and 1, in such a way that each path is a function x(t, α), where x depends on the time t and the parameter of distribution α, and where the probability that a path lies in a certain set S is the same as the measure of the set of values of α corresponding to paths in S. On this basis, almost all paths will be continuous and non-differentiable. A very interesting question is that of determining the average with respect to α of x(t1, α) ... x(tn, α).
对应于不同可能布朗运动的路径依赖于一个介于0和1之间的参数α,该参数表征了扩散的强度。
This will be 1
$$-\log_2 \frac{\text{measure of }(a,b)}{\text{measure of }(0,1)} \tag{3.03}$$
if every λk,j is even, = ∑ Ak ∏ ( number of ways of dividing λk , j terms into pairs) k
j
× (t j − t j −1 ) 2 1
= ∑ Ak ( numbers of ways of dividing n terms into pairs k
whose elements both belong in the same group of
λk , j terms into which λ is separated ) × (t j − t j −1 ) 2 1
= ∑ Aj ∑ ∏ ∫ dα [ x (t k , α ) − x (t k −1 , α )][ x (t q , α ) − x (t q −1 , α )] 0
j
Here the first ∑ sums over j; the second, over all the ways of dividing n terms in blocks, respectively, of λk,1, ..., λk,n numbers into pairs; and the ∏ is taken over those pairs of values k and q, where λk,1 of the elements to be selected from tk and tq are t1, λk,2 are t2, and so on. It immediately results that 1
这里第一个∑对j求和;第二个∑对将n项分成块的所有方式进行求和。
∫ dα x (t , α ) x (t , α ) x (t , α ) = ∑ ∏ ∫ dα x (t , α ) x (t , α ) 0
n
j
k
(3.22) where the ∑ is taken over all partitions of t1, ..., tn into distinct pairs, and the ∏ over all the pairs in each partition. In other words, when we know the averages of the products of x(tj, α) by pairs, we know the averages of all polynomials in these quantities, and thus their entire statistical distribution. Up to the present, we have considered Brownian motions x(t, α) where t is positive.
where α and β have independent uniform distributions over (0, 1), we shall obtain a distribution of ξ(t, α, β) where t runs over
当α和β在(0, 1)上具有独立的均匀分布时,我们将得到ξ的分布,其形式依赖于α和β的联合分布。
Time Series, Information, and Communication
the whole real infinite line. There is a well-known mathematical device to map a square on a line segment in such a way that area goes into length. All we need to do is to write our coordinates in the square in the decimal form:
γ = .α1β1α 2 β2 α n βn and we obtain a mapping of this sort which is one-one for almost all points both in the line segment and the square. Using this substitution, we define
γ = .α₁β₁α₂β₂…αₙβₙ,我们得到了这种对于几乎所有点都是一一对应的映射。
ξ (t , γ ) = ξ (t , α , β )
(3.25)
We now wish to define ∞
∫ K (t ) dξ (t , γ )
(3.26)
−∞
The obvious thing would be to define this as a Stieltjes3 integral, but ξ is a very irregular function of t and does not make such a definition possible. If, however, K runs sufficiently rapidly to 0 at ± ∞ and is a sufficiently smooth function, it is reasonable to put ∞
where the sum is over all partitions of τl, ..., τn into pairs, and the product is over the pairs in each partition. The expression ∞
其中求和遍及τ₁, …, τₙ的所有成对划分,乘积遍及每个划分中的各对。
∫ K (t + τ ) dξ (τ , γ ) = f (t , γ )
(3.34)
−∞
represents a very important ensemble of time series in the variable t, depending on a parameter of distribution γ. We have just shown what amounts to the statement that all the moments and hence all the statistical parameters of this distribution depend on the function ∞
表示变量t的时间序列的一个非常重要的系综,依赖于一个分布参数α。这个系综具有平稳性和遍历性。
Φ (τ ) = ∫ K ( s) K ( s + τ ) ds −∞
(3.35)
∞
= ∫ K ( s + t ) K ( s + t + τ ) ds −∞
which is the statisticians' autocorrelation function with lag τ. Thus the statistics of distribution of f(t, γ) are the same as the statistics of f(t + t1, γ); and it can be shown in fact that, if f (t + t1 , γ ) = f (t , Γ )
then the transformation of γ into Γ preserves measure. In other words, our time series f(t, γ) is in statistical equilibrium. Moreover, if we consider the average of m
together with a finite number of terms involving as factors powers of ∞
∫ K (σ + τ ) K (τ ) dτ
(3.39)
−∞
and if this approaches 0 when σ → ∞, Expression 3.38 will be the limit of Expression '3.37 under these circumstances. In other words, f(t, γ) and f(t + σ, γ) are asymptotically independent in their distributions as σ → ∞. By a more generally phrased but entirely similar argument, it may be shown that the simultaneous distribution of f(t1, γ), ..., f(tn, γ) and of f(σ + s1, γ), ..., f(σ + sm, γ) tends to the joint distribution of the first and the second set as σ → ∞.
如果当σ → ∞时这个趋近于0,那么表达式3.38将是表达式3.37在条件……下的极限
In other words, any bounded measurable functional or quantity depending on the entire distribution of the values of the function of t, f(t, γ), which we may write in the form [ f (t, γ )], must have the property that
If now [ f (t, γ )] is invariant under a translation of t, and only takes on the values 0 or 1, we shall have 1
如果现在 Φ[f(t, γ)] 在t的平移下不变,并且只取0或1两个值,那么我们就有
∫ [ f (t , γ )] dγ = ∫ { [ f (t , γ )] dγ }
(3.41)
so that the transformation group of f(t, γ) into f(t + σ, γ) is metrically transitive. It follows that if [ f (t, γ )] is any integrable functional of f as a function of t, then by the ergodic theorem 1
因此f(t, γ)变为f(t+σ, γ)的变换群是度量可递的。由此可知
T
∫ [ f (t , γ )] dγ = lim T ∫ [ f (t , γ )] dt 0
T →∞
1 0 [ f (t , γ )] dt T →∞ T ∫− T
(3.42)
= lim
for all values of γ except for a set of zero measure. That is, we can almost always read off any statistical parameter of such a time
series, and indeed any denumerable set of statistical parameters, from the past history of a single example. Actually, for such a time series, when we know lim T →∞
实际上,我们可以从单个时间序列的过去历史中读取任何可数集的统计参数。
1 0 f (t , γ ) f (t − τ , γ ) dt T ∫− T
(3.43)
we know Φ(t) in almost every case, and we have a complete statistical knowledge of the time series. There are certain quantities dependent on a time series of this sort which have quite interesting properties. In particular, it is interesting to know the average of ∞
我们在几乎所有情况下都知道Φ(t),从而对该时间序列有了完整的统计知识。
exp i ∫ K (t ) dξ (t , γ ) −∞
(3.44)
Formally, this may be written 1
∞
in
∞
∫ dγ ∑ n ! ∫ K (t ) dξ (t , γ ) 0
n=0
n
−∞
{ 2 m! {
} }
m ( −1)m ∞ [ K (t )]2 dt (2m − 1) (2m − 3) 5 ⋅ 3 ⋅ 1 ∫ −∞ m (2m ) ! m ( −1)m ∞ 2 =∑ m ∫−∞ [ K (t )] dt
It is a very interesting problem to try to build up a time series as general as possible from the simple Brownian motion series. In such constructions, the example of the Fourier developments suggests that such expansions as Expression 3.44 are convenient building blocks for this purpose. In particular, let us investigate time series of the special form
尝试从最简单的可能构建单元出发,尽可能普遍地构建时间序列,这是一个非常有趣的问题。
Chapter III
b
∞
a
−∞
∫ dλ exp i ∫ K (t + τ , λ ) dξ (τ , γ )
(3.46)
Let us suppose that we know ξ(τ, γ) as well as Expression 3.46. Then, as in Eq. 3.45, if t1 > t2, 1
假设我们知道ξ(τ, γ)以及表达式3.46。那么,如方程3.45所示,如果 t₁ > t₂,
∫ dγ exp {is [ξ (t , γ ) − ξ (t , γ )]} 1
∞
b
× ∫ d λ exp i ∫ K (t + τ , λ ) dξ (t , γ ) a −∞ b ∞ t1 s2 1 2 = ∫ d λ exp − ∫ [ K (t + τ , λ )] dt − (t 2 − t1 ) − s ∫ K (t , λ ) dt a −∞ t 2 2 2 (3.47) If we now multiply by exp[s2(t2 − t1)/2], let s(t2 − t1) = iσ, and let t2 → t1, we obtain
It is easy to see that if a is finite, it does not matter what value we give it; for our operator is not changed if we add a constant to all values of λ. We can hence make it 0. We have thus determined λ as a function of G, and thus G as a function of λ.
容易看出,如果a是有限的,我们给它什么值并不重要;因为我们的算子
Thus, by Eq. 3.55, we have determined K(t, λ). To finish the determination of Expression 3.46, we need only know b. This can be determined, however, by a comparison of
因此,通过方程3.55,我们确定了K(t, λ)。要完成表达式3.46的确定,我们需要
{
b
∞
∫ dλ exp − 2 ∫ [ K (t , λ )] dt a
−∞
}
(3.63)
with 1
b
∞
a
−∞
∫ dγ ∫ dλ exp i ∫ K (t , λ ) dξ (t , γ )
(3.64)
Thus, under certain circumstances which remain to be definitely formulated, if a time series may be written in the form of Expression 3.46 and we know ξ(t, γ) as well, we can determine the function K(t, λ) in Expression 3.46 and the numbers a and b, except for an undetermined constant added to a, λ, and b. There is no extra difficulty if b = +∞, and it is not hard to extend the reasoning to the case where a = −∞. Of course, a good deal of work remains to be done to discuss the problem of the inversion of the functions inverted when the results are not single-valued, and the general conditions of validity of the expansions concerned.
Still, we have at least taken a first step toward the solution of the problem of reducing a large class of time series to
尽管如此,我们至少已经为解决将一大类时间序列约化为规范形式的问题迈出了第一步。
Time Series, Information, and Communication
a canonical form, and this is most important for the concrete formal application of the theories of prediction and of the measurement of information, as we have sketched them earlier in this chapter. There is still one obvious limitation which we should remove from this approach to the theory of time series: the necessity which we are under of knowing ξ(t, γ) as well as the time series which we are expanding in the form of Expression 3.46. The question is: under what circumstances can we represent a time series of known statistical parameters as determined by a Brownian motion; or at least as the limit in some sense or other of time series determined by Brownian motions?
而这对于预测和滤波理论的具体形式应用是最重要的。
We shall confine ourselves to time series with the property of metrical transitivity, and with the even stronger property that if we take intervals of fixed length but remote in time, the distributions of any functionals of the segments of the time series in these intervals approach independence as the intervals recede from eachother.4 The theory to be developed here has already been sketched by the author. If K(t) is a sufficiently continuous function, it is possible to show that the zeros of ∞
我们将限于具有度量可递性以及以下性质的时间序列:
∫ K (t + τ ) dξ (τ , γ )
(3.65)
−∞
almost always have a definite density, by a theorem of M. Kac, and that this density can be made as great as we wish by a proper choice of K. Let KD be so selected that this density is D.
根据卡克定理,几乎总具有确定的密度,并且这个密度可以做到任意
Then the ∞
那么无穷∞
sequence of zeros of ∫ K D (t + τ ) dξ (τ , γ ) from −∞ to ∞ will be −∞
called Zn(D, γ), −∞ < n < ∞. Of course, in the numeration of these zeros, n is determined except for an additive constant integer.
称为Zₙ(D, γ),−∞ < n < ∞。当然,在这些零的编号中,n的确定除了一个附加常数外是明确的。
Chapter III
Now, let T(t, μ) be any time series in the continuous variable t, while μ is a parameter of distribution of the time series, varying uniformly over (0, 1). Then let TD (t , µ γ ) = T [t − Zn ( D, γ ) , µ ]
现在,设T(t, μ)是连续变量t的任意时间序列,同时μ是一个分布参数,在(0,1)上均匀分布。
(3.66)
where the Zn taken is the one just preceding t. It will be seen that for any finite set of values t1, t2, ..., tv of x the simultaneous distribution of TD(tκ, μ, γ) (κ = 1, 2, ..., v) will approach the simultaneous distribution of T(tκ, μ) for the same tκ's as D → ∞, for almost every value of μ. However, TD(t, μ, γ) is completely determined by t, μ, D, and ξ(τ, γ).
其中所取的Zₙ是刚好位于t之前的那一个。可以看出,对于任意有限组值t₁
It is therefore not inappropriate to try to express TD(t, μ, γ), for a given D and a given μ, either directly in the form of Expression 3.46 or in some way or another as a time series which has a distribution which is a limit (in the loose sense just given) of distributions of this form. It must be admitted that this is a program to be carried through in the future, rather than one which we can consider as already accomplished. Nevertheless, it is the program which, in the opinion of the author, offers the best hope for a rational, consistent treatment of the many problems associated with non-linear prediction, non-linear filtering, the evaluation of the transmission of information in non-linear situations, and the theory of the dense gas and turbulence.
因此,尝试将T_D(t, μ, γ)表示为某种规范形式是合适的。
Among these problems are perhaps the most pressing facing communication engineering. Let us now come to the prediction problem for time series of the form of Eq. 3.34. We see that the only independent statistical parameter of the time series is Φ(t), as given by Eq. 3.35; which means that the only significant quantity connected with K(t) is ∞
在这些问题中,或许包括通信工程面临的最紧迫的问题。现在让我们来
∫ K ( s) K ( s + t ) ds −∞
(3.67)
Time Series, Information, and Communication
Here of course K is real. Let us put ∞
K ( s) = ∫ k (ω ) e iω s dω −∞
(3.68)
employing a Fourier transformation. To know K(s) is to know k(ω), and vice versa. Then ∞ 1 ∞ K ( s) K ( s + τ ) ds = ∫ k (ω ) k ( −ω ) e iωτ dω −∞ 2π ∫−∞
Thus a knowledge of Φ(τ) is tantamount to a knowledge of k(ω)k(−ω). Since, however, K(s) is real, ∞
K ( s) = ∫ k (ω )e − iω s dω −∞
(3.70)
whence k(ω) = k(−ω). Thus |k(ω)|2 is a known function, which means that the real part of log |k(ω)| is a known function. If we write F (ω ) = {log [ k (ω )]}
then the determination of K(s) is equivalent to the determination of the imaginary part of log k(ω). This problem is not determinate unless we put some further restriction on k(ω). The type of restriction which we shall put is that log k(s) shall be analytic and of a sufficiently small rate of growth for ω in the upper half-plane.
那么K(s)的确定等价于log k(ω)的虚部的确定。
In order to make this restriction, k(ω) and [k(ω)−1 will be assumed to be of algebraic growth on the real axis. Then [F(ω)]2 will be even and at most logarithmically infinite, and the Cauchy principal value of G (ω ) =
为了做出这个限制,我们假设k(ω)和[k(ω)]⁻¹在实轴上具有代数增长阶,并且
1 ∞ F (u ) du π ∫−∞ u − ω
(3.72)
Chapter III
will exist. The transformation indicated by Eq. 3.72, known as the Hilbert transformation, changes cos λω into sin λω and sin λω into −cos λω. Thus F(ω) + iG(ω) is a function of the form ∞
存在。方程3.72所示的变换,称为希尔伯特变换,将余弦变换转化为正弦变换,反之亦然。
∫ e λω d [ M (λ )] i
(3.73)
and satisfies the required conditions for log |k(ω)| in the lower half-plane. If we now put k (ω ) = exp [ F (ω ) + iG (ω )]
it can be shown that k(ω) is a function which, under very general conditions, is such that K(s), as defined in Eq. 3.68, vanishes for all negative arguments. Thus ∞
可以证明,在非常一般的条件下,k(ω)是一个使得K(s)(定义如式)的函数
f (t , γ ) = ∫ K (t + τ ) dξ (τ , γ ) −t
(3.75)
On the other hand, it can be shown that we may write 1/k(ω) in the form ∞
lim ∫ e iλω dN n ( λ ) n→∞
(3.76)
where the Nn’s are properly determined; and that this can be done in such a way that τ
We shall find this result useful in getting the operator of prediction into a form concerning frequency rather than time. Thus the past and present of ξ(t, γ), or properly of the "differential" dξ(t, ξ), determine the past and present of f(t, γ), and vice versa. Now, if A > 0, f (t + A, γ ) = ∫ =∫
我们会发现这个结果在将预测算子转化为关于频率的形式时非常有用。
∞ −t − A −t −t − A
∞ −t−A −t −t−A
K (t + A + τ ) dξ (τ , γ ) K (t + A + τ ) dξ (τ , γ )
(3.83)
∞
+ ∫ K (t + A + τ ) dξ (τ , γ ) −t
Here the first term of the last expression depends on a range of dξ(τ, λ) of which a knowledge of f(σ, γ) for σ t tells us nothing, and is entirely independent of the second term. Its mean square value is
and this tells us all there is to know about it statistically. It may be shown to have a Gaussian distribution with this mean square value. It is the error of the best possible prediction of f(t + A, γ).
而这告诉了我们关于它的统计特性的一切。可以证明它具有高斯分布。
The best possible prediction itself is the last term of Eq. 3.83, ∞
可能的最佳预测本身是方程3.83的最后一项,∞
∫ k (t + A + τ ) dξ (τ , γ ) −t
∞
∞
−t
−τ
= lim ∫ k (t + A + τ ) dτ ∫ Q n (τ + σ ) f (σ , γ ) dσ n→∞
(3.85)
If we now put kA (ω ) =
1 ∞ K (t + A) e − iωt dt 2π ∫0
(3.86)
and if we apply the operator of Eq. 3.85 to etωt, obtaining ∞
∞
−t
−τ
lim ∫ K (t + A + τ ) dτ ∫ Q n (τ + σ ) e iωσ dσ = A (ω ) e iγ wt n→∞
(3.87)
we shall find out (somewhat as in Eq. 3.81) that A (ω ) = lim (2π ) 2 qn ( −ω ) kA (ω ) 3
n→∞
= kA (ω ) k (ω ) ∞ ∞ 1 = e − iω (t − A)dt ∫ k (u ) e iut du ∫ −∞ A 2π k (ω )
This is then the frequency form of the best prediction operator. The problem of filtering in the case of time series such as Eq. 3.34 is very closely allied to the prediction problem. Let our message plus noise be of the form ∞
where for symmetry we write Φ21 (ω ) = Φ12 (ω ). We can now determine k(ω) from Eq. 3.908, as we have defined k(ω) before on the basis of Eq. 3.74. Here we put Φ(t) for Φ11 (t ) + Φ22 (t ) + 2 [Φ12 (t )].
and thus the best determination of m(t), with the least mean square error, is
Time Series, Information, and Communication
Φ11 (ω ) + Φ21 (ω ) iω (t + a) e dω k (ω )
(Φ₁₁(ω) + Φ₂₁(ω))/(k(ω)) · e^{iω(t+a)} dω
(3.912)
∞
∞
−∞
∫ dξ (t − τ , γ ) ∫
Combining this with Eq. 3.89, and using an argument similar to the one by which we obtained Eq. 3.88, we see that the operator on m(t) + n(t) by which we obtain the "best" representation of m(t + a), if we write it on the frequency scale, is ∞ ∞ Φ11 (u ) + Φ 21 (u ) 1 e iut du e − iω (t − a)dt ∫ ∫ −∞ a 2π k (ω ) k (u )
将此与方程3.89结合,并使用类似于我们得到方程3.888的论证,我们得到
(3.913)
This operator constitutes a characteristic operator of what electrical engineers know as a wave filter. The quantity a is the lag of the filter. It may be either positive or negative; when it is negative, −a is known as the lead.
这个算子构成了电气工程师所知的波滤器的特征算子。
The apparatus corresponding to Expression 3.913 may always be constructed with as much accuracy as we like. The details of its construction are more for the specialist in electrical engineering than for the reader of this book. They may be found elsewhere.5 The mean square filtering error (Expression 3.902) may be represented as the sum of the mean square filtering error for infinite lag: ∞
It will be seen that the mean square error of filtering is a monotonely decreasing function of lag. Another question which is interesting in the case of messages and noises derived from the Brownian motion is the matter of rate of transmission of information. Let us consider for simplicity the case where the message and the noise are incoherent, that is, when
可以看出,滤波的均方误差是延迟的单调递减函数。延迟越长,我们可以获得的滤波精度越高。
Φ12 (ω ) ≡ Φ21 (ω ) ≡ 0
(3.916)
In this case, let us consider ∞ m (t ) = ∫ M (τ ) dξ (t − τ , γ ) −∞ ∞ n (t ) = ∫ N (τ ) dξ (t − τ , δ ) −∞
where γ and δ are distributed independently. Let us suppose we know m(t) + n(t) over (−A, A); how much information do we have concerning m(t)? Note that we should heuristically expect that it would not be very different from the amount of information concerning A
where γ and δ have independent distributions. It can, however, be shown that the nth Fourier coefficient of Expression 3.918 has a Gaussian distribution independent of all the other Fourier coefficients, and that its mean square value is proportional to
其中γ和δ具有独立分布。然而,可以证明第n个傅里叶系数
Time Series, Information, and Communication
π nτ ∫− A M (τ ) exp i A dτ A
(3.920)
Thus, by Eq. 3.09, the total amount of information available concerning M is 2
A π nτ π nτ ∫− A M (τ ) exp i A dτ + ∫− A N (τ ) exp i A dτ A
This is precisely the result which the author and Shannon have already obtained for the rate of transmission of information in this case. As will be seen, it depends not only on the width of the frequency band available for transmitting the message but also on the noise level. As a matter of fact, it has a close relation to the audiograms used to measure the amount of hearing and loss of hearing in a given individual.
这正是作者和香农已经得到的在连续情况下信息传输率的精确结果。
Here the abscissa is frequency, the ordinate of lower boundary is the logarithm of the intensity of the threshold of audible intensity--what we may call the logarithm of the intensity of the internal noise of the receiving system--and the upper boundary, the logarithm of the intensity of the greatest message the system is suited to handle. The area between them, which is a quantity of the dimension of Expression 3.922, is then taken as a measure of the rate of transmission of information with which the ear is competent to cope.
这里横坐标是频率,下边界的纵坐标是噪声强度的对数,上边界的纵坐标是信号加噪声强度的对数。
Chapter III
The theory of messages depending linearly on the Brownian motion has many important variants. The key formulae are Eqs. 3.88 and 3.914 and Expression 3.922, together, of course, with the definitions necessary to interpret these. There are a number of variants of this theory.
依赖于布朗运动的线性消息理论有许多重要的变体。关键思想保持不变。
First: the theory gives us the best possible design of predictors and of wave filters in the case in which the messages and the noises represent the response of linear resonators to Brownian motions; but in much more general cases, they represent a possible design for predictors and filters. This will not be an absolute best possible design, but it will minimize the mean square error of prediction and filtering, in so far as this can be done with apparatus performing linear operations. However, there will generally be some non-linear apparatus which gives a performance still better than that of any linear apparatus.
首先:该理论给出了在线性消息和噪声情况下的最佳预测器和波滤器的可能设计。
Next, the time series here have been simple time series, in which a single numerical variable depends on the time. There are also multiple time series, in which a number of such variables depend simultaneously on the time; and it is these which are of greatest importance in economics, meteorology, and the like. The complete weather map of the United States, taken from day to day, constitutes such a time series.
其次,这里的时间序列是简单时间序列,其中单个数值依赖于时间。但情况并非总是如此。
In this case, we have to develop a number of functions simultaneously in terms of the frequency, and the quadratic quantities such as Eq. 3.35 and the |k(ω)|2 of the arguments following Eq. 3.70 are replaced by arrays of pairs of quantities--that is, matrices. The problem of determining k(ω) in terms of |k(ω)|2, in such a way as to satisfy certain auxiliary conditions in the complex plane, becomes much more difficult, especially as the multiplication of matrices is not a permutable operation. However, the problems involved in this multidimensional theory have been solved, at least in part, by Krein and the author.
在这种情况下,我们需要同时根据频率展开多个函数,并使用向量的谱密度矩阵。
Time Series, Information, and Communication
The multidimensional theory represents a complication of the one already given. There is another closely related theory which is a simplification of it. This is the theory of prediction, filtering, and amount of information in discrete time series.
多维理论是已给出的理论的一个复杂化。还有另一类重要的时间序列:离散时间序列。
Such a series is a sequence of functions fn(α) of a parameter a, where n runs over all integer values from −∞ to ∞. The quantity a is as before the parameter of distribution, and may be taken to run uniformly over (0, 1). The time series is said to be in statistical equilibrium when the change of n to n + ν (ν an integer) is equivalent to a measure-preserving transformation into itself of the interval (0, 1) over which α runs.
这样的序列是参数α的函数fₙ(α)的一个序列,其中n取所有整数值。
The theory of discrete time series is simpler in many respects than the theory of the continuous series. It is much easier, for instance, to make them depend on a sequence of independent choices. Each term (in the mixing case) will be representable as a combination of the previous terms with a quantity independent of all previous terms, distributed uniformly over (0, 1), and the sequence of these independent factors may be taken to replace the Brownian motion which is so important in the continuous case.
在许多方面,离散时间序列的理论比连续序列的理论更简单。
If fn(α) is a time series in statistical equilibrium, and it is metrically transitive, its autocorrelation coefficient will be 1
如果fₙ(α)是一个统计平衡的时间序列,并且是度量可递的,那么它的自相关函数是
φm = ∫ f m (α ) f 0 (α ) dα 0
(3.923)
and we shall have 1 N ∑0 f k +m (α ) f k (α ) N →∞ N + 1
Then under very general conditions, k(ω) will be the boundary value on the unit circle of a function without zeros or singularities inside the unit circle if ω is the angle. We shall have k (ω ) = Φ (ω ) 2
那么在非常一般的条件下,k(ω)将是在单位圆上解析的函数的边界值
(3.930)
If now we put for the best linear prediction of fn(α) with a lead of ν ∞
∑ f ν (α ) Wν
(3.931)
n−
we shall find that ∞
∞
π
∑ Wµ e µω = 2π k (ω ) µ∑ν e ω (µ ν ) ∫ π k (u) e µ du 0
i
=
i
−
−
−i u
(3.932)
This is the analogue of Eq. 3.88. Let us note that if we put
It will clearly be the result of the way we have formed k(ω) that in a very general set of cases we can put ∞ 1 = ∑ qµ e iµω k (ω ) 0
显然,根据我们构成k(ω)的方式,在非常一般的情况下我们将
(3.935)
Then Eq. 3.934 becomes ∞
ν −1
∞
∑ Wµ e µω = e νω 1 − ∑ kµ e µω ∑ qλ e λω i
−i
i
i
(3.936)
In particular, if ν = 1, ∞
∞
∑ Wµ e µω = e ω 1 − k ∑ qλ e λω i
−i
i
(3.937)
or Wµ = − qλ +1k0
(3.938)
Thus for a prediction one step ahead, the best value for fn+1(α) is ∞
− k0 ∑ qλ +1 f n − λ (α ) 0
(3.939)
Chapter III
and by a process of step-by-step prediction, we can solve the entire problem of linear prediction for discrete time series. As in the continuous case, this will be the best prediction possible by any method if ∞
通过逐步预测的过程,我们可以解决离散序列的整个线性预测问题。
f n (α ) = ∫ K ( n − τ ) dξ (τ , α ) −∞
(3.940)
The transfer of the filtering problem from the continuous to the discrete case follows much the same lines of argument. Formula 3.913 for the frequency characteristic of the best filter takes the form ∞ π [Φ11 (u ) + Φ 21 (u )] e iuν du 1 e − iω (ν − a) ∫ ∑ − π 2π k (ω ) ν = a k (u )
将滤波问题从连续情形转移到离散情形的过程,遵循大致相同的思路。
(3.941)
where all the terms receive the same definitions as in the continuous case, except that all integrals on ω or u are from −π to π instead of from −∞ to ∞ and all sums on v are discrete sums instead of integrals on t. The filters for discrete time series are usually not so much physically constructible devices to be used with an electric circuit as mathematical procedures to enable statisticians to obtain the best results with statistically impure data. Finally, the rate of transfer of information by a discrete time series of the form ∞
其中所有项的定义与连续情形相同,只是所有积分
∫ M (n − τ ) dξ (t , γ ) −∞
(3.942)
in the presence of a noise ∞
∫ N (n − τ ) dξ (t , δ ) −∞
(3.943)
when γ and δ are independent, will be the precise analogue of Expression 3.922, namely,
Time Series, Information, and Communication
∞
∞
M (τ ) e iuτ dτ + ∫ N (τ ) e iuτ dτ ∫ 1 π −∞ −∞ du log 2 2 ∞ 2π ∫− π iuτ ∫ N ( τ ) e dτ
represents the power distribution of the message in frequency, and ∞
∫ N ( τ ) e τ dτ iu
−∞
(3.946)
that of the noise. The statistical theories we have here developed involve a full knowledge of the pasts of the time series we observe. In every case, we have to be content with less, as our observation does not run indefinitely into the past.
即噪声的强度。我们在这里发展的统计理论涉及对过去全部历史的完整知识。
The development of our theory beyond this point, as a practical statistical theory, involves an extension of existing methods of sampling. The author and others have made a beginning in this direction. It involves all the complexities of the use either of Bayes' law, on the one hand, or of those terminological tricks in the theory of likelihood,6 on the other, which seem to avoid the necessity for the use of Bayes' law but which in reality transfer the responsibility for its use to the working statistician, or the person who ultimately employs his results.
作为一个实用统计理论,我们的理论在此基础上进一步发展,需要对统计推断进行广泛的讨论。
Meanwhile, the statistical theorist is quite honestly able to say that he has said nothing which is not perfectly rigorous and unimpeachable. Finally, this chapter should end with a discussion of modern quantum mechanics. These represent the highest point of the invasion of modern physics by the theory of time series.
与此同时,统计理论家完全可以诚实地说,他所说的一切都与实践是一致的。
Chapter III
the Newtonian physics, the sequence of physical phenomena is completely determined by its past and in particular by the determination of all positions and momenta at any one moment. In the complete Gibbsian theory, it is still true that with a perfect determination of the multiple time series of the whole universe the knowledge of all positions and momenta at any one moment would determine the entire future. It is only because these are ignored, non-observed coordinates and momenta that the time series with which we actually work take on the sort of mixing property with which we have become familiar in this chapter, in the case of time series derived from the Brownian motion.
在牛顿物理学中,物理现象的序列完全由其过去和现在决定。
The great contribution of Heisenberg to physics was the replacement of this still quasi-Newtonian world of Gibbs by one in which the time series can in no way be reduced to an assembly of determinate threads of development in time. In quantum mechanics, the whole past of an individual system does not determine the future of that system in any absolute way but merely the distribution of possible futures of the system. The quantities which the classical physics demands for a knowledge of the entire course of a system are not simultaneously observable, except in a loose and approximate way, which nevertheless is sufficiently precise for the needs of the classical physics over the range of precision where it has been shown experimentally to be applicable.
海森堡对物理学的伟大贡献在于,他用一种新的统计世界观取代了这种准牛顿式的世界观。
The conditions of the observation of a momentum and its corresponding position are incompatible. To observe the position of a system as precisely as possible, we must observe it with light or electron waves or similar means of high resolving power, or short wavelength. However, the light has a particle action depending on its frequency only, and to illuminate a body with high-frequency light means to subject it to a change in its momentum which increases with the frequency.
hand, it is low-frequency light that gives the minimum change in the momenta of the particles it illuminates, and this has not a sufficient resolving power to give a sharp indication of positions. Intermediate frequencies of light give a blurred account both of positions and of momenta. In general, there is no set of observations conceivable which can give us enough information about the past of a system to give us complete information as to its future.
另一方面,低频光对它所照明粒子的动量变化最小。但这又牺牲了位置的精度。
Nevertheless, as in the case of all ensembles of time series, the theory of the amount of information which we have here developed is applicable, and consequently the theory of entropy. Since, however, we now are dealing with time series with the mixing property, even when our data are as complete as they can be, we find that our system has no absolute potential barriers, and that in the course of time any state of the system can and will transform itself into any other state. However, the probability of this depends in the long run on the relative probability or measure of the two states.
尽管如此,如同所有时间序列系综的情况一样,信息量理论可以应用于量子力学系统。
This turns out to be especially high for states which can be transformed into themselves by a large number of transformations, for states which, in the language of the quantum theorist, have a high internal resonance, or a high quantum degeneracy. The benzene ring is an example of this, since the two states are equivalent. This suggests that in a system in which various building blocks may combine themselves intimately in various ways, as when a mixture of amino acids organizes itself into protein chains, a situation where many of these chains are alike and go through a stage of close association with one another may be more stable than one in which they are different.
这对于那些可以通过大量对称操作变换为自身的状态而言特别高。
It was suggested by Haldane, in a tentative manner, that this may be the way in which genes and viruses reproduce themselves; and although he has not asserted this suggestion of
霍尔丹曾以一种试探性的方式提出,这可能是基因和病毒
Chapter III
his with anything like finality, I see no cause not to retain it as a tentative hypothesis. As Haldane himself has pointed out, as no single particle in quantum theory has a perfectly sharp individuality, it is not possible in such a case to say, with more than fragmentary accuracy, which of the two examples of a gene that has reproduced itself in this manner is the master pattern and which is the copy.
以任何终结性的方式处理这个问题——我认为没有理由不将它作为一个试探性假设来保留。正如霍尔丹
This same phenomenon of resonance is known to be very frequently represented in living matter. Szent-Györgyi has suggested its importance in the construction of muscles. Substances with high resonance very generally have an abnormal capacity for storing both energy and information, and such a storage certainly occurs in muscle contraction.
已知这种相同的共振现象在生命物质中非常频繁地出现。森特-哲尔吉
Again, the same phenomena that are concerned in reproduction probably have something to do with the extraordinary specificity of the chemical substances found in a living organism, not only from species to species but even within the individuals of a species. Such considerations may be very important in immunology.
同样,与生殖有关的相同现象可能与信息存储有关——
IV Feedback and Oscillation
V. Computing Machines and the Nervous System
V. Computing Machines and the Nervous System
V. Computing Machines and the Nervous System
1 1 1 ν1 + 2 ν 2 + + ν n + 2 2 2n
(1/2)ν₁ + (1/2²)ν₂ + ... + (1/2ⁿ)νₙ + ...
(5.07)
ν = ν0 +
where every νn is either 1 or 0. There exist at present two great types of computing machines: those like the Bush differential analyzer,1 which are known as analogy machines, where the data are represented by measurements on some continuous scale, so that the accuracy of the machine is determined by the accuracy of construction of the scale; and those, like the ordinary desk adding and multiplying machine, which we call numerical machines, where the data are represented by a set of choices among a number of contingencies, and the accuracy is determined by the sharpness with which the contingencies are distinguished, the number of alternative contingencies presented at every choice, and the number of choices given. We see that for highly accurate work, at any rate, the numerical machines are preferable, and above all, those numerical machines constructed on the binary scale, in which the number of alternatives presented at each choice is two.
其中每个νₙ是1或0。目前存在两种主要类型的计算机:那些
Our use of machines on the decimal scale is conditioned merely by the historical accident that the scale of ten, based on our fingers and thumbs, was already in use when the Hindus made the great discovery of the importance of the zero and the advantage of a positional system of notation. It is worth retaining when a large part of the work done with the aid of the machine consists in transcribing onto the machine numbers in the conventional decimal form, and in taking off the machine numbers which must be written in the same conventional form. This is, in fact, the use of the ordinary desk computing machine, as employed in banks, in business offices, and in many statistical laboratories.
我们在十进制刻度上使用机器仅仅是出于历史偶然,即人类
It is not the way that the larger and
这不是较大和
Chapter V
more automatic machines are best to be employed; in general, any computing machine is used because machine methods are faster than hand methods. In any combined use of means of computation, as in any combination of chemical reactions, it is the slowest which gives the order of magnitude of the time constants of the entire system. It is thus advantageous, as far as possible, to remove the human element from any elaborate chain of computation and to introduce it only where it is absolutely unavoidable, at the very beginning and the very end.
更自动化的机器的最佳使用方式;一般来说,使用任何计算机是因为
Under these conditions, it pays to have an instrument for the change of the scale of notation, to be used initially and finally in the chain of computations, and to perform all intermediate processes on the binary scale. The ideal computing machine must then have all its data inserted at the beginning, and must be as free as possible from human interference to the very end. This means that not only must the numerical data be inserted at the beginning, but also all the rules for combining them, in the form of instructions covering every situation which may arise in the course of the computation.
在这些条件下,值得拥有一个用于改变数制刻度的仪器,
Thus the computing machine must be a logical machine as well as an arithmetic machine and must combine contingencies in accordance with a systematic algorithm. While there are many algorithms which might be used for combining contingencies, the simplest of these is known as the algebra of logic par excellence, or the Boolean algebra. This algorithm, like the binary arithmetic, is based on the dichotomy, the choice between yes and no, the choice between being in a class and outside.
因此计算机必须是逻辑机器也是算术机器,并且必须结合
The reasons for its superiority to other systems are of the same nature as the reasons for the superiority of the binary arithmetic over other arithmetics.
它优于其他系统的原因与二进制优于其他系统的原因性质相同。
Computing Machines and the Nervous System
Thus all the data, numerical or logical, put into the machine are in the form of a set of choices between two alternatives, and all the operations on the data take the form of making a set of new choices depend on a set of old choices. When I add two one-digit numbers, A and B, I obtain a two-digit number commencing with 1, if A and B are both 1, and otherwise with 0. The second digit is 1 if A ≠ B, and is otherwise 0.
因此,所有输入机器的数据,无论是数字的还是逻辑的,都采取在几个可能性之间进行选择的形式。
The addition of numbers of more than one digit follows similar but more complicated rules. Multiplication in the binary system, as in the decimal, may be reduced to the multiplication table and the addition of numbers, and the rules for multiplication for binary numbers take on the peculiarly simple form given by the table × 0 1 0 0 0 1 0 1
多位数字的加法遵循类似但更复杂的规则。乘法
(5.08)
Thus multiplication is simply a method to determine a set of new digits when old digits are given. On the logical side, if O is a negative and I a positive decision, every operator can be derived from three: negation, which transforms I into O and O into I; logical addition, with the table ⊕ O I O O I I I I
因此乘法只是一种在给定旧数字时确定一组新数字的方法。
(5.09)
and logical multiplication, with the same table as the numerical multiplication of the (1, 0) system, namely, O I O O O I O I
以及逻辑乘法,使用与(1, 0)系统的数值乘法相同的真值表。
(5.10)
Chapter V
That is, every contingency which may arise in the operation of the machine simply demands a new set of choices of contingencies I and O, depending according to a fixed set of rules on the decisions already made. In other words, the structure of the machine is that of a bank of relays, capable each of two conditions, say "on" and "off"; while at each stage the relays assume each a position dictated by the positions of some or all the relays of the bank at a previous stage of operation. These stages of operation may be definitely "clocked" from some central clock or clocks, or the action of each relay may be held up until all the relays which should have acted earlier in the process have gone through all the steps called for.
也就是说,机器运行中可能出现的每一种情况只需要一组新的
The relays used in a computing machine may be of very varied character. They may be purely mechanical, or they may be electro-mechanical, as in the case of a solenoidal relay, in which the armature will remain in one of two possible positions of equilibrium until an appropriate impulse pulls it to the other side. They may be purely electrical systems with two alternative positions of equilibrium, either in the form of gas-filled tubes, or, what is much more rapid, in the form of high-vacuum tubes.
计算机中使用的继电器可以是各种类型的。它们可以是纯机械的。
The two possible states of a relay system may both be stable in the absence of outside interference, or only one may be stable, while the other is transitory. Always in the second case and generally in the first case, it will be desirable to have special apparatus to retain an impulse which is to act at some future time, and to avoid the clogging up of the system which will ensue if one of the relays does nothing but repeat itself indefinitely. However, we shall have more to say concerning this question of memory later.
继电器系统的两种可能状态在没有外部干扰时都可以是稳定的。
It is a noteworthy fact that the human and animal nervous systems, which are known to be capable of the work of a
值得注意的是,已知能够进行
Computing Machines and the Nervous System
computation system, contain elements which are ideally suited to act as relays. These elements are the so-called neurons or nerve cells. While they show rather complicated properties under the influence of electrical currents, in their ordinary physiological action they conform very nearly to the "all-or-none" principle; that is, they are either at rest, or when they "fire" they go through a series of changes almost independent of the nature and intensity of the stimulus.
计算的人体和动物神经系统包含理想地适合用作继电器的元素。这些元素就是
There is first an active phase, transmitted from one end to the other of the neuron with a definite velocity, to which there succeeds a refractory period during which the neuron is either incapable of being stimulated, or at any rate is not capable of being stimulated by any normal, physiological process. At the end of this effective refractory period, the nerve remains inactive, but may be stimulated again into activity. Thus the nerve may be taken to be a relay with essentially two states of activity: firing and repose.
首先有一个活动阶段,以确定的速度从神经元的一端传递到另一端。
Leaving aside those neurons which accept their messages from free endings or sensory end organs, each neuron has its message fed into it by other neurons at points of contact known as synapses. For a given outgoing neuron, these vary in number from a very few to many hundred. It is the state of the incoming impulses at the various synapses, combined with the antecedent state of the outgoing neuron itself, which determines whether it will fire or not.
除了那些从游离末梢或感觉末梢器官接收信息的神经元外,每个
If it is neither firing nor refractory, and the number of incoming synapses which "fire" within a certain very short fusion interval of time exceeds a certain threshold, then the neuron will fire after a known, fairly constant synaptic delay. This is perhaps an oversimplification of the picture: the "threshold" may not depend simply on the number of synapses but on their "weight" and their geometrical relations to one another with respect to the neuron into which they feed; and
如果它既不在发放中也不在不应期中,并且在某个短时间内接收到的"发放"的传入突触数量超过
Chapter V
there is very convincing evidence that there exist synapses of a different nature, the so-called "inhibitory synapses," which either completely prevent the firing of the outgoing neuron or at any rate raise its threshold with respect to stimulation at the ordinary synapses. What is pretty clear, however, is that some definite combinations of impulses on the incoming neurons having synaptic connections with a given neuron will cause it to fire, while others will not cause it to fire. This is not to say that there may not be other, non-neuronic influences, perhaps of a humoral nature, which produce slow, secular changes tending to vary that pattern of incoming impulses which is adequate for firing.
有非常令人信服的证据表明存在性质不同的突触,即所谓的"抑制性"
A very important function of the nervous system, and, as we have said, a function equally in demand for computing machines, is that of memory, the ability to preserve the results of past operations for use in the future. It will be seen that the uses of the memory are highly various, and it is improbable that any single mechanism can satisfy the demands of all of them. There is first the memory which is necessary for the carrying out of a current process, such as a multiplication, in which the intermediate results are of no value once the process is completed, and in which the operating apparatus should then be released for further use.
神经系统的一个非常重要的功能,也是——正如我们所说——同样需要的功能
Such a memory should record quickly, be read quickly, and be erased quickly. On the other hand, there is the memory which is intended to be part of the files, the permanent record, of the machine or the brain, and to contribute to the basis of all its future behavior, at least during a single run of the machine. Let it be remarked parenthetically that an important difference between the way in which we use the brain and the machine is that the machine is intended for many successive runs, either with no reference to each other, or with a
这样的记忆应该能快速记录、快速读取和快速擦除。另一方面,
Computing Machines and the Nervous System
minimal, limited reference, and that it can be cleared between such runs; while the brain, in the course of nature, never even approximately clears out its past records. Thus the brain, under normal circumstances, is not the complete analogue of the computing machine but rather the analogue of a single run on such a machine. We shall see later that this remark has a deep significance in psychopathology and in psychiatry.
记忆最小,引用有限,并且可以在两次运行之间清除;而人脑在
To return to the problem of memory, a very satisfactory method for constructing a short-time memory is to keep a sequence of impulses traveling around a closed circuit until this circuit is cleared by intervention from outside. There is much reason to believe that this happens in our brains during the retention of impulses, which occurs over what is known as the specious present. This method has been imitated in several devices which have been used in computing machines, or at least suggested for such a use.
回到记忆问题,构建短期记忆的一个非常令人满意的方法是
There are two conditions which are desirable in such a retentive apparatus: the impulse should be transmitted in a medium in which it is not too difficult to achieve a considerable time lag; and before the errors inherent in the instrument have blurred it too much, the impulse should be reconstructed in a form as sharp as possible. The first condition tends to rule out delays produced by the transmission of light, or even, in many cases, by electric circuits, while it favors the use of one form or another of elastic vibrations; and such vibrations have actually been employed for this purpose in computing machines. If electric circuits are used for delay purposes, the delay produced at every stage is relatively short; or, as in all pieces of linear apparatus, the deformation of the message is cumulative and very soon becomes intolerable.
这种保持装置有两个理想条件:脉冲应该被传递
To avoid this, a second consideration comes into play; we must insert somewhere in the cycle a relay which does not serve to repeat the form
为了避免这一点,我们需要考虑第二个因素;我们必须在中某处插入一个继电器
Chapter V
of the incoming message but rather to trigger off a new message of prescribed form. This is done very easily in the nervous system, where indeed all transmission is more or less of a trigger phenomenon. In the electrical industry, pieces of apparatus for this purpose have long been known and have been used in connection with telegraph circuits.
不是触发新的传入消息,而是触发一个规定形式的新消息。这是通过
They are known as telegraph-type repeaters. The great difficulty of using them for memories of long duration is that they have to function without a flaw over an enormous number of consecutive cycles of operation. Their success is all the more remarkable: in a piece of apparatus designed by Mr.
它们被称为电报式转发器。将它们用于长时间记忆的主要困难是
Williams of the University of Manchester, a device of this sort with a unit delay of the order of a hundredth of a second has continued in successful operation for several hours. What makes this more remarkable is that this apparatus was not used merely to preserve a single decision, a single "yes" or "no," but a matter of thousands of decisions. Like other forms of apparatus intended to retain a large number of decisions, this works on the scanning principle.
曼彻斯特大学的威廉姆斯,这种装置的单元延迟约为
One of the simplest modes of storing information for a relatively short time is as the charge on a condenser; and when this is supplemented by a telegraph-type repeater, it becomes an adequate method of storage. To use to the best advantage the circuit facilities attached to such a storage system, it is desirable to be able to switch successively and very rapidly from one condenser to another. The ordinary means of doing this involve mechanical inertia, and this is never consistent with very high speeds.
最简单的一种相对短时间的信息存储方式是电容器上的电荷。
A much better way is the use of a large number of condensers, in which one plate is either a small piece of metal sputtered into a dielectric, or the imperfectly insulating surface of the dielectric itself, while one of the connectors to these condensers is a pencil of cathode rays moved by the condensers and magnets of a
更好的方法是使用大量电容器,其中一块极板要么是一个小点
Computing Machines and the Nervous System
sweep circuit over a course like that of a plough in a ploughed field. There are various elaborations of this method, which indeed was employed in a somewhat different way by the Radio Corporation of America before it was used by Mr. Williams.
扫描电路沿着类似于犁过的田地的路径行进。有多种精化方法。
These last-named methods for storing information can hold a message for quite an appreciable time, if not for a period comparable with a human lifetime. For more permanent records, there is a wide variety of alternatives among which we can choose. Leaving out such bulky, slow, and unerasable methods as the use of punched cards and punched tape, we have magnetic tape, together with its modern refinements, which have largely eliminated the tendency of messages on this material to spread; phosphorescent substances; and above all, photography.
这些最后提到的信息存储方法可以在相当长的时间内保持信息,
Photography is indeed ideal for the permanence and detail of its records, ideal again from the point of view of the shortness of exposure needed to record an observation. It suffers from two grave disadvantages: the time needed for development, which has been reduced to a few seconds, but is still not small enough to make photography available for a short-time memory; and (at present [1947]) the fact that a photographic record is not subject to rapid erasure and the rapid implanting of a new record. The Eastman people have been working on just these problems, which do not seem to be necessarily insoluble, and it is possible that by this time they have found the answer.
摄影对于其记录的持久性和细节来说确实是理想的,从角度来看也是理想的
Very many of the methods of storage of information already considered have an important physical element in common. They seem to depend on systems with a high degree of quantum degeneracy, or, in other words, with a large number of modes of vibration of the same frequency. This is certainly true in the case of ferromagnetism, and is also true in the case of materials with an exceptionally high dielectric constant, which are thus
已经讨论的许多信息存储方法具有一个重要的物理元素
Chapter V
especially valuable for use in condensers for the storage of information. Phosphorescence as well is a phenomenon associated with a high quantum degeneracy, and the same sort of effect makes its appearance in the photographic process, where many of the substances which act as developers seem to have a great deal of internal resonance. Quantum degeneracy appears to be associated with the ability to make small causes produce appreciable and stable effects.
对于用于信息存储的电容器尤其有价值。磷光同样也是
We have already seen in Chapter II that substances with high quantum degeneracy appear to be associated with many of the problems of metabolism and reproduction. It is probably not an accident that here, in a non-living environment, we find them associated with a third fundamental property of living matter: the ability to receive and organize impulses and to make them effective in the outer world. We have seen in the case of photography and similar processes that it is possible to store a message in the form of a permanent alteration of certain storage elements.
我们在第二章中已经看到,具有高量子简并度的物质似乎与新陈代谢和生殖的许多问题有关。
In reinserting this information into the system, it is necessary to cause these changes to affect the messages going through the system. One of the simplest ways to do this is to have, as the storage elements which are changed, parts which normally assist in the transmission of messages, and of such a nature that the change in their character due to storage affects the manner in which they will transport messages for the entire future. In the nervous system, the neurons and the synapses are elements of this sort, and it is quite plausible that information is stored over long periods by changes in the thresholds of neurons, or, what may be regarded as another way of saying the same thing, by changes in the permeability of each synapse to messages.
在将这个信息重新插入系统时,需要使这些改变影响通过系统的消息。
Many of us think, in the absence of a better explanation of the phenomenon, that the storage of information in the brain can actually occur in this
我们中的许多人认为,在没有更好的现象解释的情况下,大脑中的信息存储实际上可以以这种方式发生。
Computing Machines and the Nervous System
It is conceivable for such a storage to take place either by the opening of new paths or by the closure of old ones. Apparently it is adequately established that no neurons are formed in the brain after birth. It is possible, though not certain, that no new synapses are formed, and it is a plausible conjecture that the chief changes of thresholds in the memory process are increased.
If this is the case, our whole life is on the pattern of Balzac's Peau de Chagrin, and the very process of learning and remembering exhausts our powers of learning and remembering until life itself squanders our capital stock of power to live. It may well be that this phenomenon does occur. This is a possible explanation for a sort of senescence.
The real phenomenon of senescence, however, is much too complicated to be explained in this way alone. We have already spoken of the computing machine, and consequently the brain, as a logical machine. It is by no means trivial to consider the light cast on logic by such machines, both natural and artificial.
Here the chief work is that of Turing.2 We have said before that the machina ratiocinatrix is nothing but the calculus ratiocinator of Leibniz with an engine in it; and just as modern mathematical logic begins with this calculus, so it is inevitable that its present engineering development should cast a new light on logic. The science of today is operational; that is, it considers every statement as essentially concerned with possible experiments or observable processes. According to this, the study of logic must reduce to the study of the logical machine, whether nervous or mechanical, with all its non-removable limitations and imperfections.
It may be said by some readers that this reduces logic to psychology, and that the two sciences are observably and demonstrably different. This is true in the sense that many psychological states and sequences of thought do not conform to the canons
of logic. Psychology contains much that is foreign to logic, but--and this is the important fact--any logic which means anything to us can contain nothing which the human mind--and hence the human nervous system--is unable to encompass. All logic is limited by the limitations of the human mind when it is engaged in that activity known as logical thinking.
For example, we devote much of mathematics to discussions involving the infinite, but these discussions and their accompanying proofs are not infinite in fact. No admissible proof involves more than a finite number of stages. It is true, a proof by mathematical induction seems to involve an infinity of stages, but this is only apparent.
In fact, it involves just the following stages: 1. Pn is a proposition involving the number n. 2. Pn has been proved for n = 1. 3.
事实上,它只包括以下步骤:1. Pₙ是涉及数n的命题。2. Pₙ已对n=1得证。3.
If Pn is true, Pn+1 is true. 4. Therefore, Pn is true for every positive integer n. It is true that somewhere in our logical assumptions there must be one which validates this argument.
However, this mathematical induction is a far different thing from complete induction over an infinite set. The same thing is true of the more refined forms of mathematical induction, such as transfinite induction, which occur in certain mathematical disciplines. Thus some very interesting situations arise, in which we may be able--with enough time and enough computational aids--to prove every single case of a theorem Pn; but if there is no systematic way of subsuming these proofs under a single argument independent of n, such as we find in mathematical induction, it may be impossible to prove Pn for all n.
recognized in what is known as metamathematics, the discipline so brilliantly developed by Gödel and his school. A proof represents a logical process which has come to a definitive conclusion in a finite number of stages. However, a logical machine following definite rules need never come to a conclusion.
It may go on grinding through different stages without ever coming to a stop, either by describing a pattern of activity of continually increasing complexity, or by going into a repetitive process like the end of a chess game in which there is a continuing cycle of perpetual check. This occurs in the case of some of the paradoxes of Cantor and Russell. Let us consider the class of all classes which are not members of themselves.
Is this class a member of itself? If it is, it is certainly not a member of itself; and if it is not, it is equally certainly a member of itself. A machine to answer this question would give the successive temporary answers: "yes," "no," "yes," "no," and so on, and would never come to equilibrium.
Bertrand Russell's solution of his own paradoxes was to affix to every statement a quantity, the so-called type, which serves to distinguish between what seems to be formally the same statement, according to the character of the objects with which it concerns itself--whether these are "things," in the simplest sense, classes of "things," classes of classes of "things," etc. The method by which we resolve the paradoxes is also to attach a parameter to each statement, this parameter being the time at which it is asserted. In both cases, we introduce what we may call a parameter of uniformization, to resolve an ambiguity which is simply due to its neglect.
We thus see that the logic of the machine resembles human logic, and, following Turing, we may employ it to throw light on human logic. Has the machine a more eminently human
characteristic as well--the ability to learn? To see that it may well have even this property, let us consider two closely related notions: that of the association of ideas and that of the conditioned reflex. In the British empirical school of philosophy, from Locke to Hume, the content of the mind was considered to be made up of certain entities known to Locke as ideas, and to the later authors as ideas and impressions.
The simple ideas or impressions were supposed to exist in a purely passive mind, as free from influence on the ideas it contained as a clean blackboard is on the symbols which may be written on it. By some sort of inner activity, hardly worthy to be called a force, these ideas were supposed to unite themselves into bundles, according to the principles of similarity, contiguity, and cause and effect. Of these principles, perhaps the most significant was contiguity: ideas or impressions which had often occurred together in time or in space were supposed to have acquired the ability of evoking one another, so that the presence of any one of them would produce the entire bundle.
In all this there is a dynamics implied, but the idea of a dynamics had not yet filtered through from physics to the biological and psychological sciences. The typical biologist of the eighteenth century was Linnaeus, the collector and classifier, with a point of view quite opposed to that of the evolutionists, the physiologists, the geneticists, the experimental embryologists of the present day. Indeed, with so much of the world to explore, the state of mind of the biologists could hardly have been different.
Similarly, in psychology, the notion of mental content dominated that of mental process. This may well have been a survival of the scholastic emphasis on substances, in a world in which the noun was hypostasized and the verb carried
little or no weight. Nevertheless, the step from these static ideas to the more dynamic point of view of the present day, as exemplified in the work of Pavlov, is perfectly clear. Pavlov worked much more with animals than with men, and he reported visible actions rather than introspective states of mind.
He found in dogs that the presence of food causes the increased secretion of saliva and of gastric juice. If then a certain visual object is shown to dogs in the presence of food and only in the presence of food, the sight of this object in the absence of food will acquire the property of being by itself able to stimulate the flow of saliva or of gastric juice. The union by continguity which Locke had observed introspectively in the case of ideas now becomes a similar union of patterns of behavior.
There is one important difference, however, between the point of view of Pavlov and that of Locke, and it is precisely due to this fact that Locke considers ideas and Pavlov patterns of action. The responses observed by Pavlov tend to carry a process to a successful conclusion or to avoid a catastrophe. Salivation is important for deglutition and for digestion, while the avoidance of what we should consider a painful stimulus tends to protect the animal from bodily injury.
Thus there enters into the conditioned reflex something that we may call affective tone. We need not associate this with our own sensations of pleasure and pain, nor need we in the abstract associate it with the advantage of the animal. The essential thing is this: that affective tone is arranged on some sort of scale from negative "pain" to positive "pleasure"; that for a considerable time, or permanently, an increase in affective tone favors all processes in the nervous system that are under way at the time and gives them a secondary power to increase affective tone; and that a decrease in affective tone
tends to inhibit all processes under way at the time and gives them a secondary ability to decrease affective tone. Biologically speaking, of course, a greater affective tone must occur predominantly in situations favorable for the perpetuation of the race, if not the individual, and a smaller affective tone in situations which are unfavorable for this perpetuation, if not disastrous. Any race not conforming to this requirement will go the way of Lewis Carroll's Bread-and-Butter Fly, and always die.
Nevertheless, even a doomed race may show a mechanism valid so long as the race lasts. In other words, even the most suicidal apportioning of affective tone will produce a definite pattern of conduct. Note that the mechanism of affective tone is itself a feedback mechanism.
It may even be given a diagram such as shown in Fig. 7. Here the totalizer for affective tone combines the affective tones given by the separate affective-tone mechanisms over a short interval in the past, according to some rule which we need not specify now. The leads back to the individual affective-tone
mechanisms serve to modify the intrinsic affective tone of each process in the direction of the output of the totalizer, and this modification stands until it is modified by later messages from the totalizer. The leads back from the totalizer to the process mechanisms serve to lower thresholds if the total affective tone is increasing, and to raise them if the total affective tone is decreasing. They likewise have a longtime effect, which endures until it is modified by another impulse from the totalizer.
This lasting effect, however, is confined to those processes actually in being at the time the return message arrives, and a similar limitation also applies to the effects on the individual affective-tone mechanisms. I wish to emphasize that I do not say that the process of the conditioned reflex operates according to the mechanism I have given; I merely say that it could so operate. If, however, we assume this or any simular mechanism, there are a good many things we can say concerning it.
One is that this mechanism is capable of learning. It has already been recognized that the conditioned reflex is a learning mechanism, and this idea has been used in the behaviorist studies of the learning of rats in a maze. All that is needed is that the inducements or punishments used have, respectively, a positive and a negative affective tone.
This is certainly the case, and the experimenter learns the nature of this affective tone by experience, not simply by a priori considerations. Another point of considerable interest is that such a mechanism involves a certain set of messages which go out generally into the nervous system, to all elements which are in a state to receive them. These are the return messages from the affective-tone totalizer, and to a certain extent the messages from the affective-tone mechanisms to the totalizers.
need not be a separate element but may merely represent some natural combinatory effect of messages arriving from the individual affective-tone mechanisms. Now, such messages ''to whom it may concern" may well be sent out most efficiently, with a smallest cost in apparatus, by channels other than nervous. In a similar manner, the ordinary communication system of a mine may consist of a telephone central with the attached wiring and pieces of apparatus.
When we want to empty a mine in a hurry, we do not trust to this, but break a tube of a mercaptan in the air intake. Chemical messengers like this, or like the hormones, are the simplest and most effective for a message not addressed to a specific recipient. For the moment, let me break into what I know to be pure fancy.
The high emotional and consequently affective content of hormonal activity is most suggestive. This does not mean that a purely nervous mechanism is not capable of affective tone and of learning, but it does mean that in the study of this aspect of our mental activity, we cannot afford to be blind to the possibilities of hormonal transmission. It may be excessively fanciful to attach this notion to the fact that in the theories of Freud the memory--the storage function of the nervous system--and the activities of sex are both involved.
Sex, on the one hand, and all affective content, on the other, contain a very strong hormonal element. This suggestion of the importance of sex and hormones has been made to me by Dr. Lettvin and Mr.
一方面,性和所有情感内容都含有非常强的激素成分。关于性和激素重要性的这一建议是由莱特文博士和
Oliver Selfridge, While at present there is no adequate evidence to prove its validity, it is not manifestly absurd in principle. There is nothing in the nature of the computing machine which forbids it to show conditioned reflexes. Let us remember that a computing machine in action is more than the concatenation of relays and storage mechanisms which the designer has
built into it. It also contains the content of its storage mechanisms, and this content is never completely cleared in the course of a single run. We have already seen that it is the run rather than the entire existence of the mechanical structure of the computing machine which corresponds to the life of the individual.
We have also seen that in the nervous computing machine it is highly probable that information is stored largely as changes in the permeability of the synapses, and it is perfectly possible to construct artificial machines where information is stored in that way. It is perfectly possible, for example, to cause any message going into storage to change in a permanent or semi-permanent way the grid bias of one or of a number of vacuum tubes, and thus to alter the numerical value of the summation of impulses which will make the tube or tubes fire. A more detailed account of learning apparatus in computing and control machines, and the uses to which it may be put, may well be left to the engineer rather than to a preliminary book like this one.
It is perhaps better to devote the rest of this chapter to the more developed, normal uses of modern computing machines. One of the chief of these is in the solution of partial differential equations. Even linear partial differential equations require the recording of an enormous mass of data to set them up, as the data involve the accurate description of functions of two or more variables.
With equations of the hyperbolic type, like the wave equation, the typical problem is that of solving the equation when the initial data are given, and this can be done in a progressive manner from the initial data to the results at any given later time. This is largely true of equations of the parabolic type as well. When it comes to equations of the elliptic type, where the natural data are boundary values rather than initial values, the natural methods of solution involve an
iterative process of successive approximation. This process is repeated a very large number of times, so that very fast methods, such as those of the modern computing machine, are almost indispensable. In non-linear partial differential equations, we miss what we have in the case of the linear equations--a reasonably adequate, purely mathematical theory.
Here computational methods are not only important for the handling of particular numerical cases, but, as von Neumann has pointed out, we need them in order to form that acquaintance with a large number of particular cases without which we can scarcely formulate a general theory. To some extent this has been done with the aid of very expensive experimental apparatus, such as wind tunnels. It is in this way that we have become acquainted with the more complicated properties of shock waves, slip surfaces, turbulence, and the like, for which we are scarcely in a position to give an adequate mathematical theory.
How many undiscovered phenomena of similar nature there may be, we do not know. The analogue machines are so much less accurate, and in many cases so much slower, than the digital machines that the latter give us much more promise for the future. It is already becoming clear in the use of these new machines that they demand purely mathematical techniques of their own, quite different from those in use in manual computation or in the use of machines of smaller capacity.
For example, even the use of machines for computing determinants of moderately high order or for the simultaneous solution of twenty or thirty simultaneous linear equations shows difficulties which do not arise when we study analogous problems of small order. Unless care is exercised in setting up a problem, these may completely deprive the solution of any significant figures whatever.
a commonplace to say that fine, effective tools like the ultra-rapid computing machine are out of place in the hands of those not possessing a sufficient degree of technical skill to take full advantage of them. The ultra-rapid computing machine will certainly not decrease the need for mathematicians with a high level of understanding and technical training. In the mechanical or electrical construction of computing machines, there are a few maxims which deserve consideration.
One is that mechanisms which are relatively frequently used, such as multiplying or adding mechanisms, should be in the form of relatively standardized assemblages adapted for one particular use and no other, while those of more occasional use should be assembled for the moment of use out of elements also available for other purposes. Closely related to this consideration is the one that in these more general mechanisms the component parts should be available in accordance with their general properties, and should not be allotted permanently to a specific association with other pieces of apparatus. There should be some part of the apparatus, like an automatic telephone-switching exchange, which will search for free components and connectors of the various sorts and allot them as they are needed.
This will eliminate much of the very large expense which is due to having a great number of unused elements which cannot be used unless their entire large assembly is used. We shall find this principle is very important when we come to consider traffic problems and overloading in the nervous system. As a final remark, let me point out that a large computing machine, whether in the form of mechanical or electric apparatus or in the form of the brain itself, uses up a considerable amount of power, all of which is wasted and dissipated in heat.
The blood leaving the brain is a fraction of a degree warmer
流出的血液比流入的血液温度高几分之一度。
Chapter V
than that entering it. No other computing machine approaches the economy of energy of the brain. In a large apparatus like the Eniac or Edvac, the filaments of the tubes consume a quantity of energy which may well be measured in kilowatts, and unless adequate ventilating and cooling apparatus is provided, the system will suffer from what is the mechanical equivalent of pyrexia, until the constants of the machine are radically changed by the heat, and its performance breaks down.
Nevertheless, the energy spent per individual operation is almost vanishingly small, and does not even begin to form an adequate measure of the performance of the apparatus. The mechanical brain does not secrete thought "as the liver does bile," as the earlier materialists claimed, nor does it put it out in the form of energy, as the muscle puts out its activity. Information is information, not matter or energy.
No materialism which does not admit this can survive at the present day.
任何不承认这一点的唯物论在今天都无法生存。
VI Gestalt and Universals
class='
VI. Gestalt and Universals
VI. Gestalt and Universals
VI. Gestalt and Universals
视觉皮层中的群扫描机制示意图
Among other things which we have discussed in the previous chapter is the possibility of assigning a neural mechanism to Locke's theory of the association of ideas. According to Locke, this occurs according to three principles: the principle of contiguity, the principle of similarity, and the principle of cause and effect. The third of these is reduced by Locke, and even more definitively by Hume, to nothing more than constant concomitance, and so is subsumed under the first, that of contiguity.
The second, that of similarity, deserves a more detailed discussion. How do we recognize the identity of the features of a man, whether we see him in profile, in three-quarters face, or in full face? How do we recognize a circle as a circle, whether it is large or small, near or far; whether, in fact, it is in a plane perpendicular to a line from the eye meeting it in the middle, and is seen as a circle, or has some other orientation, and is seen as an ellipse?
How do we see faces and animals and maps in clouds, or in the blots of a Rorschach test? All these examples refer to the eye, but similar problems extend to the other senses, and some of them have to do with intersensory relations. How do we put into words the call of a bird or the stridulations of an insect?
How do we identify the roundness of a coin by touch?
我们如何通过触觉识别一枚硬币的圆形?
Chapter VI
For the present, let us confine ourselves to the sense of vision. One important factor in the comparison of form of different objects is certainly the interaction of the eye and the muscles, whether they are the muscles within the eyeball, the muscles moving the eyeball, the muscles moving the head, or the muscles moving the body as a whole. Indeed, some form of this visual-muscular feedback system is important as low in the animal kingdom as the flatworms.
There the negative phototropism, the tendency to avoid the light, seems to be controlled by the balance of the impulses from the two eyespots. This balance is fed back to the muscles of the trunk, turning the body away from the light, and, in combination with the general impulse to move forward, carries the animal into the darkest region accessible. It is interesting to note that a combination of a pair of photocells with appropriate amplifiers, a Wheatstone bridge for balancing their outputs, and further amplifiers controlling the input into the two motors of a twinscrew mechanism would give us a very adequate negatively phototropic control for a little boat.
It would be difficult or impossible for us to compress this mechanism into the dimensions that a flatworm can carry; but here we merely have another exemplification of the fact that must by now be familiar to the reader, that living mechanisms tend to have a much smaller space scale than the mechanisms best suited to the techniques of human artificers, although, on the other hand, the use of electrical techniques gives the artificial mechanism an enormous advantage in speed over the living organism. Without going through all the intermediate stages, let us come at once to the eye-muscle feedbacks in man. Some of these are of purely homeostatic nature, as when the pupil opens in the dark and closes in the light, thus tending to confine the
flow of light into the eye between narrower bounds than would otherwise be possible. Others concern the fact that the human eye has economically confined its best form and color vision to a relatively small fovea, while its perception of motion is better on the periphery. When the peripheral vision has picked up some object conspicuous by brilliancy or light contrast or color or above all by motion, there is a reflex feedback to bring it into the fovea.
This feedback is accompanied by a complicated system of interlinked subordinate feedbacks, which tend to converge the two eyes so that the object attracting attention is in the same part of the visual field of each, and to focus the lens so that its outlines are as sharp as possible. These actions are supplemented by motions of the head and body, by which we bring the object into the center of vision if this cannot be done readily by a motion of the eyes alone, or by which we bring an object outside the visual field picked up by some other sense into that field. In the case of objects with which we are more familiar in one angular orientation than another--writing, human faces, landscapes, and the like--there is also a mechanism by which we tend to pull them into the proper orientation.
All these processes can be summed up in one sentence: we tend to bring any object that attracts our attention into a standard position and orientation, so that the visual image which we form of it varies within as small a range as possible. This does not exhaust the processes which are involved in perceiving the form and meaning of the object, but it certainly facilitates all later processes tending to this end. These later processes occur in the eye and in the visual cortex.
There is considerable evidence that for a considerable number of stages each step in this process diminishes the number of neuron channels involved in the
transmission of visual information, and brings this information one step nearer to the form in which it is used and is preserved in the memory. The first step in this concentration of visual information occurs in the transition between the retina and the optic nerve. It will be noted that while in the fovea there is almost a one-one correspondence between the rods and cones and the fibers of the optic nerve, the correspondence on the periphery is such that one optic nerve fiber corresponds to ten or more end organs.
This is quite understandable, in view of the fact that the chief function of the peripheral fibers is not so much vision itself as a pickup for the centering and focusing-directing mechanism of the eye. One of the most remarkable phenomena of vision is our ability to recognize an outline drawing. Clearly, an outline drawing of, say, the face of a man, has very little resemblance to the face itself in color, or in the massing of light and shade, yet it may be a most recognizable portrait of its subject.
The most plausible explanation of this is that, somewhere in the visual process, outlines are emphasized and some other aspects of an image are minimized in importance. The beginning of these processes is in the eye itself. Like all senses, the retina is subject to accommodation; that is, the constant maintenance of a stimulus reduces its ability to receive and to transmit that stimulus.
This is most markedly so for the receptors which record the interior of a large block of images with constant color and illumination, for even the slight fluctuations of focus and point of fixation which are inevitable in vision do not change the character of the image received. It is quite different on the boundary of two contrasting regions. Here these fluctuations produce an alternation between one stimulus and another, and this alternation, as we see in the
phenomenon of after-images, not only does not tend to exhaust the visual mechanism by accommodation but even tends to enhance its sensitivity. This is true whether the contrast between the two adjacent regions is one of light intensity or of color. As a comment on these facts, let us note that three-quarters of the fibers in the optic nerve respond only to the flashing "on" of illumination.
We thus find that the eye receives its most intense impression at boundaries, and that every visual image in fact has something of the nature of a line drawing. Probably not all of this action is peripheral. In photography, it is known that certain treatments of a plate increase its contrasts, and such phenomena, which are of non-linearity, are certainly not beyond what the nervous system can do.
They are allied to the phenomena of the telegraph-type repeater, which we have already mentioned. Like this, they use an impression which has not been blurred beyond a certain point to trigger a new impression of a standard sharpness. At any rate, they decrease the total unusable information carried by an image, and are probably correlated with a part of the reduction of the number of transmission fibers found at various stages of the visual cortex.
We have thus designated several actual or possible stages of the diagrammatization of our visual impressions. We center our images around the focus of attention and reduce them more or less to outlines. We have now to compare them with one another, or at any rate with a standard impression stored in memory, such as "circle" or "square." This may be done in several ways.
We have given a rough sketch which indicates how the Lockean principle of contiguity in association may be mechanized. Let us notice that the principle of contiguity also covers much of the other Lockean principle of similarity. The different
aspects of the same object are often to be seen in those processes which bring it to the focus of attention, and of other motions which lead us to see it, now at one distance and now at another, now from one angle and now from a distinct one. This is a general principle, not confined in its application to any particular sense and doubtless of much importance in the comparison of our more complicated experiences. It is nevertheless probably not the only process which leads to the formation of our specifically visual general ideas, or, as Locke would call them, "complex ideas." The structure of our visual cortex is too highly organized, too specific, to lead us to suppose that it operates by what is after all a highly generalized mechanism.
It leaves us the impression that we are here dealing with a special mechanism which is not merely a temporary assemblage of general-purpose elements with interchangeable parts, but a permanent sub-assembly like the adding and multiplying assemblies of a computing machine. Under the circumstances, it is worth considering how such a sub-assembly might possibly work and how we should go about designing it. The possible perspective transformations of an object form what is known as a group, in the sense in which we have already defined one in Chapter II.
This group defines several sub-groups of transformations: the affine group, in which we consider only those transformations which leave the region at infinity untouched; the homogeneous dilations about a given point, in which one point, the directions of the axes, and the equality of scale in all directions are preserved; the transformations preserving length; the rotations in two or three dimensions about a point; the set of all translations; and so on. Among these groups, the ones we have just mentioned are continuous; that is, the operations belonging to them are determined by the values of
a number of continuously varying parameters in an appropriate space. They thus form multidimensional configurations in n-space, and contain sub-sets of transformations which constitute regions in such a space. Now, just as a region in the ordinary two-dimensional plane is covered by the process of scanning known to the television engineer, by which a nearly uniformly distributed set of sample positions in that region is taken to represent the whole, so every region in a group-space, including the whole of such a space, can be represented by a process of group scanning.
In such a process, which is by no means confined to a space of three dimensions, a net of positions in the space is traversed in a one-dimensional sequence, and this net of positions is so distributed that it comes near to every position in the region, in some appropriately defined sense. It will thus contain positions as near to any we wish as may be desired. If these "positions," or sets of parameters, are actually used to generate the appropriate transformations, it means that the results of transforming a given figure by these transformations will come as near as we wish to any given transformation of the figure by a transformation operator lying in the region desired.
If our scanning is fine enough, and the region transformed has the maximum dimensionality of the regions transformed by the group considered, this means that the transformations actually traversed will give a resulting region overlapping any transform of the original region by an amount which is as large a fraction of its area as we wish. Let us then start with a fixed comparison region and a region to be compared with it. If at any stage of the scanning of the group of transformations the image of the region to be compared under some one of the transformations scanned coincides
more perfectly with the fixed pattern than a given tolerance allows, this is recorded, and the two regions are said to be alike. If this happens at no stage of the scanning process, they are said to be unlike. This process is perfectly adapted to mechanization, and serves as a method to identify the shape of a figure independently of its size or its orientation or of whatever transformations may be included in the group-region to be scanned.
If this region is not the entire group, it may well be that region A seems like region B, and that region B seems like region C, while region A does not seem like region C. This certainly happens in reality. A figure may not show any particular resemblance to the same figure inverted, at least in so far as the immediate impression--one not involving any of the higher processes--is concerned.
Nevertheless, at each stage of its inversion, there may be a considerable range of neighboring positions which appear similar. The universal "ideas" thus formed are not perfectly distinct but shade into one another. There are other more sophisticated means of using group scanning to abstract from the transformations of a group.
The groups which we here consider have a "group measure," a probability density which depends on the transformation group itself and does not change when all the transformations of the group are altered by being preceded or followed by any specific transformation of the group. It is possible to scan the group in such a way that the density of scanning of any region of a considerable class--that is, the amount of time which the variable scanning element passes within the region in any complete scanning of the group--is closely proportional to its group measure. In the case of such a uniform scanning, if we have any quantity depending on a set S of elements transformed by the group, and
if this set of elements is transformed by all the transformations of the group, let us designate the quantity depending on S by Q(S), and let us use TS to express the transform of the set S by the transformation T of the group. Then Q(TS) will be the value of the quantity replacing Q(S) when S is replaced by TS. If we average or integrate this with respect to the group measure for the group of transformations T, we shall obtain a quantity which we may write in some such form as
where the integration is over the group measure. Quantity 6.01 will be identical for all sets S interchangeable with one another under the transformations of the group, that is, for all sets S which have in some sense the same form or Gestalt. It is possible to obtain an approximate comparability of form where the integration in Quantity 6.01 is over less than the whole group, if the integrand Q(TS) is small over the region omitted.
So much for group measure. In recent years, there has been a good deal of attention to the problem of the prosthesis of one lost sense by another. The most dramatic of the attempts to accomplish this has been the design of reading devices for the blind, to work by the use of photoelectric cells.
We shall suppose that these efforts are confined to printed matter, and even to a single type face or to a small number of type faces. We shall also suppose that the alignment of the page, the centering of the lines, the traverse from line to line are taken care of either manually or, as they may well be, automatically. These processes correspond, as we may see, to the part of our visual Gestalt determination which depends on muscular feedbacks and the use of our normal centering, orienting, focusing, and converging apparatus.
of determining the shapes of the individual letters as the scanning apparatus passes over them in sequence. It has been suggested that this be done by the use of several photoelectric cells placed in a vertical sequence, each attached to a sound-making apparatus of a different pitch. This can be done with the black of the letters registering either as silence or as sound.
Let us assume the latter case, and let us assume three photocell receptors above one another. Let them record as the three notes of a chord, let us say, with the highest note on top and the lowest note below. Then the letter capital F, let us say, will record ------------
The letter capital Z will record ------------ -------------- the letter capital O ----
字母Z将记录为——字母O将记录为——
—
--and so on. With the ordinary help given by our ability to interpret, it should not be too difficult to read such an auditory code, not more difficult than to read Braille, for instance. However, all this depends on one thing: the proper relation of the photocells to the vertical height of the letters.
Even with standardized type faces, there still are great variations in the size
即使使用标准化字体,字体的大小仍然存在很大的变化。
Gestalt and Universals
of the type. Thus it is desirable for us to be able to pull the vertical scale of the scanning up or down, in order to reduce the impression of a given letter to a standard. We must at least have at our disposal, manually or automatically, some of the transformations of the vertical dilation group.
There are several ways we might do this. We might allow for a mechanical vertical adjustment of our photocells. On the other hand, we might use a rather large vertical array of photocells and change the pitch assignment with the size of type, leaving those above and below the type silent.
This may be done, for example, with the aid of a schema of two sets of connectors, the inputs coming up from the photocells, and leading to a series of switches of wider and wider divergence, and the outputs a series of vertical lines, as in Fig. 8. Here the single lines represent the leads from the photocells, the double lines the leads to the oscillators, the circles on the dotted lines the points of connections between incoming and outgoing leads, and the dotted lines themselves the leads whereby one or another of a bank of oscillators is put into action. This was the device, to which we have referred in the introduction, designed by McCulloch for the purpose of adjusting to the height of the type face.
In the first design, the selection between dotted line and dotted line was manual.
在最初的设计中,虚线之间的选择是手动的。
Fig. 8 — 光电管信号示意图
Fig. 8
Chapter VI
This was the figure which, when shown to Dr. von Bonin, suggested the fourth layer of the visual cortex. It was the connecting circles which suggested the neuron cell bodies of this layer, arranged in sub-layers of uniformly changing horizontal density, and size changing in the opposite direction to the density. The horizontal leads are probably fired in some cyclical order.
The whole apparatus seems quite suited to the process of group scanning. There must of course be some process of recombination in time of the upper outputs. This then was the device suggested by McCulloch as that actually used in the brain in the detection of visual Gestalt.
It represents a type of device usable for any sort of group scanning. Something similar occurs in other senses as well. In the ear, the transposition of music from one fundamental pitch to another is nothing but a translation of the logarithm of the frequency, and may consequently be performed by a group-scanning apparatus.
A group-scanning assembly thus has well-defined, appropriate anatomical structure. The necessary switching may be performed by independent horizontal leads which furnish enough stimulation to shift the thresholds in each level to just the proper amount to make them fire when the lead comes on. While we do not know all the details of the performance of the machinery, it is not at all difficult to conjecture a possible machine conforming to the anatomy.
In short, the group-scanning assembly is well adapted to form the sort of permanent sub-assembly of the brain corresponding to the adders or multipliers of the numerical computing machine. Lastly, the scanning apparatus should have a certain intrinsic period of operation which should be identifiable in the performance of the brain. The order of magnitude of this period
should show in the minimum time required for making direct comparison of the shapes of objects different in size. This can be done only when the comparison is between two objects not too different in size; otherwise, it is a long-time process, suggestive of the action of a non-specific assembly. When direct comparison seems to be possible, it appears to take a time of the order of magnitude of a tenth of a second.
This also seems to accord with the order of magnitude of the time needed by excitation to stimulate all the layers of transverse connectors in cyclical sequence. While this cyclical process then might be a locally determined one, there is evidence that there is a widespread synchronism in different parts of the cortex, suggesting that it is driven from some clocking center. In fact, it has the order of frequency appropriate for the alpha rhythm of the brain, as shown in electroencephalograms.
We may suspect that this alpha rhythm is associated with form perception, and that it partakes of the nature of a sweep rhythm, like the rhythm shown in the scanning process of a television apparatus. It disappears in deep sleep, and seems to be obscured and overlaid with other rhythms, precisely as we might expect, when we are actually looking at something and the sweep rhythm is acting as something like a carrier for other rhythms and activities. It is most marked when the eyes are closed in waking, or when we are staring into space at nothing in particular, as in the condition of abstraction of a yogi,1 when it shows an almost perfect periodicity.
We have just seen that the problem of sensory prosthesis--the problem of replacing the information normally conveyed through a lost sense by information through another sense still available--is important and not necessarily insoluble. What makes it more hopeful is the fact that the memory and
association areas, normally approached through one sense, are not locks with a single key but are available to store impressions gathered from other senses than the one to which they normally belong. A blinded man, as distinguished perhaps from one congenitally blind, not only retains visual memories earlier in date than his accident but is even able to store tactile and auditory impressions in a visual form. He may feel his way around a room, and yet have an image of how it ought to look.
Thus, part of his normal visual mechanism is accessible to him. On the other hand, he has lost more than his eyes: he has also lost the use of that part of his visual cortex which may be regarded as a fixed assembly for organizing the impressions of sight. It is necessary to equip him not only with artificial visual receptors but with an artificial visual cortex, which will translate the light impressions on his new receptors into a form so related to the normal output of his visual cortex that objects which ordinarily look alike will now sound alike.
Thus the criterion of the possibility of such a replacement of sight by hearing is at least in part a comparison between the number of recognizably different visual patterns and recognizably different auditory patterns at the cortical level. This is a comparison of amounts of information. In view of the somewhat similar organization of the different parts of the sensory cortex, it will probably not differ very much from a comparison between the areas of the two parts of the cortex.
This is about 100:1 as between sight and sound. If all the auditory cortex were used for vision, we might expect to get a quantity of reception of information about 1 per cent of that coming in through the eye. On the other hand, our usual scale for the estimation of vision is in terms of the relative distance at which a certain degree of resolution of pattern is obtained, and thus a 10/100 vision means
an amount of flow of information about 1 per cent of normal. This is very poor vision; it is, however, definitely not blindness, nor do people with this amount of vision necessarily consider themselves as blind. In the other direction, the picture is even more favorable.
The eye can detect all of the nuances of the ear with the use of only 1 per cent of its facilities, and still leave a vision of about 95/100, which is substantially perfect. Thus the problem of sensory prosthesis is an extremely hopeful field of work.
It is necessary that I commence this chapter with a disavowal. On the one hand, I am not a psychopathologist nor a psychiatrist, and lack any experience in a field where the guidance of experience is the only trustworthy one. On the other hand, our knowledge of the normal performance of the brain and the nervous system, and a fortiori our knowledge of their abnormal performance, is far from having reached that state of perfection where an a priori theory can command any confidence.
I therefore wish to disclaim in advance any assertion that any particular entity in psychopathology, as for example any of the morbid conditions described by Kraepelin and his disciples, is due to a specific type of defect in the organization of the brain as a computing machine. Those who may draw such specific conclusions from the considerations of this book do so at their own risk. Nevertheless, the realization that the brain and the computing machine have much in common may suggest new and valid approaches to psychopathology and even to psychiatries.
These begin with perhaps the simplest question of all: how the brain avoids gross blunders, gross miscarriages of activity, due to the malfunction of individual components. Similar questions
referring to the computing machine are of great practical importance, for here a chain of operations, each covering a fraction of a millisecond, may last a matter of hours or days. It is quite possible for a chain of computational operations to involve 109 separate steps. Under these circumstances, the chance that at least one operation will go amiss is very far from negligible, even though, it is true, the reliability of modern electronic apparatus has far exceeded the most sanguine expectations.
In ordinary computational practice by hand or by desk machines, it is the custom to check every step of the computation and, when an error is found, to localize it by a backward process starting from the first point where the error is noted. To do this with a high-speed machine, the check must proceed with the speed of the original machine, or the whole effective order of speed of the machine will conform to that of the slower process of checking. Furthermore, if the machine is made to keep all intermediate records of its performance, its complication and bulk will be increased to an intolerable point, by a factor which is likely to be enormously greater than 2 or 3.
A much better method of checking, and in fact the one generally used in practice, is to refer every operation simultaneously to two or three separate mechanisms. In the case of the use of two such mechanisms, their answers are automatically collated against each other; and if there is a discrepancy, all data are transferred to permanent storage, the machine stops, and a signal is sent to the operator that something is wrong. The operator then compares the results, and is guided by them in his search for the malfunctioning part, perhaps a tube which has burnt out and needs replacement.
If three separate mechanisms are used for each stage and single misfunctions are as rare as they are in fact, there will practically always be an agreement between two of
the three mechanisms, and this agreement will give the required result. In this case, the collation mechanism accepts the majority report, and the machine need not stop; but there is a signal indicating where and how the minority report differs from the majority report. If this occurs at the first moment of discrepancy, the indication of the position of the error may be very precise.
In a well-designed machine, no particular element is assigned to a particular stage in the sequence of operations, but at each stage there is a searching process, quite similar to that used in automatic telephone exchanges, which finds the first available element of a given sort and switches it into the sequence of operations. In this case, the removal and replacement of defective elements need not be the source of any appreciable delay. It is conceivable and not implausible that at least two of the elements of this process are also represented in the nervous system.
We can hardly expect that any important message is entrusted for transmission to a single neuron, nor that any important operation is entrusted to a single neuronal mechanism. Like the computing machine, the brain probably works on a variant of the famous principle expounded by Lewis Carroll in The Hunting of the Snark: "What I tell you three times is true." It is also improbable that the various channels available for the transfer of information generally go from one end of their course to the other without anastomosing. It is much more probable that when a message comes in to a certain level of the nervous system, it may leave that point and proceed to the next by one or more alternative members of what is known as an "internuncial pool." There may be parts of the nervous system, indeed, where this interchangeability is much limited or abolished, and these are likely to be such highly specialized parts of the cortex as those which serve as the inward extensions of the organs of
special sense. Still, the principle holds, and probably holds most clearly for the relatively unspecialized cortical areas which serve the purpose of association and of what we call the higher mental functions. So far we have been considering errors in performance which are normal, and pathological only in an extended sense.
Let us now turn to those which are much more clearly pathological. Psychopathology has been rather a disappointment to the instinctive materialism of the doctors, who have taken the point of view that every disorder must be accompanied by material lesions of some specific tissue involved. It is true that specific brain lesions, such as injuries, tumors, clots, and the like, may be accompanied by psychic symptoms, and that certain mental diseases, such as paresis, are the sequellae of general bodily disease and show a pathological condition of the brain tissue; but there is no way of identifying the brain of a schizophrenic of one of the strict Kraepelin types, nor of a manic-depressive patient, nor of a paranoiac.
These disorders we call functional, and this distinction seems to contravene the dogma of modern materialism that every disorder in function has some physiological or anatomical basis in the tissues concerned. This distinction between functional and organic disorders receives a great deal of light from the consideration of the computing machine. As we have already seen, it is not the empty physical structure of the computing machine that corresponds to the brain--to the adult brain, at least--but the combination of this structure with the instructions given it at the beginning of a chain of operations and with all the additional information stored and gained from outside in the course of this chain.
with a physical basis which vanishes when the machine is shut down or the brain dies, and part in the form of long-time memories, which are stored in a way at which we can only guess, but probably also in a form with a physical basis which vanishes at death. There is no way yet known for us to recognize in the cadaver what the threshold of a given synapse has been in life; and even if we knew this, there is no way we can trace out the chain of neurons and synapses communicating with this, and determine the significance of this chain for the ideational content which it records. There is therefore nothing surprising in considering the functional mental disorders as fundamentally diseases of memory, of the circulating information kept by the brain in the active state, and of the long-time permeability of synapses.
Even the grosser disorders such as paresis may produce a large part of their effects not so much by the destruction of tissue which they involve and the alteration of synaptic thresholds as by the secondary disturbances of traffic--the overload of what remains of the nervous system and the re-routing of messages--which must follow such primary injuries. In a system containing a large number of neurons, circular processes can hardly be stable for long periods of time. Either, as in the case of memories belonging to the specious present, they run their course, dissipate themselves, and die out, or they comprehend more and more neurons in their system, until they occupy an inordinate part of the neuron pool.
This is what we should expect to be the case in the malignant worry which accompanies anxiety neuroses. In such a case, it is possible that the patient simply does not have the room, the sufficient number of neurons, to carry out his normal processes of thought. Under such conditions, there may be less going on in the brain
to load up the neurons not yet affected, so that they are all the more readily involved in the expanding process. Furthermore, the permanent memory becomes more and more deeply involved, and the pathological process which occurred at first at the level of the circulating memories may repeat itself in a more intractable form at the level of the permanent memories. Thus what started as a relatively trivial and accidental reversal of stability may build itself up into a process totally destructive to the ordinary mental life.
Pathological processes of a somewhat similar nature are not unknown in the case of mechanical or electrical computing machines. A tooth of a wheel may slip under just such conditions that no tooth with which it engages can pull it back into its normal relations, or a high-speed electrical computing machine may go into a circular process which there seems to be no way to stop. These contingencies may depend on a highly improbable instantaneous configuration of the system, and, when remedied, may never--or very rarely--repeat themselves.
However, when they occur, they temporarily put the machine out of action. How do we deal with these accidents in the use of the machine? The first thing which we try is to clear the machine of all information, in the hope that when it starts again with different data the difficulty may not recur.
Failing this, if the difficulty is in some point permanently or temporarily inaccessible to the clearing mechanism, we shake the machine or, if it is electrical, subject it to an abnormally large electrical impulse, in the hope that we may reach the inaccessible part and throw it into a position where the false cycle of its activities will be interrupted. If even this fails, we may disconnect an erring part of the apparatus, for it is possible that what yet remains may be adequate for our purpose.
Now there is no normal process except death which completely clears the brain from all past impressions; and after death, it is impossible to set it going again. Of all normal processes, sleep comes the nearest to a non-pathological clearing. How often we find that the best way to handle a complicated worry or an intellectual muddle is to sleep over it!
However, sleep does not clear away the deeper memories, nor indeed is a sufficiently malignant state of worry compatible with an adequate sleep. We are thus often forced to resort to more violent types of intervention in the memory cycle. The more violent of these involve a surgical intervention into the brain, leaving behind it permanent damage, mutilation, and the abridgment of the powers of the victim, as the mammalian central nervous system seems to possess no powers whatever of regeneration.
The principal type of surgical intervention which has been practiced is known as prefrontal lobotomy, and consists in the removal or isolation of a portion of the prefrontal lobe of the cortex. It has recently been having a certain vogue, probably not unconnected with the fact that it makes the custodial care of many patients easier. Let me remark in passing that killing them makes their custodial care still easier.
However, prefrontal lobotomy does seem to have a genuine effect on malignant worry, not by bringing the patient nearer to a solution of his problems but by damaging or destroying the capacity for maintained worry, known in the terminology of another profession as the conscience. More genei:ally, it appears to limit all aspects of the circulating memory, the ability to keep in mind a situation not actually presented. The various forms of shock treatment--electric, insulin, metrazol--are less drastic methods of doing a very similar thing.
memory. In so far as this concerns the circulating memory, and in so far as this memory is chiefly damaged for the recent period of mental disorder, and is probably scarcely worth preserving anyhow, shock treatment has something definite to recommend it as against lobotomy; but it is not always free from deleterious effects on the permanent memory and the personality. As it stands at present, it is another violent, imperfectly understood, imperfectly controlled method to interrupt a mental vicious circle.
This does not prevent its being in many cases the best thing we can do at present. Lobotomy and shock treatment are methods which by their very nature are more suited to handle vicious circulating memories and malignant worries than the deeper-seated permanent memories, though it is not impossible that they may have some effect here too. As we have said, in long-established cases of mental disorder, the permanent memory is as badly deranged as the circulating memory.
We do not seem to possess any purely pharmaceutical or surgical weapon for intervening differentially in the permanent memory. This is where psychoanalysis and other similar psychotherapeutic measures come in. Whether psychoanalysis is taken in the orthodox Freudian sense or in the modified senses of Jung and of Adler, or whether our psychotherapy is not strictly psychoanalytic at all, our treatment is clearly based on the concept that the stored information of the mind lies on many levels of accessibility and is much richer and more varied than that which is accessible by direct unaided introspection; that it is vitally conditioned by affective experiences which we cannot always uncover by such introspection, either because they never were made explicit in our adult language, or because they have been buried by a definite mechanism, affective though generally involuntary; and that the content of these stored
experiences, as well as their affective tone, conditions much of our later activity in ways which may well be pathological. The technique of the psychoanalyst consists in a series of means to discover and interpret these hidden memories, to make the patient accept them for what they are and by their acceptance modify, if not their content, at least the affective tone they carry, and thus make them less harmful. All this is perfectly consistent with the point of view of this book.
It perhaps explains, too, why there are circumstances where a joint use of shock treatment and psychotherapy is indicated, combining a physical or pharmacological therapy for the phenomena of reverberation in the nervous system, and a psychological therapy for the long-time memories which, without interference, might re-establish from within the vicious circle broken up by the shock treatment. We have already mentioned the traffic problem of the nervous system. It has been commented on by many writers, such as D'Arcy Thompson,1 that each form of organization has an upper limit of size, beyond which it will not function.
Thus the insect organization is limited by the length of tubing over which the spiracle method of bringing air by diffusion directly to the breathing tissues will function; a land animal cannot be so big that the legs or ether portions in contact with the ground will be crushed by its weight; a tree is limited by the mechanism for transferring water and minerals from the roots to the leaves, and the products of photosynthesis from the leaves to the roots; and so on. The same sort of thing is observed in engineering constructions. Skyscrapers are limited in size by the fact that when they exceed a certain height, the elevator space needed for the upper stories consumes an excessive part of the cross section of the lower floors.
suspension bridge which can be built out of materials with given elastic properties will collapse under its own weight; and beyond a certain greater span, any structure built of a given material or materials will collapse under its own weight. Similarly, the size of a single telephone central, built according to a constant, non-expanding plan, is limited, and this limitation has been very thoroughly studied by telephone engineers. In a telephone system, the important limiting factor is the fraction of the time during which a subscriber will find it impossible to put a call through.
A 99 per cent chance of success will certainly be satisfactory for even the most exacting; 90 per cent of successful calls is probably good enough to permit business to be carried on with reasonable facility. A success of 75 per cent is already annoying but will permit business to be carried on after a fashion; while if half the calls end in failures, subscribers will begin to ask to have their telephones taken out. Now, these represent over-all figures.
If the calls go through n distinct stages of switching, and probability of failure is independent and equal for each stage, in order to get a probability of total success equal to p, the probability of success at each stage must be p1/n. Thus to obtain a 75 per cent chance of the completion of the call after five stages, we must have about 95 per cent chance of success per stage. To obtain a 90 per cent performance, we must have 98 per cent chance of success at each stage.
To obtain a 50 per cent performance, we must have 87 per cent chance of success at each stage. It will be seen that the more stages which are involved, the more rapidly the service becomes extremely bad when a critical level of failure for the individual call is exceeded, and extremely good when this critical level of failure is not quite reached. Thus a switching service involving many stages and designed for a certain level of failure shows no obvious signs of failure until the
traffic comes up to the edge of the critical point, when it goes completely to pieces, and we have a catastrophic traffic jam. Man, with the best-developed nervous system of all the animals, with behavior that probably depends on the longest chains of effectively operated neuronic chains, is then likely to perform a complicated type of behavior efficiently very close to the edge of an overload, when he will give way in a serious and catastrophic way. This overload may take place in several ways: either by an excess in the amount of traffic to be carried, by a physical removal of channels for the carrying of traffic, or by the excessive occupation of such channels by undesirable systems of traffic, like circulating memories which have increased to the extent of becoming pathological worries.
In all these cases, a point will come--quite suddenly--when the normal traffic will not have space enough allotted to it, and we shall have a form of mental breakdown, very possibly amounting to insanity. This will first affect the faculties or operations involving the longest chains of neurons. There is appreciable evidence that these are precisely the processes which are recognized to be the highest in our ordinary scale of valuation.
The evidence is this: a rise in temperature within nearly physiological limits is known to produce an increase in the ease of performance of most if not of all neuronic processes. This is greater for the higher processes, roughly in the order of our usual estimate of their degree of "highness." Now, any facilitation of a process in a single neuron-synapse system should be cumulative as the neuron is combined in series with other neurons. Thus the amount of assistance a process receives through a rise in temperature is a rough measure of the length of the neuron chain it involves.
mental disorders are certainly most conspicuous and probably most common in man. There is another more specific way of considering a very similar matter. Let us first consider two brains geometrically similar, with the weights of gray and of white matter related by the same factor of proportionality, but with different linear dimensions in the ratio A:B.
Let the volume of the cell bodies in the gray matter and the cross sections of the fibers in the white matter be of the same size in both brains. Then the number of cell bodies in the two cases bears the ratio A3:B3, and the number of long-distance connectors the ratio A2:B2. This means that for the same density of activity in the cells, the density of activity in the fibers is A:B times as great in the case of the large brain as in that of the small brain.
If we compare the human brain with that of a lower mammal, we shall find that it is much more convoluted. The relative thickness of the gray matter is much the same, but it is spread over a far more involved system of gyri and sulci. The effect of this is to increase the amount of gray matter at the expense of the amount of white matter.
Within a gyrus, this decrease of the white matter is largely a decrease in length rather than in number of fibers, as the opposing folds of a gyrus are nearer together than they would be on a smooth-surfaced brain of the same size. On the other hand, when it comes to the connectors between different gyri, the distance they have to run is increased if anything by the convolution of the brain. Thus the human brain would seem to be fairly efficient in the matter of the short-distance connectors, but quite defective in the matter of long-distance trunk lines.
This means that in case of a traffic jam the processes involving parts of the brain quite remote from one another should suffer first. That is, processes involving several centers, a number of different motor processes, and a considerable number of
association areas should be among the least stable in cases of insanity. These are precisely the processes which we should normally class as higher, and we obtain another confirmation of our expectation, which seems to be verified by experience, that the higher processes deteriorate first in insanity. There is some evidence that the long-distance paths in the brain have a tendency to run outside of the cerebrum altogether and to traverse the lower centers.
This is indicated by the remarkably small damage done by cutting some of the long-distance cerebral loops of white matter. It almost seems as if these superficial connections were so inadequate that they furnish only a small part of the connections really needed. With reference to this, the phenomena of handedness and of hemispheric dominance are interesting.
Handedness seems to occur in the lower mammals, though it is less conspicuous than in man, probably in part because of the lower degree of organization and skill demanded by the tasks which they perform. Nevertheless, the choice between the right and the left side in muscular skill does actually seem to be less than in man even in the lower primates. The right-handedness of the normal man, as is well known, is generally associated with a left-brainedness, and the left-handedness of a minority of humans with a right-brainedness.
That is, the cerebral functions are not distributed evenly over the two hemispheres, and one of these, the dominant hemisphere, has the lion's share of the higher functions. It is true that many essentially bilateral functions--those involving the fields of vision, for example--are represented each in its appropriate hemisphere, though this is not true for all bilateral functions. However, most of the "higher" areas are confined to the dominant hemisphere.
extensive injury in the secondary hemisphere is far less serious than the effect of a similar injury in the dominant hemisphere. At a relatively early age in his career, Pasteur suffered a cerebral hemorrhage on his right side which left him with a moderate degree of one-sided paralysis, a hemiplegia. When he died, his brain was examined, and he was found to be suffering from a right-sided injury, so extensive that it has been said that after his injury "he had only half a brain." There certainly were extensive lesions of the parietal and temporal regions.
Nevertheless, after this injury he did some of his best work. A similar injury of the left side in a right-handed adult would almost certainly have been fatal and would certainly reduce the patient into an animal condition of mental and nervous crippledness. It is said that the situation is considerably better in early infancy, and that in the first six months of life an extensive injury to the dominant hemisphere may compel the normally secondary hemisphere to take its place; so that the patient appears far more nearly normal than he would be had the injury occurred at a later stage.
This is quite in accordance with the general great flexibility shown by the nervous system in the early weeks of life, and the great rigidity which it rapidly develops later. It is possible that, short of such serious injuries, handedness is reasonably flexible in the very young child. However, long before the child is of school age, the natural handedness and cerebral dominance are established for life.
It used to be thought that left-handedness was a serious social disadvantage. With most tools, school desks, and sports equipment primarily made for the right-handed, it certainly is to some extent. In the past, moreover, it was viewed with some of the superstitious disapproval that has attached to so many minor variations from the human norm, such as birthmarks or red hair.
motives, many people have attempted and even succeeded, in changing the external handedness of their children by education, though of course they could not change its physiological basis in hemispheric dominance. It was then found that in very many cases these hemispheric changelings suffered from stuttering and other defects of speech, reading, and writing, to the extent of seriously wounding their prospects in life and their hopes for a normal career. We now see at least one possible explanation for the phenomenon.
With the education of the secondary hand, there has been a partial education of that part of the secondary hemisphere which deals with skilled motions, such as writing. Since, however, these motions are carried out in the closest possible association with reading, speech, and other activities which are inseparably connected with the dominant hemisphere, the neuron chains involved in processes of the sort must cross over from hemisphere to hemisphere and back; and in a process of any complication, they must do this again and again. Now, the direct connectors between the hemispheres--the cerebral commissures--in a brain as large as that of man are so few in number that they are of very little use, and the interhemispheric traffic must go by roundabout routes through the brain stem, which we know very imperfectly but which are certainly long, scanty, and subject to interruption.
As a consequence, the processes associated with speech and writing are very likely to be involved in a traffic jam, and stuttering is the most natural thing in the world. That is, the human brain is probably too large already to use in an efficient manner all the facilities which seem to be anatomically present. In a cat, the destruction of the dominant hemisphere seems to produce relatively less damage than in man, and the destruction of the secondary hemisphere probably more
damage. At any rate, the apportionment of function in the two hemispheres is more nearly equal. In man, the gain achieved by the increase in size and complication of the brain is partly nullified by the fact that less of the organ can be used effectively at one time.
It is interesting to reflect that we may be facing one of those limitations of nature in which highly specialized organs reach a level of declining efficiency and ultimately lead to the extinction of the species. The human brain may be as far along on its road to this destructive specialization as the great nose horns of the last of the titanotheres.
Two of the phenomena which we consider to be characteristic of living systems are the power to learn and the power to reproduce themselves. These properties, different as they appear, are intimately related to one another. An animal that learns is one which is capable of being transformed by its past environment into a different being and is therefore adjustable to its environment within its individual lifetime.
An animal that multiplies is able to create other animals in its own likeness at least approximately, although not so completely in its own likeness that they cannot vary in the course of time. If this variation is itself inheritable, we have the raw material on which natural selection can work. If the hereditary invariability concerns manners of behavior, then among the varied patterns of behavior which are propagated some will be found advantageous to the continuing existence of the race and will establish themselves, while others which are detrimental to this continuing existence will be eliminated.
The result is a certain sort of racial or phylogenetic learning, as contrasted with the ontogenetic learning of the individual. Both ontogenetic and phylogenetic learning are modes by which the animal can adjust itself to its environment.
Both ontogenetic and phylogenetic learning, and certainly the latter, extend themselves not only to all animals but to plants and, indeed, to all organisms which in any sense may be considered to be living. However, the degree to which these two forms of learning are found to be important in different sorts of living beings varies widely. In man, and to a lesser extent in the other mammals, ontogenetic learning and individual adaptability are raised to the highest point.
Indeed, it may be said that a large part of the phylogenetic learning of man has been devoted to establishing the possibility of good ontogenetic learning. It has been pointed out by Julian Huxley in his fundamental paper on the mind of birds1 that birds have a small capacity for ontogenetic learning. Something similar is true in the case of insects, and in both instances it may be associated with the terrific demands made on the individual by flight and the consequential pre-emption of the capabilities of the nervous system which might otherwise be applied to ontogenetic learning.
Complicated as the behavior patterns of birds are--in flying, in courtship, in the care of the young, and in nest building--they are carried out correctly the very first time without the need of any large amount of instruction from the mother. It is altogether appropriate to devote a chapter of this book to these two related subjects. Can man-made machines learn and can they reproduce themselves?
We shall try to show in this chapter that in fact they can learn and can reproduce themselves, and we shall give an account of the technique needed for both these activities. The simpler of these two processes is that of learning, and it is there that the technical development has gone furthest. I shall talk here particularly of the learning of game-playing machines
which enables them to improve the strategy and tactics of their performance by experience. There is an established theory of the playing of games--the von Neumann theory.2 It concerns a policy which is best considered by working from the end of the game rather than from the beginning. In the last move of the game, a player strives to make a winning move if possible, and if not, then at least a drawing move.
His opponent, at the previous stage, strives to make a move which will prevent the other player from making a winning or a drawing move. If he can himself make a winning move at that stage, he will do so, and this will not be the next to the last but the last stage of the game. The other player at the move before this will try to act in such a way that the very best resources of his opponent will not prevent him from ending with a winning move, and so on backward.
There are games such as ticktacktoe where the entire strategy is known, and it is possible to start this policy from the very beginning. When this is feasible, it is manifestly the best way of playing the game. However, in many games like chess and checkers our knowledge is not sufficient to permit a complete strategy of this sort, and then we can only approximate to it.
The von Neumann type of approximate theory tends to lead a player to act with the utmost caution, assuming that his opponent is the perfectly wise sort of a master. This attitude, however, is not always justified. In war, which is a sort of game, this will in general lead to an indecisive action which will often be not much better than a defeat.
Let me give two historical examples. When Napoleon fought the Austrians in Italy, it was part of his effectiveness that he knew the Austrian mode of military thought to be hidebound and traditional, so that he was quite justified in assuming that they were incapable
of taking advantage of the new decision-compelling methods of war which had been developed by the soldiers of the French Revolution. When Nelson fought the combined fleets of continental Europe, he had the advantage of fighting with a naval machine which had kept the seas for years and which had developed methods of thought of which, as he was well aware, his enemies were incapable. If he had not made the fullest possible use of this advantage, instead of acting as cautiously as he would have had to act under the supposition that he was facing an enemy of equal naval experience, he might have won in the long run but could not have won so quickly and decisively as to establish the tight naval blockade which was the ultimate downfall of Napoleon.
Thus, in both cases, the guiding factor was the known record of the commander and of his opponents, as exhibited statistically in the past of their actions, rather than an attempt to play the perfect game against the perfect opponent. Any direct use of the von Neumann method of game theory in these cases would have proved futile. In a similar way, books on chess theory are not written from the von Neumann point of view.
They are compendia of principles drawn from the practical experience of chess players playing against other chess players of high quality and wide knowledge; and they establish certain values or weightings to be given to the loss of each piece, to mobility, to command, to development, and to other factors which may vary with the stage of the game. It is not very difficult to make machines which will play chess of a sort. The mere obedience to the laws of the game, so that only legal moves are made, is easily within the power of quite simple computing machines.
Now comes the question of policy within the rules of the game. Every evaluation of pieces, command, mobility, and so forth, is intrinsically capable of being reduced to numerical terms; and when this is done, the maxims of a chess book may be used for the determination of the best moves of each stage. Such machines have been made; and they will play a very fair amateur chess, although at present not a game of master caliber.
Imagine yourself in the position of playing chess against such a machine. To make the situation fair, let us suppose you are playing correspondence chess without the knowledge that it is such a machine you are playing and without the prejudices that this knowledge may excite. Naturally, as always is the case with chess, you will come to a judgment of your opponent's chess personality.
You will find that when the same situation comes up twice on the chessboard, your opponent's reaction will be the same each time, and you will find that he has a very rigid personality. If any trick of yours will work, then it will always work under the same conditions. It is thus not too hard for an expert to get a line on his machine opponent and to defeat him every time.
However, there are machines that cannot be defeated so trivially. Let us suppose that every few games the machine takes time off and uses its facilities for another purpose. This time, it does not play against an opponent, but examines all the previous games which it has recorded on its memory to determine what weighting of the different evaluations of the worth of pieces, command, mobility, and the like, will conduce most to winning.
In this way, it learns not only from its own failures but its opponent's successes. It now replaces its earlier valuations by the new ones and goes on playing as a new and better machine. Such a machine would no longer have as rigid a personality, and the
tricks which were once successful against it will ultimately fail. More than that, it may absorb in the course of time something of the policy of its opponents. All this is very difficult to do in chess, and as a matter of fact the full development of this technique, so as to give rise to a machine that can play master chess, has not been accomplished.
Checkers offers an easier problem. The homogeneity of the values of the pieces greatly reduces the number of combinations to be considered. Moreover, partly as a consequence of this homogeneity, the checker game is much less divided into distinct stages than the chess game.
Even in checkers, the main problem of the end game is no longer to take pieces but to establish contact with the enemy so that one is in a position to take pieces. Similarly, the valuation of moves in the chess game must be made independently for the different stages. Not only is the end game different from the middle game in the considerations which are paramount, but the openings are much more devoted to getting the pieces into a position of free mobility for attack and defense than is the middle game.
The result is that we cannot be even approximately content with a uniform evaluation of the various weighting factors for the game as a whole, but must divide the learning process into a number of separate stages. Only then can we hope to construct a learning machine which can play master chess. The idea of a first-order programming, which may be linear in certain cases, combined with a second-order programming, which uses a much more extensive segment of the past for the determination of the policy to be carried out in the first-order programming, has been mentioned earlier in this book in connection with the problem of prediction.
prediction of the future by means of a linear operation; but the determination of the correct linear operation is a statistical problem in which the long past of the flight and the past of many similar flights are used to give the basis of the statistics. The statistical studies necessary to use a long past for a determination of the policy to be adopted in view of the short past are highly non-linear. As a matter of fact, in the use of the Wiener-Hopf equation for prediction,3 the determination of the coefficients of this equation is carried out in a non-linear manner.
In general, a learning machine operates by non-linear feedback. The checker-playing machine described by Samuel4 and Watanabe5 can learn to defeat the man that programmed it in a fairly consistent way on the basis of from 10 to 20 operating hours of programming. Watanabe's philosophical ideas on the use of programming machines are very exciting.
On the one hand, he is treating a method of proving an elementary geometrical theorem which shall conform in an optimal way according to certain criteria of elegance and simplicity, as a learning game to be played not against an individual opponent but against what we may call "Colonel Bogey." A similar game which Watanabe is studying is played in logical induction, when we wish to build up a theory which is optimal in a similar quasi-aesthetic way, on the basis of an evaluation of economy, directness, and the like, by the determination of the evaluation of a finite number of parameters which are left free. This, it is true, is only a limited logical induction, but it is well worth studying. Many forms of the activity of struggle, which we do not ordinarily consider as games, have a great deal of light thrown on them by the theory of game-playing machines.
points out in "Rikki-Tikki-Tavi," the mongoose is not immune to the poison of the cobra, although it is to some extent protected by its coat of stiff hairs which makes it difficult for the snake to bite home. As Kipling states, the fight is a dance with death, a struggle of muscular skill and agility. There is no reason to suppose that the individual motions of the mongoose are faster or more accurate than those of the cobra.
Yet the mongoose almost invariably kills the cobra and comes out of the contest unscathed. How is it able to do this? I am here giving an account which appears valid to me, from having seen such a fight, as well as motion pictures of other such fights.
I do not guarantee the correctness of my observations as interpretations. The mongoose begins with a feint, which provokes the snake to strike. The mongoose dodges and makes another such feint, so that we have a rhythmical pattern of activity on the part of the two animals.
However, this dance is not static but develops progressively. As it goes on, the feints of the mongoose come earlier and earlier in phase with respect to the darts of the cobra, until finally the mongoose attacks when the cobra is extended and not in a position to move rapidly. This time the mongoose's attack is not a feint but a deadly accurate bite through the cobra's brain.
In other words, the snake's pattern of action is confined to single darts, each one for itself, while the pattern of the mongoose's action involves an appreciable, if not very long, segment of the whole past of the fight. To this extent the mongoose acts like a learning machine, and the real deadliness of its attack is dependent on a much more highly organized nervous system. As a Walt Disney movie of several years ago showed, something very similar happens when one of our western birds, the road runner, attacks a rattlesnake.
beak and claws, and a mongoose with its teeth, the pattern of activity is very similar. A bullfight is a very fine example of the same thing. For it must be remembered that the bullfight is not a sport but a dance with death, to exhibit the beauty and the interlaced coordinating actions of the bull and the man.
Fairness to the bull has no part in it, and we can leave out from our point of view the preliminary goading and weakening of the bull, which have the purpose of bringing the contest to a level where the interaction of the patterns of the two participants is most highly developed. The skilled bullfighter has a large repertory of possible actions, such as the flaunting of the cape, various dodges and pirouettes, and the like, which are intended to bring the bull into a position in which it has completed its rush and is extended at the precise moment that the bullfighter is ready to plunge the estoque into the bull's heart. What I have said concerning the fight between the mongoose and the cobra, or the toreador and the bull, will also apply to physical contests between man and man.
Consider a duel with the small-sword. It consists of a sequence of feints, parries, and thrusts, with the intention on the part of each participant to bring his opponent's sword out of line to such an extent that he can thrust home without laying himself open to a double encounter. Again, in a championship game of tennis, it is not enough to serve or return the ball perfectly as far as each individual stroke is considered; the strategy is rather to force the opponent into a series of returns which put him progressively in a worse position until there is no way in which he can return the ball safely.
These physical contests and the sort of games which we have supposed the game-playing machine to play both have the same element of learning in terms of experience of the opponent's
habits as well as one's own. What is true of games of physical encounter is also true of contests in which the intellectual element is stronger, such as war and the games which simulate war, by which our staff officers win the elements of their military experience. This is true for classical war both on land and at sea, and is equally true with the new and as yet untried war with atomic weapons.
Some degree of mechanization, parallel to the mechanization of checkers by learning machines, is possible in all these. There is nothing more dangerous to contemplate than World War III. It is worth considering whether part of the danger may not be intrinsic in the unguarded use of learning machines.
Again and again I have heard the statement that learning machines cannot subject us to any new dangers, because we can turn them off when we feel like it. But can we? To turn a machine off effectively, we must be in possession of information as to whether the danger point has come.
The mere fact that we have made the machine does not guarantee that we shall have the proper information to do this. This is already implicit in the statement that the checker-playing machine can defeat the man who has programmed it, and this after a very limited time of working in. Moreover, the very speed of operation of modern digital machines stands in the way of our ability to perceive and think through the indications of danger.
The idea of non-human devices of great power and great ability to carry through a policy, and of their dangers, is nothing new. All that is new is that now we possess effective devices of this kind. In the past, similar possibilities were postulated for the techniques of magic, which forms the theme for so many legends and folk tales.
aspects of the legendary ethics of magic in an earlier book entitled The Human Use of Human Beings.6 I here repeat some of the material which I have discussed there, in order to bring it out more precisely in its new context of learning machines. One of the best-known tales of magic is Goethe's "The Sorcerer's Apprentice." In this, the sorcerer leaves his apprentice and factotum alone with the chore of fetching water. As the boy is lazy and ingenious, he passes the work over to a broom, to which he has uttered the words of magic which he has heard from his master.
The broom obligingly does the work for him and will not stop. The boy is on the verge of being drowned out. He finds that he has not learned, or has forgotten, the second incantation which is to stop the broom.
In desperation, he takes the broomstick, breaks it over his knee, and finds to his consternation that each half of the broom continues to fetch water. Luckily, before he is completely destroyed, the master returns, says the Words of Power to stop the broom, and administers a good scolding to the apprentice. Another story is the Arabian Nights tale of the fisherman and the genie.
The fisherman has dredged up in his net a jug closed with the seal of Solomon. It is one of the vessels in which Solomon has imprisoned the rebellious genie. The genie emerges in a cloud of smoke, and the gigantic figure tells the fisherman that, whereas in his first years of imprisonment he had resolved to reward his rescuer with power and fortune, he has now decided to slay him out of hand.
Luckily for himself, the fisherman finds a way to talk the genie back into the bottle, upon which he casts the jar to the bottom of the ocean. More terrible than either of these two tales is the fable of the monkey's paw, written by W. Jacobs, an English writer of the beginning of the century.
sitting at his table with his wife and a friend, a returned British sergeant-major from India. The sergeant-major shows his hosts an amulet in the form of a dried, wizened monkey's paw. This has been endowed by an Indian holy man, who has wished to show the folly of defying fate, with the power of granting three wishes to each of three people.
The soldier says that he knows nothing of the first two wishes of the first owner, but the last one was for death. He himself, as he tells his friends, was the second owner but will not talk of the horror of his own experiences. He casts the paw into the fire, but his friend retrieves it and wishes to test its powers.
His first is for £200. Shortly thereafter there is a knock at the door, and an official of the company by which his son is employed enters the room. The father learns that his son has been killed in the machinery, but that the company, without recognizing any responsibility or legal obligation, wishes to pay the father the sum of £200 as a solatium.
The grief-stricken father makes his second wish--that his son may return--and when there is another knock at the door and it is opened, something appears which, we are not told in so many words, is the ghost of the son. The final wish is that this ghost should go away. In all these stories the point is that the agencies of magic are literal-minded; and that if we ask for a boon from them, we must ask for what we really want and not for what we think we want.
The new and real agencies of the learning machine are also literal-minded. If we program a machine for winning a war, we must think well what we mean by winning. A learning machine must be programmed by experience.
The only experience of a nuclear war which is not immediately catastrophic is the experience of a war game. If we are to use this experience as a guide for our procedure in a real emergency, the values of winning which
we have employed in the programming games must be the same values which we hold at heart in the actual outcome of a war. We can fail in this only at our immediate, utter, and irretrievable peril. We cannot expect the machine to follow us in those prejudices and emotional compromises by which we enable ourselves to call destruction by the name of victory.
If we ask for victory and do not know what we mean by it, we shall find the ghost knocking at our door. So much for learning machines. Now let me say a word or two about self-propagating machines.
Here both the words machine and self-propagating are important. The machine is not only a form of matter, but an agency for accomplishing certain definite purposes. And self-propagation is not merely the creation of a tangible replica; it is the creation of a replica capable of the same functions.
Here, two different points of view come into evidence. One of these is purely combinatorial and concerns the question whether a machine can have enough parts and sufficiently complicated structure to enable self-reproduction to be among its functions. This question has been answered in the affirmative by the late John von Neumann.
The other question concerns an actual operative procedure for building self-reproducing machines. Here I shall confine my attentions to a class of machines which, while it does not embrace all machines, is of great generality. I refer to the non-linear transducer.
Such machines are apparatuses which have as an input a single function of time and which have as their output another function of time. The output is completely determined by the past of the input; but in general, the adding of inputs does not add the corresponding outputs. Such pieces of apparatus are known as transducers.
or non-linear, is an invariance with respect to a translation in time. If a machine performs a certain function, then, if the input is shifted back in time, the output is shifted back by the same amount. Basic to our theory of self-reproducing machines is a canonical form of the representation of non-linear transducers.
Here the notions of impedance and admittance, which are so essential in the theory of linear apparatus, are not fully appropriate. We shall have to refer to certain newer methods of carrying out this representation, methods developed partly by me7 and partly by Professor Dennis Gabor8 of the University of London. While both Professor Gabor's methods and my own lead to the construction of non-linear transducers, they are linear to the extent that the non-linear transducer is represented with an output which is the sum of the outputs of a set of non-linear transducers with the same input.
These outputs are combined with varying linear coefficients. This allows us to employ the theory of linear developments in the design and specification of the non-linear transducer. And in particular, this method allows us to obtain coefficients of the constituent elements by a least-square process.
If we join this to a method of statistically averaging over the set of all inputs to our apparatus, we have essentially a branch of the theory of orthogonal development. Such a statistical basis of the theory of non-linear transducers can be obtained from an actual study of the past statistics of the inputs used in each particular case. This is a rough account of Professor Gabor's methods.
While mine are essentially similar, the statistical basis for my work is slightly different. It is well known that electrical currents are not conducted continuously but by a stream of electrons which must have
statistical variations from uniformity. These statistical fluctuations can be represented fairly by the theory of the Brownian motion, or by the similar theory of shot effect or tube noise, about which I am going to say something in the next chapter. At any rate, apparatus can be made to generate a standardized shot effect with highly specific statistical distribution, and such apparatus is being manufactured commercially.
Note that tube noise is in a sense a universal input in that its fluctuations over a sufficiently long time will sooner or later approximate to any given curve. This tube noise possesses a very simple theory of integration and averaging. In terms of the statistics of tube noise, we can easily determine a closed set of normal and orthogonal non-linear operations.
If the inputs subject to these operations have the statistical distribution appropriate to tube noise, the average product of the output of two component pieces of our apparatus, where this average is taken with respect to the statistical distribution of tube noise, will be zero. Moreover, the mean square output of each apparatus can be normalized to one. The result is that the development of the general non-linear transducer in terms of these components results from an application of the familiar theory of orthonormal functions.
To be specific, our individual pieces of apparatus give outputs which are products of Hermite polynomials in the Laguerre coefficients of the past of the input. This is presented in detail in my Nonlinear Problems in Random Theory. It is of course difficult to average in the first instance over a set of possible inputs.
What makes this difficult task realizable is that the shot-effect inputs possess the property known as metric transitivity, or the ergodic property. Any integrable function of the parameter of distribution of shot-effect inputs has in
almost every instance a time average equal to its average over the ensemble. This permits us to take two pieces of apparatus with a common shot-effect input, and to determine the average of their product over the entire ensemble of the possible inputs, by taking their product and averaging it over the time. The repertory of operations needed for all these processes involves nothing more than the addition of potentials, the multiplication of potentials, and the operation of averaging over time.
Devices exist for all these. As a matter of fact, the elementary devices needed in Professor Gabor's methodology are the same as those needed in mine. One of his students has invented a particularly effective and inexpensive multiplying device depending on the piezoelectric effect on a crystal of the attraction of two magnetic coils.
What this amounts to is that we can imitate any unknown non-linear transducer by a sum of linear terms, each of fixed characteristics and with an adjustable coefficient. This coefficient can be determined as the average product of the outputs of the unknown transducer and a particular known transducer, when the same shot-effect generator is connected to the input of both. What is more, instead of computing this result on the scale of an instrument and then transferring it by hand to the appropriate transducer, thus producing a piecemeal simulation of the apparatus, there is no particular problem in automatically effecting the transfer of the coefficients to the pieces of feedback apparatus.
What we have succeeded in doing is to make a white box which can potentially assume the characteristics of any non-linear transducer whatever, and then to draw it into the similitude of a given black-box transducer by subjecting the two to the same random input and connecting the outputs of the
structures in the proper manner, so as to arrive at the suitable combination without any intervention on our part. I ask if this is philosophically very different from what is done when a gene acts as a template to form other molecules of the same gene from an indeterminate mixture of amino and nucleic acids, or when a virus guides into its own form other molecules of the same virus out of the tissues and juices of its host. I do not in the least claim that the details of these processes are the same, but I do claim that they are philosophically very similar phenomena.
在这种装置中,磁带以调频形式使用。原因是读取磁带总是涉及一定量的擦除。
X Brain Waves and Self-Organizing Systems
Notes
Notes
Notes
Foreword by Doug Hill 1. John Pfeiffer, "Man's Brain Child," New York Times, April 18, 1965, https://tinyurl.com/y9xccjpu. 2. Kai-Fu Lee, "Tech Companies Should Stop Pretending AI Won't Destroy Jobs," MIT Technology Review, February 21, 2018, https://tinyurl .com/ya7spngu.
Gabor, D., "Electronic Inventions and Their Impact on Civilization," Inaugural Lecture, March 3, 1959, Imperial College of Science and Technology, University of London, England. 7. Wiener, N., and P. Masani, "The Prediction Theory of Multivariate Stochastic Processes," Part I, Acta Mathematica, 98, 111-150 (1957); Part II, ibid., 99, 93-137 (1958).
In September 2018, Lee published AI Superpowers: China, Silicon Valley, and the New World Order. 3. See Matthew Desmond, "Americans Want to Believe Jobs Are the Solution to Poverty. They're Not." New York Times Magazine, September 11, 2018, https://tinyurl.com/y7ghzuqp; Eduardo Porter, "Tech Is Splitting the U.S.
Also Wiener, N., and E. Akutowicz, "The Definition and Ergodic Properties of the Stochastic Adjoint of a Unitary Transformation," Rendiconti del Circolo Matematico di Palermo, Ser. II, VI, 205-217 (1957). 8.
Workforce in Two," New York Times, February 4, 2019, https://nyti.ms/2DSI2KZ. 4. Amy Bernstein and Anand Raman, "The Great Decoupling: An Interview with Erik Byrnjolfsson and Andrew McAfee," Harvard Business Review, June 2015, https://hbr.org/2015/06/the-great-decoupling 5. Norbert Wiener, "Some Moral and Technical Consequences of Automation," Science, New Series 131, no. 3410 (May 6, 1960), 1355. 6.
Samuel, A. L., "Some Studies in Machine Learning, Using the Game of Checkers," IBM Journal of Research and Development, 3, 210-229 (1959). 9. Watanabe, S., "Information Theoretical Analysis of Multivariate Correlation," IBM Journal of Research and Development, 4, 66-82 (1960). 10.
On the 1965 blackout, see James Burke, Connections (Boston: Little, Brown, 1978), 1–2; David E. Sanger, "Russian Hackers Appear to Shift Focus to U.S. Power Grid," New York Times, July 27, 2018, https:// tinyurl.com/y94f4egl.
Stanley-Jones, D., and K. Stanley-Jones, Kybernetics of Natural Systems, A Study in Patterns of Control, Pergamon Press, London, 1960. 导言 1.
Notes
Nelson D. Schwartz and Louise Story, "High-Speed Trading Glitch Costs Investors Billions," New York Times, May 6, 2010, http://nyti.ms/ aAzXcc; Thomas Heath, "The Warning From JPMorgan about Flash Crashes Ahead," Washington Post, September 5, 2018, https://tinyurl .com/ychvmcve. 8. Norbert Wiener, The Human Use of Human Beings, Houghton Mifflin, 1950, 37. 9.
MacColl, L. A., Fundamental Theory of Servomechanisms, Van Nostrand, New York, 1946. 2. Rosenblueth, A., N. Wiener, and J. Bigelow, "Behavior, Purpose, and Teleology," Philosophy of Science, 10, 18-24 (1943). 3. Kolmogoroff, A.
Flo Conway and Jim Siegelman, Dark Hero of the Information Age: In Search of Nobert Wiener, the Father of Cybernetics (New York: Basic Books, 2004), 4. 10. See the joint edition of his two-volume autobiography, Norbert Wiener: A Life in Cybernetics (Cambridge, MA: MIT Press, 2017), 55, 57. Ronald R.
N., "Interpolation und Extrapolation von stationären Zufälligen Folgen," Bull. U.S.S.R., Ser. Math. 5, 3-14 (1941).
Kline's introduction to that edition mentions the original title of Ex-Prodigy on p. xii. 11. Conway and Siegelman, Dark Hero, p. 5. 12. Wiener, A Life in Cybernetics, 284. 13.
Schrödinger, Erwin, What is Life?, Cambridge University Press, Cambridge, England, 1945. 5. Maxwell, J. C., Proc. Soc. (London), 16, 270-283 (1868). 6.
Steve J. Heims, John von Neumann and Norbert Wiener: From Mathematics to the Technologies of Life and Death (Cambridge, MA: MIT Press, 1980), 140. 14. Katherine Hayles, How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics (Chicago: University of Chicago Press, 1999), 18. 15.
Turing, A. M., "On Computable Numbers, with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society, Ser. 2, 42, 230-265 (1936). 7. McCulloch, W.
The conviction that Wiener's ideas have become increasingly relevant in the digital era inspired the IEEE to sponsor two "Reintroducing Norbert Wiener in the 21st Century" conferences, one in 2014 and the second in 2016. See http://21stcenturywiener.org. 16. Conway and Siegelman, Dark Hero, 243. 17.
S., and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," Bull. Biophys, 5, 115-133 (1943). 8.
Drew Harwell, "Google to Drop Pentagon AI Contract after Employee Objections to the 'Business of War,'" Washington Post, June 1, 2018, https://tinyurl.com/y75e6bn9.
Levinson, N., J. Math. and Physics, 25, 261-278; 26, 110-119 (1947). 9. Lee, Y.
Notes
Wiener, Human Use, 17. 19. Conway and Siegelman, Dark Hero, 93–98, 217–230; Heims, John von Neumann and Norbert Wiener, 379–389. 20. Heims, 377. 21.
W., J. Math. and Physics, 11, 261-278 (1932). 10. Wiener, N., Extrapolation, Interpolation, and Smoothing of Stationary Time Series, Technology Press and Wiley, New York, 1949. 11.
Conway and Siegelman, Dark Hero, 16; Wiener, A Life in Cybernetics, 459–460; Heims, 155. 22. See, for example, David Streitfeld, "Tech Giants, Once Seen as Saviors, Are Now Viewed as Threats," New York Times, October 12, 2017, https://tinyurl.com/y8y4qdl2; Noam Cohen, "Silicon Valley Is Not Your Friend," New York Times, October 13, 2017, https://tinyurl.com/ y9omhyoe; Jamie Doward, "The Big Tech Backlash," The Guardian, January 27, 2018, https://tinyurl.com/y77vku5j; and Noah Kulwin, "The Internet Apologizes," New York Magazine, April 13, 2018, https://tinyurl .com/ybtml5d6. Foreword by Sanjoy K.
Wiener, N., and A. Rosenblueth, "The Mathematical Formulation of the Problem of Conduction of Impulses in a Network of Connected Excitable Elements, Specifically in Cardiac Muscle," Arch. Cardiol. Méx., 16, 205-265 (1946). 12. 来自墨西哥国立心脏病学研究所关于阵挛的未发表文章。13. Fortune, 32, 139-147 (October); 163-169 (November, 1945).
Mitter 1. Claude E. Shannon, "A Mathematical Theory of Communication," Bell System Technical Journal 27, no. 3 (1948): 379–423, https://doi.org/ 10.1002/j.1538-7305.1948.tb01338.x. 2.
第二章 1. Oxtoby, J. C., and S. M. Ulam, "Measure-Preserving Homeomorphisms and Metrical Transitivity." Ann. of Math., Ser. 2, 42, 874-920 (1941). 2. 然而,奥斯古德的某些早期工作代表了向勒贝格积分方向的重要一步。3. Hopf, E., "Ergodentheorie," Ergeb. Math., 5, No. 2, Springer, Berlin (1937).
Shockley, "The Theory of p-n Junctions in Semiconductors and p-n Junction Transistors," Bell System Technical Journal 28 (1949): 435–489, https://doi.org/10.1002/j.1538-7305.1949.tb03645.x. 3. Turing, "On Computable Numbers, with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society S2-42 (January 1937): 230–265, https://doi.org/10.1112/plms/ s2-42.1.230; John von Neumann, The Computer and the Brain (New Haven, CT: Yale University Press, 1958). 4. Norbert Wiener, Cybernetics, or Control and Communication in the Animal and the Machine (Cambridge, MA: MIT Press, 1948), 42. 5.
Wiener, N., The Fourier Integral and Certain of Its Applications, The University Press, Cambridge, England, 1933; Dover Publications, Inc., N.Y. 5. Haar, H., "Der Massbegriff in der Theorie der Kontinuierlichen Gruppen." Ann. of Math., Ser. 2, 34, 147-169 (1933). 第四章 1.
Wiener, Cybernetics, 162.
此处作者利用了J. von Neumann的个人通信。2. Paley, R. C., and N. Wiener, "Fourier Transforms in the Complex Domain," Colloquium Publications, Vol. 19, American Mathematical Society, New York, 1934, Chapter 10. 3. Stieltjes, T. Annales de la Fac. des Sc. de Toulouse, 1894, p. 165; Lebesgue, H., Leçons sur l'Intégration, Gauthier-Villars et Cie, Paris, 1928. 4.
Notes
See, for example, Mark Davis and Alison Etheridge, Louis Bachelier's Theory of Speculation: The Origins of Modern Finance (Princeton, NJ: Princeton University Press, 2006). 7. See Norbert Wiener, Extrapolation, Interpolation, and Smoothing of Time Series, with Engineering Applications (Cambridge, MA: MIT Press, 1949). 8. Wiener, Cybernetics, 10, 11. 9.
See R. Liptser and A. Shiryaev, "Computation of Mutual Information and Channel Capacity of a Gaussian Channel with Feedback," in Statistics of Random Processes, vol. 2 (Berlin: Springer, 1978), 190–194. 10.
见R. Fisher和J. von Neumann的著作。1. Poincaré, H., Les Méthodes Nouvelles de la Mécanique Céleste, Gauthier-Villars et fils, Paris, 1892-1899. 2.
Feller, An Introduction to Probability Theory and Its Applications, vol. 2 (New York: Wiley, 1971). 11. Kalman and R. Bucy, "New Results in Linear Filtering and Predication Theory," Journal of Basic Engineering 83 (1961): 95–108, https://doi.org/10.1115/1.3658902. 12.
Cannon, W., The Wisdom of the Body, W. Norton & Company, Inc., New York, 1932; Henderson, L. J., The Fitness of the Environment, The Macmillan Company, New York, 1913.
For an information theoretic interpretation of the Kalman–Bucy filter, see S. Mitter and N. Newton, "Information and Entropy Flow in the Kalman–Bucy Filter," Journal of Statistical Physics 118, nos. 1–2 (2005): 145–176, https://doi.org/10.1007/s10955-004-8781-9; and S.
第五章 1. Journal of the Franklin Institute, various papers, 1930 on. 2. Turing, A. M., "On Computable Numbers with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society, Ser. 2, 42, 230-265 (1936).
Mitter, "Filtering and Stochastic Control: A Historical Perspective," IEEE Control Systems 16 (1996): 67–76, https://doi.org/10.1109/37.506400. 13. Mitter, "Filtering and Stochastic Control." 14. Wiener's suggestions have been further developed by G.
第六章 1. 英国布里斯托尔的W. Grey Walter博士的个人通信。
Newton, L. Gould, and J. Kaiser in Analytical Design of Linear Feedback Controls (New York: Wiley, 1957). 15.
第七章 1. Thompson, D'Arcy, On Growth and Form, Amer. ed., The Macmillan Company, New York, 1942. 第九章 1.
See Mitter, "Filtering and Stochastic Control." 16. Wiener, Cybernetics, 113. 17. Wiener, 112. 18.
Huxley, J., Evolution: The Modern Synthesis, Harper Bros., New York, 1943. 2. von Neumann, J., and O. Morgenstern, Theory of Games and Economic Behavior, Princeton University Press, Princeton, N.J., 1944. 3. Wiener, N., Extrapolation, Interpolation, and Smoothing of Stationary Time Series with Engineering Applications, The Technology Press of M.I.T. and John Wiley & Sons, New York, 1949. 4.
Wiener, 57, 58.
Samuel, A. L., "Some Studies in Machine Learning, Using the Game of Checkers," IBM Journal of Research and Development, 3, 210-229 (1959). 5. Watanabe, S., "Information Theoretical Analysis of Multivariate Correlation," IBM Journal of Research and Development, 4, 66-82 (1960). 6.
Notes
Sandberg, J.-C. Delvenne, N. Newton, and S.
Wiener, N., The Human Use of Human Beings; Cybernetics and Society, Houghton Mifflin Company, Boston, 1950. 7. Wiener, N., Nonlinear Problems in Random Theory, The Technology Press of M.I.T. and John Wiley & Sons, Inc., New York, 1958. 8. Gabor, D., "Electronic Inventions and Their Impact on Civilization," Inaugural Lecture, March 3, 1959, Imperial College of Science and Technology, University of London, England.
Mitter, "Maximum Work Extraction and Implementation Costs for Nonequilibrium Maxwell's Demon," Physical Review E 90 (2014): 042119, https://doi.org/ 10.1103/PhysRevE.90.042119. 20. Mitter, "Filtering and Stochastic Control." 21. Wiener, Nonlinear Problems in Random Theory (Cambridge, MA: MIT Press, 1966). 22.
第十章 1. Taylor, G. I., "Diffusion by Continuous Movements," Proceedings of the London Mathematical Society, Ser. 2, 20, 196-212 (1921-1922). 2.
Ito, "Multiple Wiener Integrals," Journal of the Mathematical Society of Japan 3 (1951): 157–169, https://doi.org/10.2969/jmsj/00310157. 23. Wiener, Cybernetics, 178–180. 24. Wiener, "Generalized Harmonic Analysis," Acta Mathematica 55 (1930): 117–258; Wiener, Nonlinear Problems in Random Theory. 25.
Barlow, J. S., and R. Brown, An Analog Correlator System for Brain Potentials, Technical Report 300, Research Laboratory of Electronics, M.I.T., Cambridge, Mass. (1955). 3.
Wiener, 192. Preface to the Second Edition 1. Wiener, N., The Human Use of Human Beings; Cybernetics and Society, Houghton Mifflin Company, Boston, 1950. 2.
Wiener, N., Nonlinear Problems in Random Theory, The Technology Press of M.I.T. and John Wiley & Sons, Inc., New York, 1958. 3. Here I am using the term "non-linear system" not to exclude linear systems but to include a larger category of systems. The analysis of non-linear systems by means of random noise is also applicable to linear systems and has been so used. 4.
Wiener, N., "Generalized Harmonic Analysis," Acta Mathematica, 55, 117-258 (1930); Nonlinear Problems in Random Theory, The Technology Press of M.I.T. and John Wiley & Sons, Inc., New York, 1958. 6. Wiener, N., "The Ergodic Theorem," Duke Mathematical Journal, 5, 1-39 (1939); 亦见Modern Mathematics for the Engineer, E. Beckenbach (Ed.), McGraw-Hill, New York, 1956, pp. 166-168. 7.
The terms "black box" and "white box" are convenient and figurative expressions of not very well-determined usage. I shall understand by a black box a piece of apparatus, such as four-terminal networks with two input and two output terminals, which performs a definite operation on the present and past of the input potential, but for which we do not necessarily have any information of the structure by which this operation is performed. On the other hand, a white box will be a
Wiener, N., "Plancherel's Theorem," The Fourier Integral and Certain of Its Applications, The University Press, Cambridge, England, 1933, pp. 46-71; Dover Publications, Inc., New York. 8. 这是对实际情况的简化描述,尤其是在皮层中,因为神经元的全或无操作取决于它们是否具有足够的长度,以使神经元本身中传入脉冲形式的重建接近渐近形式。然而,例如在皮层中,由于神经元较短,同步的必要性仍然存在,尽管过程的细节要复杂得多。9.
Notes
similar network in which we have built in the relation between input and output potentials in accordance with a definite structural plan for securing a previously determined input-output relation. 5. Bose, A. G., "Nonlinear System Characterization and Optimization," IRE Transactions on Information Theory, IT–5, 30–40 (1959) (Special supplement to IRE Transactions). 6.
Gabor, D., "Electronic Inventions and Their Impact on Civilization," Inaugural Lecture, March 3, 1959, Imperial College of Science and Technology, University of London, England. 7. Wiener, N., and P. Masani, "The Prediction Theory of Multivariate Stochastic Processes," Part I, Acta Mathematica, 98, 111–150 (1957); Part II, ibid., 99, 93–137 (1958).
他所指的现象在于,在他的脑波拓扑图中,当从中心向外时,指示频率的射线被限制在相对狭窄的扇形中。10. Barlow, J. S., "Rhythmic Activity Induced by Photic Stimulation in Relation to Intrinsic Alpha Activity of the Brain in Man," EEG Clin. Neurophysiol., 12, 317-326 (1960).
Also Wiener, N., and E. Akutowicz, "The Definition and Ergodic Properties of the Stochastic Adjoint of a Unitary Transformation," Rendiconti del Circolo Matematico di Palermo, Ser. II, VI, 205–217 (1957). 8.
11. Cold Spring Harbor Symposium on Quantitative Biology, Volume XXV (Biological Clocks), The Biological Laboratory, Cold Spring Harbor, L.I., N.Y., 1960.
Samuel, A. L., "Some Studies in Machine Learning, Using the Game of Checkers," IBM Journal of Research and Development, 3, 210–229 (1959). 9. Watanabe, S., "Information Theoretical Analysis of Multivariate Correlation," IBM Journal of Research and Development, 4, 66–82 (1960). 10.
Samuel, A. L., "Some Studies in Machine Learning, Using the Game of Checkers," IBM Journal of Research and Development, 3, 210-229 (1959).
Stanley-Jones, D., and K. Stanley-Jones, Kybernetics of Natural Systems, A Study in Patterns of Control, Pergamon Press, London, 1960. Introduction 1.
Stanley-Jones, D., and K. Stanley-Jones, Kybernetics of Natural Systems, A Study in Patterns of Control, Pergamon Press, London, 1960. 导言
MacColl, L. A., Fundamental Theory of Servomechanisms, Van Nostrand, New York, 1946. 2. Rosenblueth, A., N.
MacColl, L. A., Fundamental Theory of Servomechanisms, Van Nostrand, New York, 1946. 2. Rosenblueth, A., N.
Wiener, and J. Bigelow, "Behavior, Purpose, and Teleology," Philosophy of Science, 10, 18–24 (1943). 3. Kolmogoroff, A.
Wiener, and J. Bigelow, "Behavior, Purpose, and Teleology," Philosophy of Science, 10, 18-24 (1943). 3. Kolmogoroff, A.
N., "Interpolation und Extrapolation von stationären Zufälligen Folgen," Bull. U.S.S.R., Ser. Math. 5, 3–14 (1941).
N., "Interpolation und Extrapolation von stationaren Zufalligen Folgen," Bull. U.S.S.R., Ser. Math. 5, 3-14 (1941).
Notes
Schrödinger, Erwin, What is Life?, Cambridge University Press, Cambridge, England, 1945. 5. Maxwell, J, C., Proc. Soc. (London), 16, 270–283, (1868). 6.
Schrodinger, Erwin, What is Life?, Cambridge University Press, Cambridge, England, 1945. 5. Maxwell, J. C., Proc. Soc. (London), 16, 270-283 (1868). 6.
Turing, A. M., "On Computable Numbers, with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society, Ser. 2, 42, 230–265 (1936). 7. McCulloch, W.
Turing, A. M., "On Computable Numbers, with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society, Ser. 2, 42, 230-265 (1936). 7. McCulloch, W.
S., and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," Bull. Biophys, 5, 115–133 (1943). 8.
S., and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," Bull. Biophys, 5, 115-133 (1943). 8.
Levinson, N., J. Math. and Physics, 25, 261–278; 26, 110–119 (1947). 9. Lee, Y.
Levinson, N., J. Math. and Physics, 25, 261-278; 26, 110-119 (1947). 9. Lee, Y.
W., J. Math. and Physics, 11, 261–278 (1932). 10. Wiener, N., Extrapolation, Interpolation, and Smoothing of Stationary Time Series, Technology Press and Wiley, New York, 1949. 11.
W., J. Math. and Physics, 11, 261-278 (1932). 10. Wiener, N., Extrapolation, Interpolation, and Smoothing of Stationary Time Series, Technology Press and Wiley, New York, 1949. 11.
Wiener, N., and A. Rosenblueth, "The Mathematical Formulation of the Problem of Conduction of Impulses in a Network of Connected Excitable Elements, Specifically in Cardiac Muscle," Arch. Cardiol.
Wiener, N., and A. Rosenblueth, "The Mathematical Formulation of the Problem of Conduction of Impulses in a Network of Connected Excitable Elements, Specifically in Cardiac Muscle," Arch. Cardiol. Mex., 16, 205-265 (1946). 12. 来自墨西哥国立心脏病学研究所关于阵挛的未发表文章。13. Fortune, 32, 139-147 (October); 163-169 (November, 1945).
Méx., 16, 205–265 (1946). 12. Unpublished articles on clonus from the Instituto Nacional de Cardiología, Mexico. 13. Fortune, 32, 139–147 (October); 163–169 (November, 1945).
第二章 1. Oxtoby, J. C., and S. M. Ulam, "Measure-Preserving Homeomorphisms and Metrical Transitivity." Ann. of Math., Ser. 2, 42, 874-920 (1941). 2. 然而,奥斯古德的某些早期工作代表了向勒贝格积分方向的重要一步。3. Hopf, E., "Ergodentheorie," Ergeb.
Chapter II 1. Oxtoby, J. C., and S.
Math., 5, No. 2, Springer, Berlin (1937).
Ulam, "Measure-Preserving Homeomorphisms and Metrical Transitivity." Ann. of Math., Ser. 2, 42, 874–920 (1941). 2. Nevertheless some of Osgood's early work represented an important step in the direction of the Lebesgue integral. 3. Hopf, E., "Ergodentheorie," Ergeb.
Ulam, "Measure-Preserving Homeomorphisms and Metrical Transitivity." Ann. of Math., Ser. 2, 42, 874-920 (1941). 2. 然而,奥斯古德的某些早期工作代表了向勒贝格积分方向的重要一步。3. Hopf, E., "Ergodentheorie," Ergeb. Math., 5, No. 2, Springer, Berlin (1937).
Math., 5, No. 2, Springer, Berlin (1937).
Math., 5, No. 2, Springer, Berlin (1937).
Notes
Wiener, N., The Fourier Integral and Certain of Its Applications, The University Press, Cambridge, England, 1933; Dover Publications, Inc., N.Y. 5. Haar, H., "Der Massbegriff in der Theorie der Kontinuierlichen Gruppen." Ann. of Math., Ser. 2, 34, 147–169 (1933). Chapter IV 1.
Wiener, N., The Fourier Integral and Certain of Its Applications, The University Press, Cambridge, England, 1933; Dover Publications, Inc., N.Y. 5. Haar, H., "Der Massbegriff in der Theorie der Kontinuierlichen Gruppen." Ann. of Math., Ser. 2, 34, 147-169 (1933). 第四章 1.
Here the author makes use of a personal communication of J. von Neumann. 2. Paley, R. C., and N.
此处作者利用了J. von Neumann的个人通信。2. Paley, R. C., and N. Wiener, "Fourier Transforms in the Complex Domain," Colloquium Publications, Vol. 19, American Mathematical Society, New York, 1934, Chapter 10. 3. Stieltjes, T. Annales de la Fac. des Sc. de Toulouse, 1894, p. 165; Lebesgue, H., Lecons sur l'Integration, Gauthier-Villars et Cie, Paris, 1928. 4.
Wiener, "Fourier Transforms in the Complex Domain," Colloquium Publications, Vol. 19, American Mathematical Society, New York, 1934, Chapter 10. 3. Stieltjes, T. Annales de la Fae. des Sc. de Toulouse, 1894, p. 165; Lebesgue, H., Leçons sur l'Intégration, Gauthier-Villars et Cie, Paris, 1928. 4.
Wiener, "Fourier Transforms in the Complex Domain," Colloquium Publications, Vol. 19, American Mathematical Society, New York, 1934, Chapter 10. 3. Stieltjes, T. Annales de la Fac. des Sc. de Toulouse, 1894, p. 165; Lebesgue, H., Lecons sur l'Integration, Gauthier-Villars et Cie, Paris, 1928. 4.
This is the mixing property of Koopman, which is the necessary and sufficient ergodic assumption to justify statistical mechanics. 5. We refer especially to recent papers by Dr. Lee. 6.
See writings of R. Fisher and J. von Neumann. 1. Poincaré, H., Les Méthodes Nouvelles de la Mécanique Céleste, Gauthier-Villars et fils, Paris, 1892–1899. 2.
见R. Fisher和J. von Neumann的著作。1. Poincare, H., Les Methodes Nouvelles de la Mecanique Celeste, Gauthier-Villars et fils, Paris, 1892-1899. 2.
Cannon, W., The Wisdom of the Body, W. Norton & Company, Inc., New York, 1932; Henderson, L. J., The Fitness of the Environment, The Macmillan Company, New York, 1913.
Cannon, W., The Wisdom of the Body, W. Norton & Company, Inc., New York, 1932; Henderson, L. J., The Fitness of the Environment, The Macmillan Company, New York, 1913.
Chapter V 1. Journal of the Franklin Institute, various papers, 1930 on. 2. Turing, A.
第五章 1. Journal of the Franklin Institute, various papers, 1930 on. 2. Turing, A.
M., "On Computable Numbers with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society, Ser. 2, 42, 230–265 (1936).
M., "On Computable Numbers with an Application to the Entscheidungsproblem," Proceedings of the London Mathematical Society, Ser. 2, 42, 230-265 (1936).
Notes
Chapter VI 1. Personal communication of Dr. Grey Walter, of Bristol, England.
第六章 1. 英国布里斯托尔的W. Grey Walter博士的个人通信。
Chapter VII 1. Thompson, D'Arcy, On Growth and Form, Amer. ed., The Macmillan Company, New York, 1942. Chapter IX 1.
第七章 1. Thompson, D'Arcy, On Growth and Form, Amer. ed., The Macmillan Company, New York, 1942. 第九章 1.
Huxley, J., Evolution: The Modern Synthesis, Harper Bros., New York, 1943. 2. von Neumann, J., and O. Morgenstern, Theory of Games and Economic Behavior, Princeton University Press, Princeton, N.J., 1944. 3. Wiener, N., Extrapolation, Interpolation, and Smoothing of Stationary Time Series with Engineering Applications, The Technology Press of M.I.T. and John Wiley & Sons, New York, 1949. 4.
Huxley, J., Evolution: The Modern Synthesis, Harper Bros., New York, 1943. 2. von Neumann, J., and O. Morgenstern, Theory of Games and Economic Behavior, Princeton University Press, Princeton, N.J., 1944. 3. Wiener, N., Extrapolation, Interpolation, and Smoothing of Stationary Time Series with Engineering Applications, The Technology Press of M.I.T. and John Wiley & Sons, New York, 1949. 4.
Samuel, A. L., "Some Studies in Machine Learning, Using the Game of Checkers," IBM Journal of Research and Development, 3, 210–229 (1959). 5. Watanabe, S., "Information Theoretical Analysis of Multivariate Correlation," IBM Journal of Research and Development, 4, 66–82 (1960). 6.
Samuel, A. L., "Some Studies in Machine Learning, Using the Game of Checkers," IBM Journal of Research and Development, 3, 210-229 (1959). 5. Watanabe, S., "Information Theoretical Analysis of Multivariate Correlation," IBM Journal of Research and Development, 4, 66-82 (1960). 6.
Wiener, N., The Human Use of Human Beings; Cybernetics and Society, Houghton Mifflin Company, Boston, 1950. 7. Wiener, N., Nonlinear Problems in Random Theory, The Technology Press of M.I.T. and John Wiley & Sons, Inc., New York, 1958. 8. Gabor, D., "Electronic Inventions and Their Impact on Civilization," Inaugural Lecture, March 3, 1959, Imperial College of Science and Technology, University of London, England.
Wiener, N., The Human Use of Human Beings; Cybernetics and Society, Houghton Mifflin Company, Boston, 1950. 7. Wiener, N., Nonlinear Problems in Random Theory, The Technology Press of M.I.T. and John Wiley & Sons, Inc., New York, 1958. 8. Gabor, D., "Electronic Inventions and Their Impact on Civilization," Inaugural Lecture, March 3, 1959, Imperial College of Science and Technology, University of London, England.
Notes
Chapter X 1. Taylor, G. I., "Diffusion by Continuous Movements," Proceedings of the London Mathematical Society, Ser. 2, 20, 196–212 (1921–1922). 2.
第十章 1. Taylor, G. I., "Diffusion by Continuous Movements," Proceedings of the London Mathematical Society, Ser. 2, 20, 196-212 (1921-1922). 2.
Barlow, J. S., and R. Brown, An Analog Correlator System for Brain Potentials, Technical Report 300, Research Laboratory of Electronics, M.I.T., Cambridge, Mass. (1955). 3.
Barlow, J. S., and R. Brown, An Analog Correlator System for Brain Potentials, Technical Report 300, Research Laboratory of Electronics, M.I.T., Cambridge, Mass. (1955). 3.
This work was undertaken with the cooperation of the Neurophysiology Laboratory of the Massachusetts General Hospital and the Communications Biophysics Laboratory of M.I.T. 4. The IBM-709 at the M.I.T. Computation Center was used. 5.
Wiener, N., "Generalized Harmonic Analysis," Acta Mathematica, 55, 117–258 (1930); Nonlinear Problems in Random Theory, The Technology Press of M.I.T. and John Wiley & Sons, Inc., New York, 1958. 6. Wiener, N., "The Ergodic Theorem," Duke Mathematical Journal, 5, 1–39 (1939); also in Modern Mathematics for the Engineer, E. Beckenbach (Ed.), McGraw-Hill, New York, 1956, pp. 166–168. 7.
Wiener, N., "Generalized Harmonic Analysis," Acta Mathematica, 55, 117-258 (1930); Nonlinear Problems in Random Theory, The Technology Press of M.I.T. and John Wiley & Sons, Inc., New York, 1958. 6. Wiener, N., "The Ergodic Theorem," Duke Mathematical Journal, 5, 1-39 (1939); 亦见Modern Mathematics for the Engineer, E. Beckenbach (Ed.), McGraw-Hill, New York, 1956, pp. 166-168. 7.
Wiener, N., "Plancherel's Theorem," The Fourier Integral and Certain of Its Applications, The University Press, Cambridge, England, 1933, pp. 46–71; Dover Publications, Inc., New York. 8. This is a simplified picture of what happens, especially in the cortex, since the all-or-none operation of neurons depends on their being of a sufficient length so that the remaking of the form of the incoming impulses in the neuron itself approaches an asymptotic form. However, in the cortex for example, owing to the shortness of the neurons, the necessity of synchronization still exists, although the details of the process are much more complicated. 9.
Wiener, N., "Plancherel's Theorem," The Fourier Integral and Certain of Its Applications, The University Press, Cambridge, England, 1933, pp. 46-71; Dover Publications, Inc., New York. 8. 这是对实际情况的简化描述,尤其是在皮层中,因为神经元的全或无操作取决于它们是否具有足够的长度,以使神经元本身中传入脉冲形式的重建接近渐近形式。然而,例如在皮层中,由于神经元较短,同步的必要性仍然存在,尽管过程的细节要复杂得多。9.
I must say that some evidence of the existence of narrow central rhythms has been obtained by Dr. Grey Walter of the Burden Neurological Institute in Bristol, England. I am not acquainted with the full details of his methodology; however, I understand that the phenomenon
to which he refers consists in the fact that in his toposcopic pictures of brain waves, as one goes out from the center, the rays indicating the frequency are confined to relatively narrow sectors. 10. Barlow, J. S., "Rhythmic Activity Induced by Photic Stimulation in Relation to Intrinsic Alpha Activity of the Brain in Man," EEG Clin.
to which he refers consists in the fact that in his toposcopic pictures of brain waves, as one goes out from the center, the rays indicating the frequency are confined to relatively narrow sectors. 10. Barlow, J. S., "Rhythmic Activity Induced by Photic Stimulation in Relation to Intrinsic Alpha Activity of the Brain in Man," EEG Clin.
Neurophysiol., 12, 317–326 (1960). 11. Cold Spring Harbor Symposium on Quantitative Biology, Volume XXV (Biological Clocks), The Biological Laboratory, Cold Spring Harbor, L.I., N.Y., 1960.
Neurophysiol., 12, 317-326 (1960). 11. Cold Spring Harbor Symposium on Quantitative Biology, Volume XXV (Biological Clocks), The Biological Laboratory, Cold Spring Harbor, L.I., N.Y., 1960.
Artificial intelligence and neuroscience, xvi, xxxiii–xxxiv
Artificial limbs, 38
Astronomy, 43–50 gravitational, 51
Astrophysics, 48–49
Ataxia, 12–13, 129–130
Autocorrelation, 254–259 of brain waves, 259–265
Automata, 57–62 coupled to the external world, 60–61
Automation, x–xi, 39 in early science, 56–57 literal-minded computers and, xii speed of, xii
B
Background noise, 16
Barlow, John S., xlvii
Bartlett, F. C., 34
Bateson, Gregory, 36
Bergson, Henri, 62–63
Bigelow, Julian H., 10, 11, 13, 14, 18, 23
Blind persons, reading by, 33, 191–192
Blueprints, xliv–xlvi
Bohr, Niels, 53
Borel, Emile, 65
Bose, Amar, xli
Boyd, Constance D., xlvi
Braille, 33, 191–192
Brain. See also Nervous system alpha rhythm of, 34, 195, 273 comparison of human versus lower mammal, 210–214 death effects on, 205 functional and organic disorders of, 202–203 handedness and, 211–213 higher mental functions in, injury to, 13, 212 neurons of, 165–166, 203–204, 209–210 power consumed by, 181–182 shock treatment of, 205–206 sleep effects on, 205 synapses of, 20–22, 25, 29, 165–166, 171, 271–272
Brain waves autocorrelation of, 259–265 defined, 251–252 fireflies experiment and, 275–276 harmonic analysis of, 254, 263, magnetic tape for recording, 254–255 oscillators of frequencies, 273–274 sampling problem, 265–271 self-organization of, 251, 277–279 technology for measurement of, 252–254
Brazier, Mary A. B., 255
Bread-and-Butter Fly, 176
British empirical school of philosophy, 174
Brownian motion, xxv, xxvi, 120 time series, 98–105
Communication engineering, 14–17 message-noise problem in, 93–94
Communication technologies, xi
Communities, 215–216 group information in, 219–221 homeostasis in small, closely knit, 222–223 intercommunication in, 216–217 social sciences and, 227–228
Computation errors, correction of, 200
Computing machines, 7, 8, 19–21, 34, 39, 56, 61, 85, as analogy machines, 161, 165–166 brain as, 20–22, 25, 29, 165–166, 171 conditioned reflexes in, 178–179 contingencies affecting operation of, 163–164 data inserted at beginning in ideal, 162–163 defined, 159 as essential for the war effort, 22–23 game-playing, 228–230, 234–239, 241–242 induction by, 172–173 learning apparatus in, 179–181 logic of, 173–174 mathematical logic and, xxiii–xxiv, 19–20 memory and, 166–169 numerical, 161–162, 194 pathological processes in, 204 power consumed by, 181–182 for recording information, 159–161 relays used in, 164 for solution of partial differential equations, 6, 161, 180–181 telegraph-type repeaters, 168–169 two great types of, 161
Conditional probabilities, 96–97
Conditioned reflexes in computing machines, 178–179
Conductivity, phenomena of, 25–26
Conscience, 205
Control by informative feedback, 154–155
Control systems, 132–133
Conway, Flo, xiv, xvii
Cybernetics, or Control and
Communication in the Animal and the Machine on cybernetics as lingua franca of animals and machines, xv–xvi, xxi–xxii first reissue of, ix
Cybernetics, ix–xi recognition of uncertainty, xviii, stances against science taken by, xvi–xvii statistical mechanics and, xxix–xxxii
D
Dark Hero of the Information Age, xiv
Darwin, Charles, 51–53
Darwin, George, 50, 52
Davis, James W., xlvii
Death effects on the brain, 205
De Santillana, G., 35
Descartes, René, 57–58
Discrete time series, theory of, 121–125
Dubé, Georges, 42
Durchmusterung, 44–45
Dynamics, 174–175
E
Effective boundary, 137
Electrical currents and circuits, xxxvi, xxxvii, 29, 246–247
Electroencephalography, 253–254
Electrophysiology, 252–253
Energy computing machine, 181–182 human, 59–60, 181–182
Engineering, ancient, 54–55
Entropy, 80–83, 88, 90
Ergodic hypothesis, 71, 77–80
Ergodic transformations, 78–80
Ericsson, M., 27
Evolution, Darwinian, 51–52
Ex-Prodigy: My Childhood and
Exterior points, 137
Entscheidungsproblem,” xxi
F
Facebook, xi, xii, xviii
Feedback control, xxx, 11–12, double system of, 146
Feedback(s) affective tone as, 176–177 anticipatory, 153 chain of transmission and return of information as, 129–131 compensator and effector, 153–155 complicated system stabilized by two, 144–145 control by informative, 154–155 differential operators and ranges of, 139–143 double system of control and, eye-muscle, 184–185 homeostatic, 156–157 in human animal reflexes, 153–154 infinity and, 137–138 information and, 132–133 interior points and, 137–139 mathematics of, 133–137 mechanical systems of, 132–133 negative, 132 oscillations due to excessive amount of, 148 of voluntary activity, 132, 146–147
Feedback systems, information, xxviii–xxix
Ferromagnetism, 169–170
Filtering problem, time series, 114–121
Fireflies, 275–276
First-order programming, 238–239
Fisher, R. A., 17
Fourier transformation, 105–108, 115–119
Fremont-Smith, Frank, 26–27
Frequency modulation, 255
Freud, Sigmund, 178
Functional disorders, 202–203
Fourier transformation, 105–108, 115–119 multiple, 97–98 prediction problem for, 110–114, 120 theory of amount of information and, 127–128 theory of discrete, 121–125 zeros in, 109–110
G
God and, 58–59 nineteenth-century study of, 59–60
Greek origin of name Cybernetics and, 18 moral position of, 41–42 preface to second edition of, xxxv–xlvii
Gibbs, Willard, xviii, 53–54, 70 ergodic hypothesis of, 71 statistical mechanics work of, 65–67, 70–71 on transformation groups, 71–76
God, 57–59
Goethe, Johann Wolfgang von, xix, 243
Google, xviii
Greed, xiii
Group information, 219–221
Group scanning, 189–191
Gibbs’ work on, 65–67
H
Haldane, J. B. S., 34, 128
Handedness, 211–213
Harmonic analysis, 34–35, 254, 263, 275
Harrison, George, 32
Harvard Business Review, x
Hayles, N. Katherine, xvi
Hearing, sight replaced by, 196–197
Heims, Steve J., xvi, xviii
Heisenberg, Werner, 53
Hey, W., 43
Homeostasis, 156–157 in small, closely knit communities, 222–223
Human body, energy of, 59–60
Human logic, 173–174
Hume, David, 174, 183
Huxley, Julian, 234
I
Index
Index
Index
Index
I Am a Mathematician, xv
Ikehara, Shikao, xlvi
Induction, 172–173
Industrial revolutions, 40–41
Infinity, 137–138
Information amount of, 86–91, 127–128 chain of transmission and return of, 129–131 and communication as mechanisms of organization, 27–28 conveyed in odors, 217
Index
Information (cont.) group, 219–221 statistical theory of amount of, 16–17 storage of, 166–171 various forms of communication of, 217–219
Information feedback, xxviii–xxix
Intercommunication in organizations, 216–217
Interior points, 137–139
Internal noise of receiving system,
Invariants of the group, 74
Ito, xxvi
Index loss of, 203–204 sensory prosthesis and, 195–196
Index
Index
Index concentration of visual information in, 185–186 eye-muscle feedbacks in, 184–185 group scanning in, 189–191 outline drawing recognition and, 186–187 reading by persons without, 33, 191–192 replaced by hearing, 196–197 sensory prosthesis and, 191, 195–196 stages in diagrammatization of impressions in, 187–188 transformations in, 188–189
Index contributions to mathematics by, xxiii–xxiv, 19–21 depression experienced by, xvii–xviii design of linear control system and, xxvii–xxix on greed, xiii information defined by, xxiv–xxv introduction by, x on mass media of his day, xi non-linear systems and, xxxii–xxxiii, xxxvi–xxxvii overt warnings about technology, xi, xviii prophecy of, through
J
Jacobs, W. W., xix, 243
K
Kalman, R. E., xxvii
Kalman–Bucy filter, xxvii
Kipling, Rudyard, xviii, 239–240
Koopman, 77, 79
Kotelly, John C., xlvii
L
Lettvin, J., 21, 178
Lewin, Kurt, 27
Lies, 221–222
Limbs, artificial, 38
Lindsley, Donald B., 272
Linear control system design, xxvii–xxix
Linear invariants, 74, 77
Linear operations, 238–239
Linear prediction, theory of, xliii–xliv
Lobotomy, 205–206
Locke, John, 174, 175, 183, 187,
Logic, human, 173–174
Lorente de Nó, Rafael, 23
Learning machines, xlii–xlv, 179–181 games and, 228–245 magic and, 242–244
Lebesgue, Henri, 65, 69 work on theory of trigonometric series, 67–68
Lee, Kai-Fu, x
Lee, Yuk Wing, xlvi, 6, 24
Leibniz, Gottfried Wilhelm, 4, 19, 58–59, 215
Lebegue’s work on trigonometric series and, 67–68, 69
Maxwel demons in, 81–83 time average in, 70 transformation groups in, 71–76
N
Negative feedbacks, 132
Nervous system. See also Brain affective tone and, 175–177, 207 artificial intelligence and, xvi, xxxiii–xxxiv ataxia of, 12–13, 129–130 chain of transmission and return of information in, 129–131 limits to organization of, 207 mathematical logic and, 20–22 memory and, 166–170 messages to, 177–178 neurons in, 165–166 synapses in, 20–22, 25, 29, 165–166, 171, 179, 271–272 tabes dorsalis of, 13, 129–130 treatment of discrepancies by, 201–202 vision and (See Vision)
Neurons, 165–166 lengths of chains of, 209–210 memory loss and, 203–204
Newtonian mechanics, 53–54 in astronomy, 45–50
New York Times, ix, xii
Nikodym, theorem of, 96
Noisy-channel coding theorem, xxiii
Non-linear prediction, xliii–xliv
Nonlinear Problems in Random
Non-linear systems, xxxii–xxxiii, xxxvi–xli, 239 oscillations of frequencies in, 273–274
Non-linear systems of relaxation oscillations, 150–151
Non-linear transducers, 246–249
Northrup, F. C. S., 28
Numerical machines, 161–162,
O
“On Computable Numbers, with an Application to the
Ontogenetic learning, 233–234
Organic disorders, 202–203
Organisms, multicellular, 215–216
Organizations. See Communities
Organ pipes, 148–149
Oscillation, 29, 30–31 of frequencies in non-linear mechanisms, 273–274 mathematics symbolism of, 133–137 non-sinusoidal, 148 in organ pipes, 148–149 relaxation, 149–150
Osgood, W. F., 70
Outline drawing recognition, 186–187
P
Paleontology, 51–52
Parkinsonism, 146
Partial differential equations, 6, 161, 180–181
Pascal, Blaise, 19
Pavlov, Ivan, 175
Pfeiffer, John, ix
Phase average, 95–96
Phosphorescence, 170
Photocells, 193
Photography, 169, 170, 187
Phylogenetic learning, 233–234,
Physical contests as games, 241–242
Pitts, Walter, 20–21, 23, 26, 33, 42
Planck’s law, 53
Prediction linear and non-linear, xliii–xliv time series problem, 110–114,
Prefrontal lobotomy, 205
Principle of Uncertainty, xviii, 15
Probabilities, conditional, 96–97
Probability, a priori versus a posteriori, 86–87
Probability density, 90–93
Probability zero, 66–67, 96
Proofs, logical, 173
Psychology, 171–172
Psychopathology commonalities between the brain and computing machines and, 199–200 distinction between functional and organic disorders in, 202–203 traffic jams and, 207–209
Ptolemaic geocentric system, xxxvii
Purpose tremor, 13, 146
Q
Quantum degeneracy, 170
Quantum mechanics, 125–126
R
Raisbeck, Gordon, xlvi
Ramos, F. Garcia, 26
Random noise, xxxix–xl
Reading by blind persons, 33, 191–192
Reflexes, feedback in, 153–154
Relaxation oscillation, 149–150 non-linear systems of, 150–151
Notes
Notes
Notes
Notes
Notes
Notes
Notes
Notes
Notes
Notes
Notes
Notes
Notes
Index
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
Y
Z
Cybernetics · Second Edition (1961) · Norbert Wiener
Forewords by Doug Hill & Sanjoy K. Mitter · 10 章 + 导论