by
(Omitted--Section I: Information Theory, and Section II: Application to the History of Art)
Note for the Web Version, Feb 2000: Most of what appears here was derived conceptually from a most important but apparently ignored paper by Samuel Bagno (1955--Ref 1 below). Had Bagno's contribution been understood at the time, we might have been spared the disastrous intervention of Harvard economical theory in Russia, and might now be faced with a friendly and prosperous Russia instead of the hurting giant that threatens our future. Bagno's argument seemed to me to be incontrovertibly true in 1955, and nothing that has happened since has caused me to change that opinion. Two of Bagno's conclusions are:
Both of these conclusions directly contradict the currently prevailing wisdom. This, then, is the relevant part of my Bachelor's Essay. It is not a simple re-hash of Bagno, and says some things he did not say--and Bagno said much that is not here, and said what is here better than I could at that time. The interested reader should search out the IRE Convention Record Part 4 for 1955, and read Bagno.
I.1. Another application of information theory is to economics. This can be seen intuitively by noting that all social activity depends largely on communication. The exchange of goods involves an exchange of information between the manufacturer who says "I can make this article", the retailer who says "I can sell it", and the buyer to whom the retailer says "I have it" and who says "I want it". It can be seen that no articles would be made except for the personal needs of the manufacturer unless there is an information flow. If there were no information flow, then each would have to provide his own needs, and also learn how to do it by trial and error. The whole flow of goods depends basically on the flow of information.
I.2. Any goods can be considered purely as organization (1). This organization is considered as caused by some decision sequence involving the schooling of the laborer, the designer and the manufacturer, and the expressed desires of the buyer, as well as the actual manufacturing decisions. Each item can therefore be considered equivalent to n bits of information corresponding to the probability of its coming into being by random association of the parts. The only other thing entering into the manufacture of an article is the power involved, and in general this is cheap and its cost is that of the peopled involved. If this is conceded, then we must allow that what determines the cost of an item is its probability of occurring, at the point in space time when it does occur, by chance. In other words, we pay for decisions, for organization. it can be seen that the random probability of two identical objects conforming with a given pattern is the square of the random probability of one, yet we would expect the cost to be double, This indicates the our logarithmic probability should be used to denote the organization, and suggest the applicability of information theory.
II.1. Let us denote the organizational entropy by "selectivity", any random or noise entropy by "entropy" and the overall entropy by "entropy". This is to distinguish "selectivity" which reduces the entropy of the universe from what it would be in an unorganized system, from the random entropy and the overall system entropy, which are not what we pay for.
II.2. Now if we pay for decision sequences, i.e. selectivity, then we can equate money to potential selectivity, in so far as one can exchange money of objects. It will be shown that one cannot equate a unit of money with a unit of selectivity because the relation changes with time.
II.3. A manufactured object involves a certain selectivity, and certain random entropy. A usage involves an object with certain properties, (Selectivity). All other properties are random entropy. If one pays for selectivity, then, an attempt is made to match the usage selectivity to the manufacturer's selectivity. For instance, if the usage requires a block of something to hold up a window, one does not buy a block of stainless steel machined cubic to tolerances of .0001². One obtains a block of wood or scrap metal, where the object selectivity is low. Here the money is a channel transferring the selectivity from the manufacturer to the consumer. Money can in general be considered as a communication channel with a certain capacity for transferring selectivity.
II.4. From the preceding example, it can be seen that the money channel also conveys what is from the users point of view random entropy, or properties the user does not need, but still pays for because from the manufacturers point of view they represent selectivity. Also in the price paid for an object includes the ³overhead² involved in its manufacture--material waste, upkeep, etc., and this is random entropy from the manufacturers point of view. therefore the money channel carries random entropy as well as selectivity.
II.5 We are now in a position to point out a few analogies between the economic situation and information theory.
These are as follows:
| Selective Entropy | Selectivity (value of goods) |
| Noise | Waste, non-match of article to use, etc. |
| Channel | Money |
| Channel Capacity | Money supply,. selectivity per unit |
| Redundancy | Excess money (extra selectivity potential) for same goods, or extra goods (selectivity) for same money. Anything that increases the flow of goods overall. |
III.1. The main difference between the applications of information theory to economics and to communication is that in the communication case we are concerned with a simple one-way link, and information transmitted is not re-transmitted while in the economic case we are dealing with an information net, where information received is either lost or re-transmitted. Since there is loss of information (consumption, waste or spoilage) at various places in the net, there must be redundancy added to the transmitted information, if the amount of information circulating at any given time is to be constant.
III.2. This redundancy can be in terms of verbal information (do-it-yourself books), more money for a given amount of goods, or more goods for a given amount of money. This is possible because the information transmitted from any node in the net is not the same as that received by the node. However, the entropy output of a node must equal or exceed the entropy input, if we allow savings (hoarding). but the selectivity output may be less than the input, since noise is added at every node.
III.3. A typical partial net is shown in Fig.1. It deals with the newsprint industry. Goods flow and money flow are as shown. This is considered a closed system in that the selectivity output to the blocks marked ³Other output² is assumed equal to the selectivity input (money) from them, and then there is no loss of selectivity in them. All other blocks have an output selectivity different from the input. The natural resource blocks have no input from the economy, and their selectivity output is low, consisting of the evolutionary selectivity of growth and condensation.
III.4. The manufacturers (paper, chemicals, and machinery in the example) give money to the lumber camps and mines, and get in return some selectivity (semi-refined ores, cut wood and delivery). This selectivity will in general be less than that permitted by the amount of money exchanged, since there is wastage caused by excess refinements of cutting the wood for pulping, shipping of rock to the machine factory, accidents in the mines and forests, and other things not useful which cost money. Thus the selectivity at the manufacturing level is less than the potential before the transaction. Similarly the selectivity potential of the money in the hands of the primary producers-viewed now as consumers--is diminished because the manufacturers must ask more money per unit for the received selectivity. Also the manufacturers must ask money to cover their own waste, etc., and thus the selectivity potential of the money in the hands of the secondary consumers (consumer product manufacturers) is reduced. Thus the potentials in the hands of the consumer is reduced, and he must ask more money from his employers to counter his own waste, consumption, etc. The loop is now closed, and the selectivity potential per unit of money can be seen to be continually dropping.
IV.1. For this reason a source of money must be provided. This source of money will normally be the government by deficit budgeting, but it need not be. Any node from which the selectivity output exceeds the selectivity input when averaged over a long time (where selectivity includes potential) will serve as a source. A non-profit charity is a source as the members contribute time (and selectivity) free, using money merely as a transfer variable from donations to goods for the charity. If the members of the charity do not contribute selectivity, it is no longer a source, but a transfer node where noise is added because of uncertainty of the contributors as to the destination of the contributions. A potential source might be a bank, where money deposited is both loaned out and used by the original depositor. The loan may be just in the form of a credit note in which case the money may be lent to someone else as well. However, if too much potential is generated at a single node, a localized runaway may form (see VII 4-5 for theory of runaway inflation), and the bank fail, in which case its credits will be worthless. This is the reason for large banks.
IV.2. Now as one cause of loss of potential is mismatch of manufactured selectivity with useful selectivity, and as this implies mismatch of money potential with consumer selectivity, then this loss must be least when the consumer knows best how much selectivity he wants and what he ought to pay for it. A complete knowledge of the correct amount of money to match a desired amount of selectivity requires an infinite time for learning, during which the number in question is constant. This is so because the process involves statistical analysis of prices for all goods made by all manufacturers. Such a situation has been heuristically shown to be impossible, as the selectivity potential of money continually decreases because of other noise generators. Therefore there is always a loss due to channel mismatch caused by insufficient learning, and relearning of the instantaneous selectivity potential of money is always necessary. This involves time delay. (see 3.VII and 1.VIII 4)
V.1. If the change in selectivity due to other causes is very fast, then the relearning will be poor, and the channel mismatch will be bad, thus reducing the selectivity still further. If this process gets bad enough the economy will become unstable, and the selectivity potential will drop almost to zero. If this occurs, the only recourse is to barter, where selectivity is traded for selectivity directly, without the intermediate step of money potential. The economy must then stabilize, but at a very low level, because the transfer of selectivity in goods is not nearly as easy as in money, because the goods transfer must be person to person, and it is much more unlikely that any given person will have goods satisfactory to the party of the second part than that he will have the money to pay him.
V.2. Under such conditions, credit buying on personal notes of exchange will begin, and these notes will be for goods, probably annotated ³or equivalent². Thus every man becomes his own bank. At this stage, the government will probably recall all bills of exchange in return for an arbitrarily assigned unit of money, and the cycle begins again, with no runaway inflation, if people have confidence in the new money and know what things are worth in it. Incidentally, it is quite feasible for the government at any time to retire the currency and substitute a new one with a different unit value. This is just equivalent to a change of scale; but if such a change induces or increases lack of confidence in the money, there is a danger of inflation again.
VI.1. Now loss of confidence in the money is precisely equivalent to loss of confidence in the output of a communication channel, or equivocation. Anything which cause the output of the channel to be in doubt is equivalent to equivocation. This includes doubts as to the designs of large companies, and the stability of the government, as well as all the things previously mentioned. The stability of the government is particularly important in these days of the welfare state, since a lot of a person¹s income goes to the government in return for services. Of course the government contributes waste and other equivocations just as does any other selectivity transducer, but its main contribution to the equivocation of the channel is public uncertainty as to where the money is going to go. In other words, there is a probable mismatch in the public mind between potential lost and selectivity gained, and hence equivocation.
VI.2. Since the government is the largest user of money, this uncertainty of purpose is a very prominent part of the equivocation in the channel. Thus a vacillatory government may well become a major source of inflation, especially if its vacillations are to do with its fiscal policy. It would thus seem that supplementary budgets to combat inflation are unwise unless they are intended to warn the public that excessive use of money is undesirable. If less transactions per unit time occur, less noise is added to the system because of mismatch, waste and any other equivocation which varies with the amount of selectivity transfer. However, noise due to unemployment is liable to happen if this process is carried too far. Taxes designed to reduce the buying power of the public, are also unwise except under the same conditions since such taxes increase the amount of money about which uncertainty exists. Government scandals also tend to produce inflation, as does political instability. One would expect then, to find the worst inflation conditions occurring just before an election in a welfare state when there is a distinct possibility of a change of government and when the government has not been putting out statements of purpose for the use of money.
VI.3. The question of employment arises. If we recall that for a noisy channel, the amount of information lost per error is most for low error rates (1% error loses 9% of information, 50% error loses 100% of the information in a binary symmetric channel(1)), then the result by analogy is obvious. A high degree of unemployment would be expected to disorganize the economy because there is a large total degree of uncertainty and thus a fast loss of selectivity in the money. Similarly a very small amount of unemployment tends to cause inflation as the amount of noise inserted per person unemployed is high, since the employers are forced to employ unsuitable people in critical jobs, and those who are unemployed can pick and choose their jobs from among several choices. These will normally choose the highest paying jobs offered, thus increasing the part of the money circulation to which excessive noise is added. Therefore the wastage and other noises increase to a greater extent than for medium unemployment. In this way inflationary tendencies increase. Looking at the question from another point of view, over-employment gives trade unions an even more excessive power to increase wages and thus cause inflationary tendencies. This aspect of the case is straying from the information theory point of view towards the game-theory point (2) of view. A complete economic analysis will probably require an amalgamation of both.
VII.I. None of these inflationary tendencies will matter if the channel capacity does not decrease, since it is only the channel capacity which determines the flow of goods. Now if we assume that the noise added per transaction is N bits per bit, then the percentage change is entropy/money unit (not selectivity) is proportional to N. This may be written
d(H/C)/(H/C)dt=KN VIIa.
where K has the dimensions 1/T, if T is the time-constant of the economy for relearning the code. This is a reasonable interpretation since if d(H/C)/(H/C)dt is small, then the bits/bit of selectivity lost per unit time will be small. In addition to the transaction loss, however, there is a loss due to waste and consumption which is just proportional to time, and so
d(H/C)/(H/C)dt=KN+LM VIIb.
then
H/C=e^(KN+LM)t VIIc.
VII.2. Also the capacity will be reduced compared to Co (The capacity at time to) in the same manner, viz.
d(C/Co)/(C/Co)dt=-K1N-M VIId,
or
C/Co=e^-(K1N+M)t VIIe,
where K1 is another time constant. This holds if the money supply, is constant, and show that without an ever-increasing money supply, the economy will run down. This is intuitively correct, although some economist¹s believe that any money supply is sufficient. Their reasoning is not obvious.
VII.3. The expression above states the selectivity/money unit decreases exponentially,. and therefore that for a constant capacity the money supply should increase exponentially at a rate A/Ao=e^(K1N+M)t. The actual value of the change will probably be slightly higher than the bank rate in normal times. Such a change must affect N in some way, since the faster the percentage rate of change of money in circulation, the higher the relearning loss per unit.
VII.4. If the relearning loss per unit goes up with the rate of increase of money supply, then it follows that at a certain rate of increase, the increased redundancy is only compensating for the equivocation it introduces by changing the code. The consumer is always using a code which was in effect at some time T previous and if the code is changing at some rate, use of the wrong code will introduce equivocation.
VII.5. This equivocation
will depend on the use of the channel. If the channel is used to capacity, then
use of a non-ideal code will cause equivocation. Assuming that the channel is
used nearly to capacity, then the efficiency of the code, which was correct
a time T previous to the time in question, will have some functional dependence
on the rate of increase in money. This dependence will be such that the faster
the increase, the less the efficiency. Also the rate of increase of ideal channel
will vary linearly with the rate of increase in money. Therefore the rate of
increase of time channel capacity will have some functional dependence which
is a multiplicative function of the efficiency of coding. The efficiency of
coding may well be negative if the increase of equivocation exceeds the increase
in redundancy. In such a case, the channel capacity goes down, regardless, and
the inflation is runaway. There is, on this analysis, a rate of increase of
money (in units per existing unit) which allows maximum increase in channel
capacity. Any higher rate of increase is liable to be regenerated and send the
system into runaway inflation. Physically this would be caused by some misguided
source who attempts to increase the capacity by adding money. The only way that
the system could be saved from this situation is by the unpaid contribution
of selectivity, since the true criterion is not really % rate of increase of
money, but rate of increase of money/unit selectivity ratio, or prices. If this
is constant, prices are constant, and the code is constant.
VII.6. The exponential solution only takes into account the redundancy introduced by extra money. Redundancy introduced by selectivity of goods will increase channel capacity without an increase in money, and this is why it can compensate for the negative slope portion of the rate of increase of C. This means that one of two alternative must be chosen. First, the workers must take wage cuts, or second, production per worker must increase. It is unlikely that the first alternative would be chosen in a time of incipient runaway inflation. The second alternative is the function of automation. If the channel capacity is increased by this method, then it can increase faster than by a mere increase in the money available. However this can also cause equivocation due to job dislocation. Therefore we can expect there to be an optimum rate of introduction of automation. Thus it would seem that the channel capacity cannot increase at any rate greater than a derivable maximum, if automation and money control are the only measures. If any attempt is made to increase it at a greater rate, it will increase slower. It is safer to run at a rate of increase below the maximum, because otherwise the system is liable to become unstable.
VII.7. It will be noted that this result has no rigorous derivation, but is arrived at by analog methods, and may therefore be wrong.However, the story of historical runaway inflation is just that which would be predicted by the theory. Also it is interesting to note that the great depression of the thirties was preceded by a period of excessive prosperity. This is suggestive of the truth of the theory.
VIII.1. Consider
a general block diagram of the economy, concerned only with money flow. This
diagram can have any of its blocks split into many cross-connecting units (e.g.
3.III.2.), but basically it represents the whole economy, without banks, which
are not necessary because they act only as sources, and we are going to consider
at present only an ideal lossless economy. We shall add the result obtained
from this model to the lossy economy considered earlier, to obtain a more complete
solution for the whole economy. In the block diagram, money--or selectivity
potential--flow is as shown. Any person may belong at different times to any
or all of the blocks, depending on his instantaneous function, so that the composition
by people is indeterminate. However, at any given instant, a given person is
in only one block, and he has a definite probability of being in any given block.
The composition of the individual blocks is thus determined statistically, and
will fluctuate. Also the composition of the total economy changes as people
die, boards of directors are voted out of office, and so forth, and generally
the control of segments of the money flow changes hands.
VIII.2. The function of any block in this diagram is to delay the passage of money; this is to say that any particular impulse in the input is not immediately felt at the output. Any unit of a block then has a transfer characteristic including a delay. This delay is equivalent to a temporary saving. It may only be the saving of part of a Thursday paycheck to cover next Wednesday¹s lunch, but the delay is there. It is evident too that the money will not all be spent a uniform time after it is earned, but that the spending will be spread out over a more or less long time. This is equivalent to assigning a probability distribution to the delay (analogous to the impulse response of a system). This distribution will not be the same for all people, or even for one person under different circumstances. There will be a probability distribution of delay characteristics across the economy with character and time. This means that the economy has three statistical degrees of freedom, and the output corresponding to a sudden input (somebody has a windfall) will be probabalistically determined as a function of time.
VIII.3. Previously we have been considering a completely determined economy, where the money was spent as soon as it was made. If there were just one delay time, then the previous discussion is still valid for the effects of waste, consumption, and general doubt, but the effects of the delay must be added. On the servo-mechanism analogue, it seems likely that the system would go into an oscillation, with a period equal to the loop delay time. Now if the delay time is probability controlled, with a certain peak time, the frequency of the oscillation will be indeterminate, but will in general center around the frequency corresponding to the most probable delay time. (The oscillation variable is the amount of money involved in transactions in unit time.) Thus it seems likely that unless the delays involved are ³white², which means that all delays are equally probable if it is not known which subunit in a block will handle a give bit of money, the autocorrelation function of the money function will not be identically zero for any but zero interval, but will have some non-zero values at other intervals. Thus the standard of living even in a lossless economy will fluctuate, more or less periodically. This explains the ³business cycle². When we add this result to the effects of waste, we obtain an economy which, for constant rules, has a standard of living which fluctuates about an exponential rise or fall. This is in full accord with history.
VIII.4. It seems that the effect of the oscillation will be minimized when the delay time are distributed most evenly over all periods, with about the same amount of money being subject to any given delay. A reasonable measure of the average delay through a unit might be the time it takes at the normal rate of expenditure for the unit to use completely its reserves. On the basis, it would seem that the larger the amount of money handled by a unit, the longer its delay. This is obviously not always the case, but it is sufficiently close to be used as a working rule. This means that the amount of money handled by all companies of a given size should not vary with the size range under consideration. This is to say that the number of companies of a given size in an optimum economy is governed by the size, so that the total number of people employed by all companies of a given size should not vary with the size. This is assuming that the delay varies with the number of people employed, which is probably only correct to a poor approximation. Bagno (1), on an assumption nearly equivalent to this, has calculated the optimum distribution of firms in an economy the size of the U.S. economy, and has obtained a result not far removed from the distribution which prevails in the country. Differences may be attributed to taxation and anti-trust laws. The effects of the governmental interferences have not been investigated.
VIII.5. It may be seen from the foregoing that free enterprise, or Capitalism, is an efficient system, and that the less government control there is, the more efficient it is. On the other hand, Socialism is highly inefficient, as all trade is done with the government, and it is not to be anticipated that the loop delay will have a flat spectrum when one block consists of the whole government, and the other of individuals acting alone. Also, on the basis of the first discussion, Socialism is wasteful, and leads to a tendency toward inflation. Thus Socialism should lead to larger fluctuations, which will preclude it from approaching the peak in the inflation-expansion curve (3 VII 5), and it will have a high inherent inflation rate. Thus the Socialist economy cannot expand as fast as the Capitalist economy. This is not to condemn Government enterprise. Government is just as legitimate a member of the ³manufacturer² box as is any other enterprise, and so long as the government portion of the circulation does not get so large as to have the indicated effects, there is no problem. On the whole, restrictive legislation tends to harm the economy. This is just as true of anti-trust, (though probably not anti-combine) laws as is of anti-union laws, or laws against strike breaking. It would seem that the freest economy is the best.
IX.1. This has been a fairly cursory survey of the possibilities of Information Theory in the investigation of Economics. It will be noted that we have not mentioned any limitations imposed by the availability of resources. This is an additional problem, not within our frame of reference. This attach on the problem is in a way supplemental to that of Von Neumann and Morgenstern (2), whose Game Theory treats the formation of the units of the blocks, and the strategies to be followed by them. A fuller answer to the Economic problem might be obtained by a formal combination of the two approaches into one Dynamic Theory of Games, In any case, it would appear, if only because of the inherent impossibility (noted by Wiener) (2 Sec 1) of obtaining good statistics, that the synthetic approach here exemplified is superior to the analytic approach common in economic literature. With poor statistics, the answers obtained from an analytic method will be far from unique, as is seen from the many conflicting theories which abound, whereas a synthetic approach will give answers which depend solely on the initial assumptions. These may be in error, but they are not as likely to be far enough in error to invalidate the approach, and if they are, then some answer which is contradicted by the practical situation will occur to point up the error.
IX.2. The object of the thesis, as has been pointed out, is to show the very wide range of usefulness of the concepts of Information Theory. We have applied it to electrical communication, linguistics, music and the other arts, and to economics. In the literature, much wider use is made of the theory. In particular, application is often made to Optics. Thus we see that the range of use of Information Theory is very wide, as it encompasses the whole range of communication.
1. S. Bagno The Communication Theory Model and Economics. IRE Convention Record 1955 Part 4 (Computers, Information Theory, Automatic Control)..
2. J. von Neumann and O. Morgenstern Theory of Games and Economic Behaviours
3. W.R.G. Baker Automation. C.R.
4. R.W. Bolz Automation, C.R.
5. E.C. Cherry Generalized Concepts of Networks. Information Networks. Brooklyn Polytech., 1952