Monday, October 25, 2010

Just How Inefficient is Our Society?

I was listening to a conversation about efficiency in the U.S. economy the other day. It’s actually amazing how far below our actual capabilities we are right now when you stop and think about it. I’m thinking in terms of fully utilizing our human resources.

I recently spent 11 months unemployed. I went from being an experienced and productive computer engineer to someone who produced zero net contribution to our economy for those months. I worked hard every day, but that effort was devoted to my job search routine, filling out job applications, preparing for interviews, etc. (I also did volunteer work and spent more time with my family, but I’m not counting that here.) Fortunately I have a job that allows me to be productive again, but when I look back at that time I spent unemployed I shake my head at the waste of time and effort that could have been put to much better use.

Today we have about 10% of our workforce in this situation – representing maybe a 10% waste in our productivity.  It’s hard to estimate what portion of the potential workforce is not counted in that number because they have given up or see no opportunity for a meaningful job. Some people estimate that the real number when these people are taken into account is closer to 20%. Then add a few percent more cover the people currently in prison. We can argue about adding the number of people in the military to this amount too, but that’s a relatively small number compared to other factors.

The next big factor, and potentially the biggest factor of all, is the number of people underemployed. These are people who are working at jobs that, for whatever reason, do not allow them to fully contribute to society what they are capable of. By the time you add all these factors in, our human workforce as a group is almost certainly contributing less than half of what they are capable of.

You could spend considerable time debating what portion of this is due to personal choices, cultural factors, or inefficiencies in the market. I don’t know the correct answer here. I’m sure of two things though: (1) We as a society are doing a fraction of what we are capable of and there’s room for considerable improvement, and (2) we are in a situation where there’s a considerable need for that extra human potential to solve a growing list of problems.

Thursday, October 7, 2010

Positive Feedback is Making the Stock Market Increasingly Unstable

On May 6th 2010, the Dow Jones Industrial Average for the NY Stock Exchange dropped by 1,000 points in a matter a minutes in an event than has become known as the “Flash Crash”. Fortunately for people invested in the market, the Dow regained most of that loss very quickly. This was an unprecedented case, and a warning – how could the stock market change so much so quickly?


It turns out that the flash crash was caused by a single large stock trade, and the series of high frequency computer trades that quickly followed. This should be taken as a very serious warning of how unstable the market has become.

Stock markets are inherently unstable because of positive feedback. Positive feedback is a basic phenomena in control theory in which parts of a system function to increase the size of any change in either the positive or negative direction (negative feedback works to reduce or dampen any change). The market is full of positive feedback loops . Increasing stock prices attract more investors which produces even greater increases in stock prices. Falling stock prices scare away investors which causes even greater reductions in stock prices.

It is a basic principle of control theory that when positive feedback becomes large enough, the system becomes very unstable and begins to experience wild oscillations. The loud squealing you sometimes hear when there is too much feedback in a sound system with a microphone is closely related to this.

Unfortunately, the amount of positive feedback in the stock market is increasing in size and speed each year, being driven mainly by very high speed computer trading. There was a recent story about a new transatlantic cable being laid between England and the US primarily to allow faster computer trading on the NY stock exchange by firms in London. The current cables have a 65 millisecond delay, and the new cables will reduce the delay to under 60 milliseconds. I’ll save the commentary about the tremendous amount of waste in physical resources and intellectual talent that is going into this for another time. For now, I’ll just point to it as an example of the increasing speed and strength of the positive feedback effecting stock markets, bringing them ever closer to the point of wild instability.