Movie review: The Corporation.

A few days ago I watched a very nice documentary: The Corporation. The movie explores the concept of corporation and (not surprisingly) discovers that the goals of humanity as a whole and the goals of corporations do not align well at all. It explains why that is so but does not offer a way to fix the situation - and I don't think it should. I think there will be some very interesting and surprising facts for everyone in this documentary, no matter how well educated you are. So if not for other reasons, you should see this movie for its educational value. And those who perceive themselves as critical thinking individuals should definitely watch 'The Corporation', because it will give you food for thought on a topic that few think about these days...

Main ...(read more)strength of the movie is that it (hopefully) will make people realize there is a topic here at all, something worth discussing and fighting over. Corporations are becoming (or have become) the primary way of producing goods in most economies, and most of us take them for granted, as something natural and inevitable. Perhaps this movie will make you realize that corporation is actually an unnatural and inhumane way to create necessary products for humanity, and therefore we need to either invent a new way, or do a better job at keeping corporations in check.

Main weakness of the movie is that it doesn't address one of biggest problems with corporations, namely that in USA and many other countries the corporations are allowed to donate huge amounts of money to politicians, effectively buying votes. So not only do we have a non-living entity being given many rights that are normally reserved for humans, these non-living entities are allowed to have a HUGE effect on the election results, much larger than any single voter, or even all the voters working in a corporation, could possibly have. Countries where corporations are not allowed by law to lobby politicians have a lot fewer problems with corporations sabotaging human population interests.

Book review: "Dreaming in code"

If I were to describe this book to someone in one sentence, I would say that this book explores the philosophy of software development. I guess this topic is so much out of the ordinary that it is only useful and interesting to people whose lives revolve around producing software. And yet this isn't even a technical book, but a fairly successful attempt at fiction. Anyone should be able to read it, although people with little technical savvy may have to look up a meaning of a word or two here and there. The author, Scott Rosenberg, is not a famous programmer himself, but he is a co-founder of Salon magazine and has been around famous software projects for many many years, and is well qualified to write about the subject.

He sets out to explore a deceptively simple question: why is software so hard to write? Why is so much of it so bad, and why is it never finished on time? Why is everyone, both users and developers, always complaining about software? Essentially, why can't we build software like we build cars, like we build bridges?
He decides to explore the topic by joining a moderately large software project (Chandler) as an independent observer. He uses the story of this project it as a skeleton around which he builds his book by discussing all the innumerable troubles that befall almost all software projects.

The approach works, but sometimes Chandler gets too much in the way. Details of how project evolved really take too much space in the book, but thankfully you can get over those parts quickly and dive straight into the juicy parts. And to me the juiciest parts were where other (in)famous software projects are mentioned and discussed. The books is full of little anecdotes, stories and quotes, starting from the humble beginnings of Computer Science and ending in present. And this is really where the book shines. The author really did his homework and went in every damn dusty corner of software development history looking for answers. You will see quotes from Turing, Brooks and, of course, Knuth. You will hear many stories of software disasters of epic proportions, of project failures that are so tragic they rival Babylonian Tower itself. Obviously, Rosenberg sprinkles his own conclusions and thought all over that, but in a really non-intrusive way. He acknowledges that he doesn't really have answers and you are always free to draw your own conclusions from the stories he tells. I appreciated that attitude; and I did draw my own conclusions which were actually different from the ones author made.

The book ends by reluctantly admitting that we don't really know why we keep screwing up our software, maybe because the reasons are many, or maybe because we just haven't found that magic recipe for building perfect programs.

What is weird, is that I felt exactly the same - but before I read the book. However, once I was done I had a completely different feeling - I felt that I actually understood something and could now answer the book's question. I do not have a recipe for baking perfect code, but I think I kind of can explain now why software can be so bad and ugly.

The epiphany came (and I hope the author had one too) when I was already reading the epilogue. In it, Rosenberg suddenly comes back to the questions he set out to answer, and especially "Why can't we build software like we build bridges?". As if he has given up on finding the answer in the history of software, he suddenly asks: "And how, really, do we build bridges?" And that is where it becomes obvious that when we build bridges we mostly suck, too. Not only did bridges crash left and right for most of our history (only very recently did we learn to build them well, and even now there is a bridge fall every now and then) but there are plenty of examples where bridges could not be built on time and on schedule. So there really isn't anything special about software world. We are people, we are not perfect, and we are prone to screw up every project we work on, no matter what field. Maybe the only thing different about software is that it is so flexible and easy to change, it takes much less effort to mess it up.

So in the end, even though the author felt like he hasn't found the answer, I felt like I have. Here is my take on the question: "Why is software so hard?" It really breaks down into three questions:

1. Why is most code of such a bad quality?
The main problem with software is that it is too easy to change it. When you build a bridge, you know it will be very costly to change design mid-way. With software, people are tempted by the ease with which they can make changes. This leads to two traps: first one is in planning. People think that they can change anything they want later and start building a project without a firm idea of what it is. You know, like starting to build a bridge, but not knowing where the other side is, and if it exists at all. Of course in many cases that will lead you to spaghetti code. Another trap is enhancements. Once people have completed the original design (if they had one) they are tempted to keep "improving" it. After all, it is so easy to add features - a line of code here, a line there... This process (almost by definition!) continues until it is no longer easy to add new features - why? - because code has become a mess of extras. That is when projects get scrapped, recoded, replaced, etc. And everyone blames the old team who "couldn't program" and created a mess. But of course, the new team eventually ends up with the same quality code, after all the useful extra features.

2. Why are deadlines so often missed on software projects?
Well, simple. Programming is in large part a work of art and another part is invention. Writing software is not like building cars, it's like designing them. Ask any scientist how well they meet deadlines on their research - most don't even set deadlines! The thing is, we do build software better than cars - it's called copying an executable, and we do it very well. But designing software is not easy. Almost every software project is doing something that no one has ever done before - otherwise they would just copy the code! When you are doing something that nobody ever done before, how do you know how long is it going to take? Who knows.

3. Why is most software so buggy and under-developed?
See previous point. Most managers insist that software is shipped by certain date, no matter what. Since it isn't really possible to predict how much work will be done by that point (see above), a large fraction of projects go live in unfinished condition. In my experience, only companies that do not set deadlines on their work produce quality results. There are many examples of companies who just said "to hell with deadlines!" and they usually produced nearly perfect, very successful products. Blizzard Entertainment is one example.

As a conclusion, my opinion on this book is mixed. I do recommend to read it as a collection of extremely fascinating stories about the history of software development. But in all other respects, your mileage may vary - it may help you think about software development process, but it may or may not give you insights into how to improve it. You may also find all the Chandler-related stuff somewhat boring.

More about Social Security Trust Fund.

My previous post about SS fund being whole was a little misleading. It was interpreted by some people as meaning that "there is no problem, nobody panic". But the correct interpretation should be slightly different. What the analysis done by Bruce Webb shows is that the trust fund has enough assets in it right now to remain whole indefinitely if those assets remain at their expected values. The catch here is that fund's assets are mostly invested in US government bonds rather than in cash. In other words, US government borrowed money from the Social Security Trust Fund. The Fund therefore has direct exposure to US debts, and therein lies the problem. While US government debt is still (officially) AAA-rated, some economists have argued that US has amassed such a huge debt burden that it cannot possibly repay it in a normal way, so some kind of default has to happen. Since outright default of the government (like in Russia or Argentina) is unlikely, debt will be dealt with in a combination of other ways. Some of it will be devalued through inflation and dollar devaluation, some may be paid off through tax increases, and some through budget cuts. None of these three ways alone will be sufficient to deal with the debt. For example, if US lets inflation climb too high, it will devalue some debt that is sitting in things like non-inflation-indexed 10 and 30 years bonds but it will only make harder to pay other debts, such as the ones in short-term bonds that are getting rolled over and on which the interest rates would climb very high. Tax increases are very unpopular and extreme (read impossible) hikes would be needed to fix the budgets. Therefore the most likely course of action by the government would be to slash the spending, and some of the biggest items in the budget are Social Security and Medicare. Essentially, government may have to cut pensions to pay off the debts.

And this is the real threat to Social Security, not the lack of assets in the Trust Fund. The source of the problem is in the unbalanced federal budgets, not the Social Security structure. The reason we hear so much talk in the media about Social Security being broke is twofold. First, politicians want people to start getting used to the idea that Social Security is not guaranteed and will be scaled down. It is much easier to negotiate pension payments down if you can make people believe that they should expect no pension at all. The second reason is that government tries to divert people's attention from the real problem. If people understood that the root of the problem is in unbalanced budgets, not in Social Security itself, they would demand that budgets are cut by reducing things like military spending, and politicians don't want to hear that. It seems safer to convince people that their pensions are slashed because the Social Security system was broken rather than tell them that it's because the money was spent fighting a set of unpopular wars.

So no, Social Security is not broken. The government spending habits are.

Why good scientists should do bad science.

After reading about yet another speculation about nature of the Universe on Slashdot, I decided it was time to organize my thoughts on the matter a bit. This particular Slashdot article was about a scientific paper that asserts the need for scientists to seriously consider an idea that our Universe is someone's virtual reality (VR) simulation (so called VR world hypothesis). This was noticed by Internet community and a lengthy discussion followed on New Scientist and from there on Slashdot. Interestingly enough, the paper was submitted as Computer Science one, but maybe more appropriate topic would be Philosophy or Physics. In fact, here is another paper on a similar topic (and it is much more thought through), but this one is more appropriately marked as Physics.

The predictable reactions ranged from "Who cares, it's impossible to prove or disprove", to "This is pseudo-science, these people are bad scientists and should be ashamed". I think both statements are wrong and too many take them for granted.

Let me start by admitting that currently theories like VR world are not scientific. But that is simply because science is usually defined as "things that could be subjected to scientific method". Just because right now we can't think of any experiments to test these theories or any new predictions that these theories make does not mean we never will. Science is full of examples when seemingly simple problems could not be solved for centuries and I am sure many felt tempted to say that "maybe these problems can never be solved" only to be proven wrong. So to me it seems like thinking about currently unscientific models and trying to bring them into the realm of science is something scientists should do, and it should be respectable work. Working on such seemingly absurd topics does not make you a bad scientist, only a courageous one; you would only be a bad scientist if you were spitting out propaganda and claimed that these theories are scientific and plausible now.

Also, think about this: people who go around saying "all such theories can never be proved or disproved" are themselves making a statement which is not known to be either true or false, and trying to make it sound like it is a logically sound statement. To me they do qualify as bad scientists, spreading a false claim. Until somebody conclusively and logically proves that all the non-traditional theories about the nature of the Universe are not verifiable, I will continue to put these people in the same bucket as religious zealots banning evolution in schools. The only theories which cannot be proved or disproved are the ones which were never formulated: for example, nobody ever defined what God is or supposed to be, so we got nothing to work with if we want to prove God's existence. But once you formulate a theory (like: world if VR) you are in a completely different realm.

In conclusion: it is a respectable effort for scientists to think about issues related to the nature of the Universe and try to build a bridge between alternative theories (like VR world) and today's science. The fact that such theories are not science today does not mean they can not become science tomorrow.

It seems especially ironic that alternative theories continue to get ridiculed by 'science' zealots while in the meantime humans themselves have come very close to building an artificial universe. If we can already build a rat's brain cell replica inside a computer, how far away can we be from building an exact copy of human brain, except represented as a computer program? How hard would it be then to feed this 'software brain' with whatever inputs we want and make it believe it lives in a world we simulate? So maybe instead of ridiculing Matrix-like theories of the Universe, we should realize we are about to create such "matrices" for others - so time has come to start thinking how would we test our own environment for being Matrix-like.

Finally, there is a direct connection between such efforts and religion. If ever we are able to find strong confirmation for one of the alternative theories of the Universe, it could simultaneously confirm and overthrow the belief in the existence of God. For example, if we discovered that our entire Universe is just a computer simulation run by some other beings, it would confirm many things that religions taught us about the world (such as an Act of Creation or presence of some ultimate all-powerful being) but it seems to me that such a revelation would nevertheless be considered a disastrous blow by many who believe in God.

Social Security may not be broke.

I am sure you have all heard the story before: Social Security is so unprepared to deal with its future liabilities that it is completely broke for all practical purposes. Well, one person decided to sit down and actually do the numbers on his own, and documented the calculations in his blog. His conclusion? Social Security is not broke. The blog makes for a fun read for the number crunchers among us.