I just read this article by Murray Rothbard on the national debt. I found it highly enlightening. Partly to organize my own thoughts, I'm writing out some of what I took away from it, with some inferences and additions of my own. (I urge you to read the article yourself.)
First, consider the nature of private debt. C, the creditor, gives D, a debtor, the use of some of C's money for a specified period of time and at a specified price (interest payment). D prefers to pay more in the future in order to have the money now, and C prefers to give up use of the money now in order to get more in the future. This is basically a voluntary sale, from which both parties benefit. It is positive-sum and everyone is happy. Also very important is the obvious fact that both parties are incurring all the costs involved in securing the benefits they seek, and thus have every reason to be careful. Unless criminal or regulatory activity (eg., deception or "easy credit" policies) distort the incentives involved, there is no reason in the world why this sort of debt would grow beyond reasonable bounds.
Now consider public debt. First there is a beneficiary, who for the moment I will call E, who gets a sum of money from the government. This money comes from B, in exchange for a promise from the government to return it with interest. This money will then come, not from E, but from a third party, P. Unlike private debt, this is a coercive, zero-sum game that E and B are playing at P's expense. They come out ahead (in fact, E comes out way ahead) while P suffers a pure loss. (E stands for "tax eater," B stands for "bond buyer," and P stands for "tax payer.")
For various reasons, this system contains virtually no incentive to keep within reasonable bounds the burden that E and B impose on P. Unlike C and D, E B and P are not even individual human beings, but anonymous swarms of people. There is no way E and B can put a face on their victim. In fact, P may not even have a face, not having been born yet. If the system is a democracy, the votes of E and B will have a far stronger effect than those of P, who may be unborn or under-age. Further, the costs imposed on P are spread thinly over a very large group of people, while the benefits reaped by E and B are focused on them: E and B have far stronger motives to run up more of this debt than P has to avoid it.
Barring criminal and regulatory distortions, private debt is economically efficient, in that the social benefit exceeds the social cost: because D and C are bearing their own costs, they are forced to make sure they are exceeded by their benefits. Public debt on the other hand will not be economically efficient. Since E and B are coercively shoving their joint costs on to the hapless P, they have no reason to take them into account and minimize them.
Given the incentive structure of the E-B-P relationship, there is nowhere for P's burden -- ie., the debt -- to go but up. It will grow and grow until P is bled dry and the system bankrupts itself.
Not only is public debt economically dubious, it is ethically problematic as well, and this is so no matter what ethical standard you use. For instance, if you believe in any sort of distributive justice, you must think that it is "unequal," and "unfair" to P. Notice that B is likely much more wealthy than P is. Public debt is regressively redistributive. (That is one of the reasons Alexander Hamilton liked it: it creates a new class that is both beholden to the state and powerful enough to back it up.) On the other hand, if you are a Lockean individualist, you should think that E and B are "robbing" P. And if you are a utilitarian or any sort of consequentialist you will be duly horrified by the system's gross inefficiency and by the fact that it leads to catastrophe if unchecked.
Only twice in our history has the public debt been paid off. It was liquidated by two presidents -- Jefferson and Jackson -- who understood its economic and moral perniciousness and hated it with a towering passion. Perhaps we still have a chance to undergo a moral revolution and return to the wisdom of Jefferson and Jackson. That may be our only hope.
(Hat-tip to Nat Hunt for discussing these issues with me.)