(This post is far outside my area of competence; for excellent political coverage by people who - unlike me - actually do know what they're talking about, look no further than MTC at Shisaku, Jun Okumura at Globaltalk 21 and Tobias Harris at Observing Japan)
To an outsider with no direct stake in the outcome, Japanese politics is a wonderful combination of spectator sport and soap opera (to the affected insider, of course, it more resembles a Greek tragedy re-imagined by Samuel Beckett). Even so, however, the behavior of the perpetual government party LDP has been stranger than usual lately.
To very briefly recap, LDP together with minor religious party New Komeito holds a two-thirds majority in the Lower House. Opposition DPJ has the majority in the Upper House - a rarity in Japanese politics. The Upper House needs to vote in favour of any legislation passed in the Lower House for it to become law. But, with a two-thirds majority, LDP and New Komeito can override the Upper House vote after 60 days for most (but not all) legislation.
On the face of it, it means LDP can ignore the opposition and do what it wants. The current reality is messier. First, the supermajority is slim, and dependent on a small coalition partner (granted, New Komeito has the role of doormat to LDP down to a fine art, but they're still a different party, with a different agenda) and an increasingly independent-minded diet membership to bring the needed votes. And some legislation, like personnel appointments, can't be overridden; LDP must get the assent of the opposition, or get enough opposition Upper House members to defect their party over the issue.
Also, the next lower house election is due no later than next year with LDP virtually certain to lose the supermajority, so they are facing years of governing with divided houses. Plainly, obviously, high time to find ways to work with the opposition DPJ and accommodate them in order to get things done. Governments in many other countries are able to do so in this kind of situation after all.
The LDP, however, seems incapable of doing so. They have been pushing through legislation using the supermajority override with little regard of the effect on any future cooperation with the opposition. In fact, and more strangely, they have been acting as if the opposition did not exist; as if the LDP still had control over both houses. One example was the appointment of the new head of the Bank of Japan. As I mentioned above, personnel appointments are among the legislative acts that can't be overridden. And yet, the LDP managed to field a total of four candidates in a row guaranteed beforehand not to be accepted by the opposition, overrunning the end of the previous chairman's term, leaving the chairmanship vacant for weeks. In the end, the acting chairman was promoted almost by default.
Another current example is the botched renewal of the perpetual temporary gasoline tax used to fill the coffers of road and bridge construction industries (a major LDP supporter). The DPJ opposed the earmark to the construction industry as Japan has a large surfeit of rarely used motorways and bridges to nowhere already, leaving LDP to use the supermajority to get the bill through. They inexplicably failed to do so on time, however, letting the tax expire and making a lot of motorists happy with the opposition for opposing the bill. This leaves the government with the self-inflicted unenviable task of reintroducing an publicly unpopular tax by walking roughshod over an opposition they desperately need in a year or less.
The LDP is clearly not acting sensibly; but as it happens, I have seen behavior just like this in another field:
The Prefrontal Cortex is a collection of brain areas that regulate and inhibit other areas as needed when the situation changes and their behaviour is no longer appropriate. Some psychologists talk about "executive function", and while I don't like the term, here it is appropriate. Malfunctions in these areas can be diagnosed by a simple test called the "Wisconsin Card Sorting Test".
The subject has a pile of cards with various symbols and colors, and sorts them into piles according to some rule (like "red cards to the left, black to the right"). They don't know the rule for sorting, but are told if they are sorting correctly or not; most people quickly find the right rule. Then, without telling the subject, the rule is changed. Normal subjects get confused for a little while as the old rule no longer applies, but then explore and find the new rule. But subject with damage to their prefrontal cortex frequently can not do this - they will continue to follow the old rule even as it is obvious that it no longer applies. They can sometimes articulate this, telling the experimenter that the rule no longer applies; some can even say what the new rule must be, and still they can not change their own old rule-following behaviour.
Now take a look at LDP behaviour: repeatedly enacting legislation that won't pass the Upper House and missing deadlines as if the supermajority override was not going to be necessary; acting as if the opposition didn't even exist. Then the expressions of incredulous bafflement when the opposition does what it's supposed to be doing: statements that "the situation is intolerable"; that it must be unconstitutional to oppose the government; accusing the opposition of "playing politics" with passing a bill (when, pray, are you ever supposed to "play politics" if not as a politician in the nations premier political arena)? They are, in fact, acting remarkably like a patient with impaired prefrontal cortex function.
The reason for this, I believe, is in the nature of the LDP: it (like the opposition DPJ) isn't really a political party in the the sense of being a coherent political organization tied together by a common ideology, shared history and worldview. It is rather a Frankensteinian agglomeration of multiple small political parties, factions, organizations, special interest groups, ministries and agencies. A look through LDP history shows it constantly splitting off and reabsorbing smaller organizations and members. It is less a party than a political mini-state of its own, with complex internal processes for determining the outward policy stance.
And as any Byzanthine administrator would have been able to tell us, keeping such a diverse array of selfish interests together requires lots of procedures and formalisms for any decision, lest the party tear itself apart. A large, complex set of explicit and implicit rules and regulations become necessary within such a party for all decisions in order to balance out interests and factions with each other. Appointments must be weighed carefully, with a candidate beholden to this faction and beneficiaries from that bureaucracy being repayment to some other group, and creating new obligations to yet other groups in turn. Any legislation must also be very carefully weighed, vetted and gradually moulded in a complex internal process to finally emerge as the consensus decision of the party.
This process has, I'm willing to bet, been gradually solidified and formalised into a "decision machine" over many years of party power until it by now largely is the party. The value of most positions within the party is after all due to how much influence, direct or indirect, the position gives over these decision-making processes. LDP depends utterly on following them; breaking them means losing internal support as the power bases that depend on the process erode. Abandoning these processes mean disintegrating the LDP.
But now the process is disrupted by a viable opposition with power to wield. But people within the LDP can not alter the process even when they see the end result will not be viable; any attempt to change the outcome will inevitably rob some participant or other of a benefit they were counting on, leading to destructive retaliation. And as there is no leadership strong or unified enough to inhibit and alter this decision-making process - no prefrontal cortex, to stretch the analogy - the process continues as it always has, unable to accommodate the widening rift between the past political world it codifies and the changing reality.
So, is LDP doomed to oblivion? Of course not. If you take people that fail to adapt to the changed card sorting rule out of the test room and bring them into a different one, they can suddenly adapt their sorting behaviour just fine. The learned rule is learned within the context of that test, in that room; all they need is for the context to change. Likewise, if the political landscape would really shift - say an opposition Lower House win in the election after the next - that would probably create enough of a context shift for LDP to reform and adapt to current reality.