Newly-minted Ph.D. Dave (fun blog) has a post on doing a Ph.D. in Japan. As he actually went and did exactly this, his perspective is of course a good deal more authoritative then mine. Read it.
I mostly agree with what he writes, but I do have a few minor disagreements stemming from us simply being active at different institutions. While Japanese is helpful, the places I have worked with so far (ATR, NAIST and OIST) are all basically English-language institutes, with coursework and seminars in English, and the international grad students have not had any study-related language problems.
Of course, your social life may well suffer without Japanese. But then, your social life will suffer no matter where you are, if you do not know the language. You'd have the same problem fitting in in Germany, Sweden, Italy... And if you go to Britain or the US, there's a reason most, for example, Chinese students tend to socialize mostly with themselves; North European students with themselves; Japanese with themselves and so on. Even when you do sort-of know the language, lack of confidence and cultural differences mean you're still likely to opt for the comfortable and familiar.
Also, his assessment on thesis advisors are spot on — but not in any way specific to Japan. What kind of lab and what kind of advisor to choose is a huge subject, and there's countless blogs out there hashing through the pros and the cons. Most important, I think, is to have an "acting" advisor (who may or may not be your formal one) that you work with, and that you're comfortable working with.
Having a Big Name advisor that see you twice a year is no problem at all if they have the foresight to hand you off to a younger but skilled researcher to teach you the ropes in their stead. In a way that is the best of both worlds: you get frequent hands-on tutoring from someone still young enough to know what you're going through; and you get a Big Name vouching for you when you scramble for that first Post-doc once you're done.