Here is an interesting document from the National Security Archives at George Washington University, dated July 24, 1945.
http://nsarchive.gwu.edu/NSAEBB/NSAEBB162/42.pdf
The key paragraphs are the last three, in which Ambassador (to Moscow) Sato (1) sent a long message home recommending the acceptance of any peace terms, including unconditional surrender, as the only way of preserving the Emporer and the State; (2) concludes that he may be alone in his quest for peace but that his conscience demanded that he pursue it; and (3) the Japanese War Council received his message, rejected it, and determined that “the war must be fought with all of the vigor and bitterness of which it is capable as long as the only alternative is the unconditional surrender.”
That was pretty much what I meant. There were still some rational people in the Japanese government, but they took their lives in their hands if they did this in anything other than a very quiet, behind the scenes, private process known to very few.
You could have a bunch of fanatics with samurai swords and pistols show up at your house, as happened.