I admit it took me awhile to come to the same conclusion, but you're exactly right. I watched it a second time, I 'get it' now... ;-)
What bugged me in Eastwood's routine was his occasional stumbling and bumbling... he talked for too long, and I couldn't understand what he said at times. I did understand the empty chair routine, though. Wasn't that done in the Vaudeville days? I know comedians like Bob Newhart did that schtick as well.
Yes, it’s an old routine dating back to before the invention of nutmeg.