You are most correct that God guides and directs the affairs of man. My statement I suppose lies in how He brings that about. I suspect that sometimes He uses demonic forces to move us into accomplishing His will. Other times He understands our nature so well that He can effect His plan just by knowing what we'll do. Probably the most famous case of these two examples is Adam and Eve. Certainly God put the tree and the serpent in the garden and then issued the command. The serpent deceived Eve into sinning and falling. Yet, there was no serpent to tempt Adam. Only the lust of his own heart which gave way to sin. God knew which direction he would go and He created the circumstance. But more important He had to show Adam what he was actually like. It wouldn't have done any good if Adam was deceived for then he would never know that he would trade Paradise over a piece of fruit.
God used two different methods to accomplish this outcome. It was clearly within God's will that both would fall-yet they fell in different ways. But, as scripture teaches us, Adam's fall was the greatest of all. Had he not been presented with this choice, we would never have know our rebellious nature or the love of God to want to reconcile His elect to Himself.
To speak to your point Harley, I wonder what would be the point of God creating a universe of puppets? I tend to believe that the overall arch of the "story" is guided to God's purposes. For example: an apostle would betray Jesus. Perhaps it wasn't pre-determined to be Judas, but something about Judas made him the most succeptible to give in to that temptation?
I hesitate to go so far as to say we can "surprise" God, or thwart his will. But unless we can choose and act with some independence, I don't understand why God would spend all this time and energy with us.