I also believe that it's very, very difficult to explain free will as emanating from an algorithmic machine (and without at least considering the realm of the quantum). One possibility of course is that free will as people experience it is just an illusion. But everything in our minds rebels against that possibility. One possibility that's been bandied about is that a conflux of self-referential algorithmic loops and such in the brain brings about consciousness and free will. For many reasons, I doubt that possibility as well. I'm not totally sure what Tortoise means by a 'finite state machine,' and admit I need to do some reading up on this. Then finally, on the question of free will, it becomes, in very general terms, hard to say that humans are only reacting to their environment in a prescribed way, when a great part of human existence is given over to moral questions, and avoiding the temptations to do evil or bad things.
A finite state machine is a fundamental concept in computational theory. The mathematical definition (which you probably don't want) is any system with a non-infinite Kolmogorov complexity. A more pedestrian (and not quite correct) definition is any deterministic process that can be expressed in a finite (though possibly extremely large) amount of memory. Generally speaking, any finite state machine can be perfectly emulated on any other finite state machine that is larger. The reality is a bit more complicated, because we typically try and emulate large FSMs on smaller FSMs, which opens things up to all sorts of quirks and problems. Proper and useful coverage of FSMs is a lengthy topic with very interesting tangents that could easily spiral out of control on their own.