Seagal hasn't been a "Hollywood tough guy" for a long, long time, unless you're talking about it being tough for him to button his pants.
I remember reading that he used to beat his ex-wife, Kelly LeBrock. I liked “Under Siege”, but Seagal always gave me the creeps.