“He wants revenge for Michonne killing his zombie daughter that he still had insane hopes of curing.”
I had forgotten Michonne killed the daughter - so, he’s got two scores to settle with her.
I also think the governor is afraid of Merle because Merle kills first and doesn’t even think later - he just kills when he thinks it’s appropriate - or not.