Davis did impose martial law, conscription and repressive measures at various times. The book to read on this subject is Mark Neely's "Southern Rights: Political Prisoners and the Myth of Confederate Constitutionalism." Because we all know about Lincoln, people judge him in a vacuum and assume that Confederate leaders weren't doing similar things.
Davis was also a prime backer of our military expansion into the Southwest. A good thing in my estimation, but not the sort of thing non-interventionists and anti-statists would automatically approve of. Davis was not a regular guy just trying to get government off his back, either. He was a part of an elite of its own that aspired to build up a state for itself. Many Southern highlanders recognized this and either were unimpressed, uncooperative or outright rebellious. The fact that he was not a libertarian in the modern sense doesn't in itself make Davis worse than other statebuilders, but why pretend he was something that he wasn't just because he didn't win?