The numbers in border states who “fought” for one side can be deceiving. Often they were not really trusted by the feds who agreed to let them stay at home or they only fought if their state was invaded. I had one such ancestor from Kentucky. He served in the union army officially as he was required to do by the draft but his military records show he never left Kentucky.
Lincoln jailed the Maryland legislature without charge or trial because he feared they were about to pass an ordnance of secession. See ex parte Merryman. He also jailed sitting US Congressman from Maryland Henry May again without charge or trial.
Might want to check your dates.