That was all the Yankees that moved in - every knows Confederates were barred from holding office.
Black Codes were actually an antebellum construct that came out of yankeeland. Desiring to preserve their lily white populations, yankee states placed severe restrictions on free black residents than most southern states placed on them. In fact, the Ohio River was a statutory Berlin Wall to free blacks before the war because most of the states to its north barred blacks from settling or even doing business in their territory. As a result, the largest free black populations before the war were in slave states.
As for repayment of the black confederate veterans, that may be found in pension records and the accounts of confederate veterans reunions from the late 19th century through the 1930's when increasingly fewer veterans were still alive. Many white confederate veterans fought long and hard to get pensions established for the black ones, and several sponsored their applications. The blacks were also universally honored at the reunions. There was a large one at Gettysburg, I believe, during which the yankee organizers shunned the black confederates and failed to provide them with housing during the reunion so some southern whites gave up part of the space that had been allotted to them.