Found this:
"After his exchange was negotiated, Goff was returned north by the Confederates and paroled on September 1, 1864, at Varina, Virginia, an exchange station on the James River about seventeen miles below Richmond.
There he boarded a Federal steamer flying a flag of truce, and was taken down the James and up Chesapeake Bay to Camp Parole near Annapolis, Maryland. Upon his arrival on September 3, he was given forty-eight hours leave and ordered to report to Colonel William Hoffman, Commissary General of Prisoners at Washington, D. C.
While in Washington, Goff, according to later testimony, had an interview with President Lincoln and Stanton, concerning his treatment as a prisoner of war. The effect of this particular interview is not known, but soon thereafter the government adopted a policy of releasing and exchanging sick and wounded prisoners who would be unfit for active service for sixty days on a man-for-man basis. The new policy was acceptable to General Ulysses S. Grant because Confederate prisoners sent south could not materially affect the war effort. Lincoln's supporters thought that chances of success in the forthcoming presidential election would be enhanced by a concession to public clamor for a general exchange."
http://www.wvculture.org/history/journal_wvh/wvh14-1.html
That doesn't seem to square with what your sources are suggesting.
Too, what possible reason would President Lincoln's administration have for antagonizing the voters in this issue? There's more to it.
Walt
You reminded me of something I read. Someone thought the South's offer in the summer of 1864 to release 10,000 to 15,000 Federal prisoners with no equivalent exchanges of Confederate prisoners was a ploy designed to affect the Federal elections, what with the prisoners mad at Lincoln's government for sacrificing them in prison.
If it was a ploy, it didn't work because the Feds did not arrange to come pick up the prisoners until months later in late November, after the election, leaving the prisoners dying in prison until that time and unable to vote.
You might be interested in the opinion of Walt Whitman in a December 1864 letter to the New York Times:
In my opinion, the Secretary has taken and obstinately held a position of cold-blooded policy, (that is, he thinks it policy) in this matter, more cruel than anything done by the secessionists. ... In my opinion, the anguish and death of these ten to fifteen thousand American young men, with all the added and incalculable sorrow, long drawn out, amid families at home, rests mainly on the heads of members of our own Government..."
Source: Immortal Captives