You are correct. The terms of surrender allowed all Confederate officers to retain their personal side arms. General Lee tendered his sword to General Grant at the surrender meeting. General Grant declined to accept Lee’s sword.
Too bad all the Yankee Generals didn’t fair so well. I have General Crook’s sidearm sitting in a cabinet. One of my ancestors rode with McNeil’s raiders. When they snatched Crook he relieved the good General of his Whitney Dragoon and then passed it on down. :-)