My information comes from several sources. With regard to Anderson's original orders to do nothing to improve his position and the armistice with the Governor of SC, see the following:
"United States War Department, The War of the Rebellion: Official Records of the Union and Confederate Armies" (Washington, D.C.: Government Printing Office, 1880-1901), Series I, Volume I, page 3-73.
In "Hearts in Conflict", author Anders says that Buell gave unapproved leway to evacuate to Ft. Sumter if physically threatened.
When he was questioned about threats that prompted his move to Ft. Sumter, Anderson claimed that Confederate small boats cruising around Sullivan's island were menacing him, and that those boats caused him to retire to Sumter.
With regard to the quotes from Floyd and Buchanan see:
"Genesis of the Civil War" (New York: J.A. Hill and Company, 1887, pages 143-144), and again, see "War of Rebellion", volume 1, pgs 3-73.