Was it really an insane asylum?
L
Yep.
From the Wikipedia entry for Roosevelt Island:
Through the 19th century, the island housed several hospitals and a prison. ...By 1839, the New York City Lunatic Asylum opened, including the Octagon Tower, which still stands but as a residential building; it was renovated and reopened in April 2006. The asylum, which was designed by Alexander Jackson Davis, at one point held 1,700 inmates, twice its designed capacity. In 1852, a workhouse was built on the island to hold petty violators in 220 cells. ... The Asylum was renamed Metropolitan Hospital. However, the last convicts were not moved off the island until 1935, when the penitentiary on Rikers Island opened.