According to this article, the department currently has about 350 undergraduate majors. That's spread over (at least) 3 classes, probably 3 1/2 would be a fairer estimate. So, about a 100 graduates per year, more or less.
Columbia College increased the number of students it admitted back in the late 1990's and early 2000's, so the number back in Obama's day was likely a bit smaller. The total undergraduate population is about 20% higher these days, so if the major has 100 graduating students per year now, it likely had 80 or fewer back in the 1980's.
But it's still pretty easy to get lost in a crowd of 80 students if you don't do anything to distinguish yourself. If he studied on his own, took only large lecture classes, lived off campus, didn't participate in many student clubs or events, and wasn't an academic star, it's pretty plausible that very few of his classmates or professors would remember him.