The fascination may have began with the frustrating fact that no one could find a formula that would predict the n
th prime given knowledge of some or all of the (n-1)st primes preceding it. Seems simple enough, but even to this day nobody has been able to do it. See my comments
here and
here for other reasons, which is by no means an exhaustive list. You could easily spend your whole life studying prime numbers.