Today's smartphone has more computing power than the computers used on Apollo 11.
While Apollo 11's computers were state at the art at the time, they weren't particularly powerful. They were soon eclipsed by the first widely available PCs. What made Apollo's computers unique at the time was the compact size and weight.
If you want to compare to a modern smartphone, it's more appropriate to use an late 90's era supercomputer. The iPhone 5S is competitive with even a top-of-the-line Cray supercomputer from 1999, using a benchmark in common use at the time.