I’ve been an iPhone user for a couple of years now and just yesterday, I was looking at some of my pictures and noticed there’s a map icon which shows where the pictures were taken. I had no idea that was even there.
Geo tags can be prevented by changing the setting at:
Settings | General | Location Services
All digital cameras record metadata in a picture's files. If the camera has GPS built in, then it records the location. It's not just the iPhone that does this.
If your camera does not have GPS, you can take the photo into iPhoto on the Mac and record a location. I haven't tried this, but I would guess that you can change the location if you want, or perhaps even delete it.