Who did they pray to?
They don’t. The reflect on the mysteries of the Rosary while they say the Our Father, Hail Mary and Glory Be.
The Rosary, in a way, asks Mary to pray for us. Christ to pray for us, the Holy Spirit to pray for us.