And there we have it, from Peter's own lips.
If they had the Spirit of God at that point then that is when they would have been placed into the Body of Christ. “For by one Spirit are we all baptized into one body, whether we be Jews or Gentiles, whether we be bond or free; and have been all made to drink into one Spirit.” (1 Corinthians 12:13)
But true faith is one that will be expressed in baptism, if able, which normally is the first formal confession of Christ, and can be the actual time of conversion, and i think it should normally be concomitant with the conversion event, although there is no real difference btwn confessing with thy mouth the Lord Jesus, as per the promise of Rm. 10:9,10, confession being the essential aspect so even mutes can be saved, versus doing it by body language in baptism, if able. Only when one makes the latter the only way one can be born again does it become a problem, esp if you make restrict it to one particular church.
Rome does believe in baptism by desire with perfect contrition (with the desire being counted for the act), but this is disputed by some traditional RCs.