Not an accurate conclusion. What the historical instance of rejecting infant baptism in the 3rd or 4th century shows, is that infant baptism was being practiced then - nothing more.
First rejecting it then - or, more accurately, the first record of it being rejected - only shows that there was a perceived need to reject it, i.e. it was being practiced.
It may well have been as you conclude, but it is not logically established by your argument.
If we can establish that it was indeed practiced during the days of the apostles, and if we can establish that the account of which you speak was the first time it was challenged, then we can conclude that it was an accepted practice. However, this accepted practice would still be without doctrinal foundation, imo, since it was not directly addressed in the New Testament.
Even Peter was not infallible - Paul writes that he “withstood him to his face” regarding the treatment of Gentile believers.
Again, I am not dogmatic on the point, but I question the soundness of the doctrine.
Of course, it is metaphysically possible that infant baptism was not practiced until the Fourth Century, and was suddenly introduced at that time, and someone objected to it.
But if that were the case, then the rejection of infant baptism would have been the ORTHODOX position.
As it was, the rejection of infant baptism was a heretical position.
Which means the more likely course of events was: Infant baptism was normal from the beginning. In the Fourth Century some heretics objected to it, and the orthodox Christian position was to defend it and continue it.