I'd bet that figure hadn't been revised since the death rates dropped radically.
The citation is from "Moral Theology" by McHugh & Callan, published 1958.
WHO (I know, a lousy source) uses 2.1% as a replacement rate. Even if "3 or 4" is translated as precisely 3.5, that allows for infertility, a large clerical class, and healthy growth.
You are confusing two closely related concepts. 1) The total fertility rate for the whole female population of child-bearing age and 2) the number of children per family.
The famous 2.1 figure is the replacement level of child-bearing and includes an allowance for children to die in childhood, etc.
The distinction is that the first group includes women who will never marry (about 5%), women who get married to late to have children (about 5%), women who are sterile or have sterile partners (about 10-20% of the remainder), and women who never have children, despite being able (unknown to me, but probably less than about 5%).
When you tally all that up, about 25% of women never have biological children.
To make up for that, the remaining 3 of every 4 women who do have children must have 8 children between them.
To allow for a percentage of children who die while growing up or prior to having children of their own, those remaining women should probably have 9 children in total (no fractional children in a small example), so 3 each. That works out to a fertility rate of 2.25.
To allow for population growth, a minimum of 4 each by these 3 women would be needed, since you can't have fractional children, and the needs of society are the equal duty of all. That works out to a fertility rate of 3.0.
Considering the number of families who have just 1 or 2 children each, those of us having more have quite a hurdle to jump to try to push the whole population over the 2.1 marker for total fertility. In essence, with about 2/3 of familes with children having 2, perhaps 10% with 1, and 20% with 3, the remaining 5% is left with the bag. In order to hit the 1.9 mark, where the US is really at, that 5% has to average about 5 children per family, or 5% of families with children are having 15% of the children, and 25% of families with children (roughly the number with 3 and up), are having roughly 50% of the children.