I think you're right. This quote: "The then head of the Vatican Pontifical Council for the Family, Cardinal Alfonso Lopez Trujillo, said: "The Aids virus is roughly 450 times smaller than the spermatozoon. The spermatozoon can easily pass through the 'net' that is formed by the condom." seems to agree with your assertion. It's from an article from Timesonline. Here is the
link.
The article ends with the statement, 'The WHO said that "consistent and correct" condom use reduces the risk of HIV infection by 90 per cent.' Doesn't sound like great odds!