I read one account of a visit to Africa by President Bush where it sounded like it took Air Force One the better part of a day to return from Africa to the US. A commercial flight would probably take a little longer, especially if there was a layover or two. If you add a day for time-zone difference, you could conceivably have Wilson leaving the US on the 21st US time=22nd Niger time and reaching Africa sometime on the 23rd or maybe the 24th. That still leaves a day-plus to account for, though. He does mention in his book that all of his conversations in Niger did not involve his uranium investigation: "Not all of the conversations focused on the uranium industry; some of my callers were interested in discussing the business climate. . ." (Page 23)
It takes 18 hours ( flying commerical ) , IIRC, to get to South Africa, from America and that's not counting the time difference. But I don't know how long it takes to get to Niger; somewhat the same amount of time, I should imagine.