There have always been various levels of authority. An archdiocese is a big territorial area that takes in several dioceses, and the archbishop is responsible for the bishops of these dioceses (that is, in the sense of coordinating things with them, making sure their dioceses are running properly, etc.). Cardinal is basically an honorary title for someone, usually a bishop, who is one of the Pope's advisers. Generally, archbishops of major archdioceses (New York, Madrid, etc.) are made Cardinals. In the past, however, even laymen have been made cardinals, and Pope JPII elevated some priests, like Henri de Lubac.
Bishops' conferences came out of Vatican II, but I don't believe they were specifically ordered by any of the initial documents. Many things seem to have emerged rather mysteriously from Vatican II, without much foundation or a clear idea of who was behind them or why. The conferences do not have canonical status; that is, they are not dealt with in canon law as an established heirarchical body. But on the other hand, they seem to have taken over and exert a lot of pressure on individual bishops and feel that they have a lot of power over the Pope (judging by the stupid anti-Tridentine statements coming out of the French bishops' conference). Unfortunately, it seems that they are usually led by the most liberal among the bishops and devote themselves to extending their vision of a theologically vague, socially liberal, committee-formed Christianity. We can all see how convincing that has been!
Actually, the first document issued by Vatican II, Sacrosanctum Concilium, established regional bishops' conferences, which were supposed to deal with liturgical matters. However, Vatican II never vested the bishops' conferences with the kind of power that the USCCB has exercised. It never intended for these conferences to become intermediaries between the Holy See and individual bishops nor did it give them the authority to substitute their own directives for unpopular directives from Rome.
Unfortunately, at least in the US, the bishops' conferences have become and done just that. The NCCB did, in fact, consider itself a kind of superdiocese and it was a superdiocese that had its own bloated bureaucracy that was not even accountable to the bishops, let alone the laity. Dioceses across the country created their own bureaucracies which mirrored the one in Washington and which were just as unaccountable to the Catholics they supposedly served.
Finally, I disagree with Shaw's claim that the NCCB/USCCB would have handled the sex abuse crisis better if it had been stronger. The bishops' conference has mishandled just about everything that has come before it. Indeed, the bishops exempted themselves from the "zero tolerance" policy that they imposed on the rest of the clergy. Apparently, its provisions were too draconian to apply to themselves.