There have always been various levels of authority. An archdiocese is a big territorial area that takes in several dioceses, and the archbishop is responsible for the bishops of these dioceses (that is, in the sense of coordinating things with them, making sure their dioceses are running properly, etc.). Cardinal is basically an honorary title for someone, usually a bishop, who is one of the Pope's advisers. Generally, archbishops of major archdioceses (New York, Madrid, etc.) are made Cardinals. In the past, however, even laymen have been made cardinals, and Pope JPII elevated some priests, like Henri de Lubac.
Bishops' conferences came out of Vatican II, but I don't believe they were specifically ordered by any of the initial documents. Many things seem to have emerged rather mysteriously from Vatican II, without much foundation or a clear idea of who was behind them or why. The conferences do not have canonical status; that is, they are not dealt with in canon law as an established heirarchical body. But on the other hand, they seem to have taken over and exert a lot of pressure on individual bishops and feel that they have a lot of power over the Pope (judging by the stupid anti-Tridentine statements coming out of the French bishops' conference). Unfortunately, it seems that they are usually led by the most liberal among the bishops and devote themselves to extending their vision of a theologically vague, socially liberal, committee-formed Christianity. We can all see how convincing that has been!