The big problem is always #3 and #4. One of them is always shortchanged. You have a tougher argument that #5 is the best team in the country.
Just have a four team playoff, #1 v #4, 2 v #3. Just 2 games. Heck, they could hold them next week. Have two bowl games reserved each year. One for #1, one for #3. Problem solved better. Not perfect, but manageable.