Practically speaking, organisms can add genes for greater diversity, and delete genes to save energy. This is why a fern called adders-tongue has the largest number of chromosomes at 1440; but a species of ant, called the Jack jumper ant has only two chromosomes in the female, and one in the male.
http://en.wikipedia.org/wiki/List_of_organisms_by_chromosome_count
Humans seem to be deleting some of our chromosomes. It is believed that the male ‘Y’ sex chromosome used to be an ‘X’ chromosome, but lost one of its “arms”. It seems to be failing as well, so another ‘X’ chromosome will eventually take its place. Right now, a major cause of retardation is called “fragile X syndrome”, which is an ‘X’ chromosome with a weak “arm”, that may eventually break off altogether.