You’re right that 5’4” is an average height for women, but the clothing industry considers 5’4” and under to be petite. The sizing has something to do with proportions, too.
The fashion industry might be trying to curry favor with average size women by calling them petite.