I tried it once when I was in New Zealand. It tasted like solidified soy sauce. Absolutely disgusting. One bite and I was done.
I described the taste/tecture as heavily reduced soy sauce.
Not sure I'd like it as a vegemite sandwich, but as a condiment or flavoring in a main dish, sure.