True there were some....but they were never a majority or even close to it. They theorized of some south slav union which sounded good to them but as soon as Yugoslavia was put together and it became pretty clear that it was Serb dominated, pretty much all of them immediately hated that and since the Serbs weren't numerous or strong enough to impose their will effectively, Yugoslavia immediately became riven with internal strife.
Interestingly Tito managed the delicate juggling act pretty effectively but he was not a Serb nationalist. He was a Croat/Slovene himself.
Yes, you are right - the Serbs were the victors of WW one and held a relative majority. Thus they dominated interwar Yugoslavia and suppressed the others, especially the Croats (the dread of the Ustasha didn’t just suddenly fall from the sky).
Even Tito, though not a Serb himself, mainly relied on Serbs for support. Especially the Tito era army was heavily Serb officered.