Using spectroscopy you can measure the velocity oscillations of the host star and planet in the direction of the earth very very precisely. Based upon this you can determined the planets effect on the motion of the star and that gives you a measure of the relative mass. The masses of stars are generally determined from their luminosity since there is a very strong correlation between mass and luninosity for “main sequence” stars.
Not to mention that you can determine the orbital periodicity by watching the red shift and blue shift over time.