So if they had my email or username, they could feed in batches of passwords to find one that might match?
Yep, that's the concept.
And that's why it's a good idea for you and me to have long passwords, for example mine are at least 12 characters, and for really important things I use 16 or more characters.
And a password manager program, so I don't have to remember them. :-)