Rather than a II V in the key of A, I would rather call it a V I in the key of E, since the B resolves to the E.
If it was a II V, the first chord, a B, would the II, and the second chord, the E, would be the V, making the I chord an A.
But since the progression never resolves to the A, I find it hard to think of the A being the I.
Another reason is that you can noodle around with E blues scale or E Mixolydian scale and it sounds right, whereas the A scales don’t sound right at all.
Or it could be a I IV with the Bm being the I chord, and the E being the IV.. this makes even more sense, since the Bm blues scale sounds wonderful over these chords.
Just like Oya Como Va by Santana - Am to D7 - which I would always consider to be in the key of Am, since it never resolved to G.
You are right that it could be a V I, however, in that case, the V chord is a minor chord, and it defies traditional harmony in a major key. You can have a minor as a V chord in a minor key, but in this case the minor is resolving to a major chord (E9) rather than a minor I (Em). Though there is the case of minor V resolving to a minor I and then the minor I become a major I (Picardy cadence).
There is a lot to play on top of this progression. I don’t mind notes in the key of A even though the A chord is never heard. That is what modes do — they shift the tonal center away from I chord. As well, you can play major and minor pentatonic in E, and some outside stuff like a diminished scale, Lydian Dominant mode etc. It’s kind of wide open.