The very same thing happened to me last year in the early part of the season. It was for the bluebirds and the chickadees went into it and looked like they were building and it wasn’t long until I had bluebirds in there. I never knew if the bluebirds ran them out or if they just left! (They didn’t leave me even a thank you for the house..LOL.)
Last year was my first year with it and I can’t remember who occupied it. It wasn’t bluebirds and wasn’t sparrows, I remember that.