Superficially, that may be correct. But if I wanted to gather at the meeting house with other believers and you choose to keep the doors shut, it's wrong. Besides, I don't know your habits and am not pointing specifically at you, but VERY few who claim as you (the place of worship makes no difference) take the opportunity to worship at all outside the setting of gathered believers.
Exactly my point - a person that only worships when at church is not a believer in the first place.
Being a believer is not a two hour a week exercise. Being a true believer means that Christ is at the center of your world. The church building is just a meeting location for fellowship.