You're starting out just fine! Get a central list of 5-10 people, meet, talk, FReep and post about your exploits! If you build it, they will come!
BTW - Don't worry about not having any structure. Conservatives in general and FReepes in particular rebel at structure, so keep it loose and you will succeed!
It looks like 5 is all we will have to start with. I had the same ideaa. Start small and word would get out. I'm looking forward to getting started.