The Bible states that congregations are to “put away from amount yourselves” the wicked. The analogy used to illustrate the teaching is that of leavening (yeast) in making bread, how just a little leaven leaventh the whole lump of dough.
In context, the entire chapter, 1 Corinthians, Chapter 5:
1 It is reported commonly that there is fornication among you, and such fornication as is not so much as named among the Gentiles, that one should have his father’s wife.
2 And ye are puffed up, and have not rather mourned, that he that hath done this deed might be taken away from among you.
3 For I verily, as absent in body, but present in spirit, have judged already, as though I were present, concerning him that hath so done this deed,
4 In the name of our Lord Jesus Christ, when ye are gathered together, and my spirit, with the power of our Lord Jesus Christ,
5 To deliver such an one unto Satan for the destruction of the flesh, that the spirit may be saved in the day of the Lord Jesus.
6 Your glorying is not good. Know ye not that a little leaven leaveneth the whole lump?
7 Purge out therefore the old leaven, that ye may be a new lump, as ye are unleavened. For even Christ our passover is sacrificed for us:
8 Therefore let us keep the feast, not with old leaven, neither with the leaven of malice and wickedness; but with the unleavened bread of sincerity and truth.
9 I wrote unto you in an epistle not to company with fornicators:
10 Yet not altogether with the fornicators of this world, or with the covetous, or extortioners, or with idolaters; for then must ye needs go out of the world.
11 But now I have written unto you not to keep company, if any man that is called a brother be a fornicator, or covetous, or an idolator, or a railer, or a drunkard, or an extortioner; with such an one no not to eat.
12 For what have I to do to judge them also that are without? do not ye judge them that are within?
13 But them that are without God judgeth. Therefore put away from among yourselves that wicked person.
Are you implying the lost should not be allowed in church gatherings? If that were the case there would never be anyone come forward for salvation. If someone did they should be thrown out.
The Corinthian passage was talking about tolerating sin in professing Christians within the church, not about unbelievers coming to the gathering seeking truth. There are other passages in the same letter with instructions on those people.