My ESV Study Bible notes say this note on leaven: "Not yeast (which was uncommon in the ancient world) but fermented dough, a little of which would be left from the pervious week to be added to a new lump of dough. By analogy, when publicly known sin in the church is not subjected to church discipline, it will silently spread its destructive consequences throughout the whole fellowship."
I also like to think of it in regards to sin. When people have a sin that they are aware of that they are not repenting of, they don't realize that that it affects the whole body and life spiritually. I don't know about you, but when I am refusing to admit my sin, every other area of my life seems to be affected. When we allow sin to enter our life, even more when you are not repenting, you allow the evil one to have a stronghold. There will be other areas you will notice he will try to tempt you and weaken your relationship with God. Sin does not just affect you- it affects the people around you too. That's why my study notes refer to the sin affecting the whole fellowship.