I have set my own criteria for when to tell people to get the hell out.
I don't open the door for people unless I want to stay the entire time they are there and I know them well enough to count them as my guests. If someone wants to break in I usually pretend I don't notice, unless I know they don't live in the neighborhood, they are unaccompanied teens/kids, or they start hell raising.
I would rather enjoy the time with the kids and not deal with extra drama if I don't need too...