You don't know your neighbors, you have no idea why people are stopping by. It's not their responsibility to inform you when they come or go.
Instead of knocking on a door to introduce yourself to your next door neighbor, you call the cops.
You said it's not meth, unless they have three pits on chains in the front yard and boarded up windows, I don't know what you're worried about.
I'm willing to bet a young family with little kids would appreciate a friendly neighbor.
Maybe if you know them and they know you, they will be less likely to let their friends disturb you.
I had a neighbor call the cops to say I was a dealer once because cars came and went all night long. I worked nights. I was coming home for lunch.