I really like the title of one of Richard Feynman's autobiographies - "What do you care what other people think?". His wife Arlene said it to him at one point, and it had a transformative effect on his life.
I used to think that being concerned about what other people think about you was wrong. I've changed my mind, because what other people think about you affects your communication to them and your relationships with them, and that matters.
On the other hand, I think what I meant, even though I didn't understand it clearly, is that we shouldn't fear other people compared to our attitude to God. We shouldn't try to avoid displeasing others at the expense of displeasing God. We should be far far more concerned about what God thinks of our actions than what others think.
On a more humourous note, I came across this quotation the other day, but have slightly changed the wording in my head so have no idea who said it originally.
The key to not worrying about what other people think about you is to realise that by and large they don't.