I hope they are only interested in sex. Save us a lot of time and money.

You should put men in the caveman box. We are. Most people go wrong by trying to overly complicate a man's motives. Dave Chappel said it well, "Shut up, suck his dick, rub his balls a little, and make him a sandwich." Simple. Your friend probably just thought he could get some from you with minimal effort. Good for you for not giving in that easy and making a person respect you enough to put forth a little effort.

No, you are not wrong. It's a personal choice, it can't be wrong. People change at puberty and move away from family, particularly parents. It is called differentiation. That is how we learn to figure out who we are and become our own people. Marriage and kids become the priority after that. Was there problems at home, was he unhappy, was there something he was running away from? What do you get out of your relationship with your brother that you are pursuing? Can you get those needs met somewhere else?