My concern is that a manager started the group. He has explicitly selected a favored group of people to hang out with. Historically, the people in a group like this will be the ones getting the more interesting assignments, the promotions etc. from this manager. They will end up having more impact on design decisions (and will likely make a bunch of them in the bar where you and the other excluded will have no chance to give any input.) and in general the people excluded will become more isolated over time at work. They will be considered not good enough for the group and that is not a good place to be.

You are right to be upset especially when the jerk told you specifically that you weren't good enough to join because you were female. Certainly, I would have no respect for this manager and not trust him on any issue in the future. He has basically told you that you are not a person of value to him. Act accordingly.

What to do about it is another matter however. You can raise the issue to HR and become a pariah with the group even if you win. You can go work in a different company where you have more enlightened managers. You can create another group of people who meet outside of work on a different night and include all the men in the group except him. You can tell him that you are insulted that he doesn't consider you a friend. None of those are wildly good though.

What you probably should do is watch out for signs the group is getting preference at work. Especially note when there are decisions or opportunities made that you are unaware of because you aren't in the group. If you find yourself in that position, ask when this was decided. You want visibility to the idea that this group is getting preference or more impact on decisions.

Make sure to be heard when design decisions are being made and make sure to push for the assignments you want. If decisions are being made excluding you that you professionally disagree with, then push back on them and explain why those ideas are bad. Make it clear that you are a contributing professional who deserves respect. You will have to work harder to get noticed positively at work than these guys will. That is a given. So make sure you do. Is that fair? No. But life isn't fair.

Now personally if I were this guy's boss, I would have a talk to him about favoritism. What he has done is totally unprofessional. He is opening the company to liability when he wants to give these people promotions, etc. There is is reason why many managers will not pursue friendships with the people who report to them.

Actually there are a lot of them. If these guys are doing something wrong, is he as likely to correct them or let it slide because of the friendships? Managers cannot, in general, afford to be friends with their subordinates. It causes workplace issues. It causes good people who are not in the "In" group to move on to other jobs. It causes poor decision making because everyone with a stake in the decision doesn't get an input. It cause people to take advantage of the friendship and ask for favored treatment the others not in the group don't get. And it creates resentment. It also makes the men in the group feel that excluding women or minorities is OK and contribute to turning them into crappy managers themselves someday.

(And by the way, there are plenty of places with more than one female programmer. Where I work there are not only several female devs (at one point we had more than 20 of us) but female managers all the way up to and including the VP in charge of all development. They just are outside certain sections of the industry like start ups and game companies. The next time you are looking, look for a place like this; you will find out that it is much easier to be dealt with fairly in a place like that.)

From the comments, it appears foreign workers are excluded too and that your boss and his boss attend. My advice to you is to leave this company. If your own boss won't speak up for you when he attends and your boss's boss thinks it is ok to exclude those who are different, you have no future at this place. This is the group who will receive preference for everything. Leave now. Only white males need apply here.