I don't think it is selfish at all, at least they are invited and have the option to go. I think you will find they are probably just a little dissapointed that you will not be having the big white wedding at home. I'm sure they will come around.
Your wedding should be about the 2 of you and what what makes you happy. I think if you try to make everyone else happy you won't end up having the wedding you want. I'm sure if these are the people that love and care about you they will be supportive even if you do not choose the things they would have wanted. Maybe you could include them in the planning so they feel more a part of the wedding.
We got married in Fiji in May this year and took only our parents and siblings (and their partners), we didn't even tell anyone else we were getting married there so they didn't have the option of coming. We had a party at our house (BBQ and drinks style) when we returned and invited all of our friends and extended family then. Some of them were dissapointed not to have been there to witness the actual wedding but nobody was mad at us and everyone understood why we we choode to do it that way.