And that is perfectly fine because those who do want will reproduce more and soon your gene's will be gone.

Less well represented as a percentage doesn't mean those genes will die out.

It does if the trend continues. Yay math.

With a growing population, a more slowly growing subpopulation can nevertheless be a smaller and smaller percentage.

But the growth would also have to continue. In all real populations, population size has an upper limit(see http://en.wikipedia.org/wiki/Carrying_capacity.) Of course, it's easy to forget that if you think about humans because it appears that technology has increased the carrying capacity for humans to a yet-unknown level. It's true that humans are going through a population explosion right now, but of course at some point we'll hit an upper limit.