I have both a Gen 2 and a Gen 4.
The machining and plastic molding on the Gen 4 is nicer. The Gen 2s tend to have more tooling marks and molding flaws from what I have seen. Also, the grip is a little thinner on the Gen 4. The gen 4 has a reversible mag catch but I'm a lefty and i don't even bother swapping it. The gen 4 also has a snazzy double spring guide rod that is suppose to help with felt recoil but I've never noticed a difference.
Advantages for the Gen 2 is no finger groves if you don't like those and a slightly more crisp trigger (at least when I compare my copy to my gen 4)
Finally, make sure your Gen 2 isn't on the recall list or make sure your Gen 4 is of the latest manufacture that has the updated ejector and recoil spring. Glock perfection aside...
It's a toss up honestly. If you go for a used gen 2, make sure to look it over. I bought mine that had only been shot once, oiled, and put into storage for years. The firing pin block was stuck open with old oil and needed a good cleaning in order to get the gun into a condition where it was safe to fire and carry.