Most of them work fine. The ones with wire ear supports have the least interference. My Remington glasses are of that type, as are my regular prescription specs, which have safety lenses.
You should also look at the muffs themselves. If the pads are too stiff, you'll have more trouble getting a seal around your glasses. I suspect many of the muffs are designed without considering use with glasses. I have this issue when using some of the el-cheapo muffs. Don't buy that crap. Instead use ear plugs. Then get yourself some decent muffs. Mine happen to be from Dillon Precision. They're old, but good. They're also electronic, which I highly, highly recommend. They allow you to hear a pin drop, but they do an excellent job of clamping down against loud noises. You can hold conversations in a regular voice, hear what's going on in your surroundings, and you're well-protected at the same time.