Yes, it was caused by the glasses...or most likely not looking through the same place on the lens every time.
W/a M-14 and the M4 there is such a thing called, "Spot-weld." W/the M-14 it is putting your cheekbone against your thumb, and w/the M4 it is putting your nose against the charging handle. The purpose of it is to have a set reference point so that your eye, which is a lens, will be located in the same place each time. If not, then you will see the sights differently and your groups will suffer.
I suspect that: A-you had a poor spot-weld and that you were either closer, up or down (also very important) in reference to your scope or sights or B-looking through your glasses differently, probably w/your head in the same place, but w/a minute difference in the way that your head was tilted, so that you are not looking through the same part of the lens each time or C-All of the above.
This is common, for me, shooting a pistol in the kneeling position w/my elbow on my knee. This causes the head to tilt forward, changing the focal points and part of the eye that is seeing the sights (very important to look at the sights w/the same part of your iris...which is a lens). The result is that I'd still shoot a tennis ball sized group, but it would be off the center of the target and usually diagonally. And in the prone position things can really get messed up (w/a pistol).
In short, I suspect that you were looking at your sights w/a different part of the lens of your eye, which would also be looking through a dif portion of your glasses.