I often remove my glasses when listening to music only or sleeping. There's no appreciable difference. Of course, the type of eyeglass temple piece may make a difference; mine are titanium frames with relatively thin wire.