It doesn't totally break it, but causes any image accompanied by a caption with either a single quote or question mark not to display. Those with single quotes display the typical "image missing" icon; those with question marks present a gray box in the thumbnails and a perpetual Jukebox spinner in the image area.

Gallery was built on a Mac running El Capitan v10.11.2. Problem occurs using either Mac Safari 9.0.2 or Firefox 44.0.