You could maybe create a custom font (with whatever icons you want to use as characters) with an online app such as IcoMoon.
You could then load the font into your gallery's web page and use your new custom characters in your image titles and captions.
You could use HTML formatting within your image titles and captions (as noted in this FAQ below) to ensure that the correct font is used when required.
How do I add HTML formatting to image captions and titles?
For example, the 'juicebox' font (which contains the icons for the gallery) is already loaded into a gallery page but you could use one of its icons in an image caption as follows:
<caption><![CDATA[This is the Juicebox 'Back Button' icon: <span style="font-family: juicebox;"></span>]]></caption>More information about using IcoMoon in conjunction with Juicebox can be found in the Theming Guide. (It might not be directly related to your query but you might find it interesting.)
I hope this points you in the right direction.