The icons within a Juicebox gallery are, indeed, characters in a custom font.
The font files are stored inside the gallery's 'jbcore' folder in this location: jbcore/classic/fonts/
Inside the 'fonts' folder should be the following files:
juicebox.eot
juicebox.svg
juicebox.ttf
juicebox.woff
Make sure that all these font files are in the correct location on your web server (just reinstate your gallery's entire 'jbcore' folder with a fresh version from the Juicebox-Pro zip package) and also that the permissions of these files (and their parent folders) are not too restrictive. Default permissions of 644 for files and 755 for folders should be fine. You should be able to check and change file and folder permissions with a dedicated FTP program (such as Filezilla) or via your web hosting account's online file manager.
Hopefully the suggestions above will help to resolve your problem. However, If you continue to experience difficulties, then please ask your web host to check the MIME types for these font files on your web server.