This is by design. The 'Toggle Info' button toggles the visibility of all elements in the OVERLAY (as defined in the Button Bar Options section of the Juicebox Configuration Options page.
You could either set the position of your Back Button to 'TOP' or overlay it on top of the gallery (but not as part of the gallery) as you have done with your SimpleViewer galleries.
As this is not technically a bug, it might be wise to post your suggestion in the 'Feature Requests' thread so that it is not overlooked. Thanks.

I see these two issues, too (Win 7 SP1 x64).
I do not know if they can be fixed but I will log a bug report nonetheless.
Thank you for reporting.

This is due to the default setting of enableKeyboardControls="TRUE".
If you click on the browser's address bar (to take focus away from the gallery), the F5 key becomes active again.
If you set enableKeyboardControls="FALSE", then the F5 key is active, even when the user has been clicking in the gallery area.
I will try to find out whether the F5 key can remain active for both values of enableKeyboardControls in a future version.

This is by design. Small Screen Mode does not support color options.

You have two instances of Juicebox on your page:

        new juicebox({
            containerid : 'juicebox-container'
        });

... starting at line 49 and:

new juicebox({
containerId : "juicebox-container",
backgroundColor: "#FFFFFF"
});

... starting at line 93.
Both instances are being embedded into the same <div> and the first one is momentarily displayed before the second one replaces it. Because the first one has no backgroundColor explicitly specified, it will use the default color of '#222222' which is what you are seeing as flashing.
Use only one instance and delete the other. If you choose to use the first instance, add the backgroundColor : '#ffffff' parameter to it and leave the following code on your page where the second instance currently is:

<div id="juicebox-content">
    <div id="juicebox-container"></div>
</div>