A Flickr description such as the following should work fine:
Text <a href="http://www.example.com/">Link</a>
I have just created a test gallery using the above line as a Flickr description in one of my images.
The gallery uses exactly the same configuration options as your gallery (other than your Flickr entries) and the link in the caption display and functions correctly.
Check the CSS on your page to make sure that there are no generalized CSS rules which your gallery will inherit.
I notice in your 'bootstrap.css' page that there are CSS rules that apply to all <a> tags on your page, including those in your gallery. Check your other CSS files for similar entries and try temporarily removing them to see if it makes a difference.
Also, can I change the Juicebox display so that captions will show only if provided while navigation will only show upon rollover?
If you position your captions anywhere other than on the overlay, (e.g. BELOW_IMAGE, BOTTOM or BELOW THUMBS), then they will be always be displayed whereas the main image navigation buttons (which are on the overlay) will still be displayed only when the user hovers over the gallery.