Thank you for providing the link to your gallery's web page.
I have viewed your gallery in Edge, Chrome 67, Firefox 61.0.1, Internet Explorer 11 and Opera 54.0 (on a Windows 10 PC).
The rounded corners on the thumbnails seem stable in all the browsers I've noted above. (I've not yet seen the rounded corners change to rectangular corners in any of these browsers.)
The only problem I encountered was when viewing your gallery in Safari 5.1.7 (the last version of Safari for the PC).
(I do not currently have access to a Mac so cannot view your gallery in the latest version of Safari.)
In Safari 5.1.7, the thumbnails all have rectangular corners (and are stable as such) but this is probably due to Safari 5.1.7 being woefully out-of-date now (released May 2012) and having limited (or no) support for CSS 3. (Apparently, Safari 6.1 and earlier did not apply border-radius correctly to image borders, as noted here.)
My results in Safari 5.1.7 are essentially of no relevance but I include them here simply for the sake of completeness and because Safari is the only browser that you see the problem in (albeit most likely a different version).
Out of interest, what version of Safari are you using?
There are really only a couple of things that spring to mind that I would try myself.
(1) Try resetting your Safari browser (clearing the cache and temporarily disabling all extensions) to see if this helps.
(2) Try validating your web page with the W3C Markup Validator and fix any errors reported.
I notice that your web page has several errors (many are trivial, others not so) and fixing the errors so that code on your web page validates correctly should help your web page to be displayed with greater consistency and predictability across different browsers.
For example, I notice that your web page has some HTML code (loading 6 external JavaScript files) after the closing </html> tag. (This code should be inside either the <head> ... </head> section or the <body> ... </body> section of your web page.)
I hope my notes above help somewhat, even though I'm not currently able to see (or replicate) the problem myself (which makes troubleshooting that much more difficult).