When a theme is switched over to another theme, first time when the page is loaded, the refresh button appears (as the widget will be empty). Later after re-loading the page, this button appears only if the widget won't be loaded asynchronously. This is the normal behavior of this widget and is not an issue. Tested the same in all the browsers (IE 7, 8, Mozilla & Chrome) changing all the themes & also by switching to ‘No Theme’.
More Like This