Unity avatar integration not always loading avatar configuration

I'm having an issue where sometimes the avatar configuration based on a supplied user ID fails to load properly and defaults to the standard grey mesh avatar. Its rare but happens enough that its an issue. My best guess is that there was a timeout in the reply from Oculus and the fallback is the default avatar? Is there any way to ensure that the correct load out has been retrieved or reattempt it if it hasn't?