Ray Ban Displays - Audio auto play

Been using the Ray Ban Displays as a teleprompter the last two weeks as I speak at church. They have been amazing, the only thing is occasionally the glasses will start to play audio from my phone (podcasts). Am I missing something? I have turned off auto pause and even disconnected my phone. I feel like it's a bluetooth thing, but I'm not sure what to do. Any suggestions? Since I was wearing a microphone, I had to take the glasses off mid sermon because the audio was playing through.