This has happened to me, except it was a rebel army rather than a navy.
Basically what happened was I declared war on the macedonian rebels to exploit their civil war and I occupied a bunch of the rebel provinces. When Macedonia took over the last rebel province, thereby ending the civil war, a stack of rebel troops was busy sieging one of the provinces I had occupied. When the civil war ended, all the provinces I had occupied became mine, but that one stack of rebels stayed and sat there for the rest of the game.
Funnily enough, the stack not only stayed around after the macedonian rebels ceased to exist, it also stuck around later in the game after Macedonia itself ceased to exist.
This bug doesn't break anything in the game fortunately, you just end up with this odd stack that should not exist sitting their doing nothing. I have a screen shot I can send to you once I get home if that would help.