Moving your capital, which is what it loos like you did, is different from the decision to move your de jure capital. If you take the decision to move your duchy's de jure capital, it should destroy any pre-existing duchy buildings. So, you either didn't take the decision or its a bug.