I just don't think what people call themselves matters that much.
Oh, but it matters quite a lot. If English colonists in America called themselves "English" it'd be a bit strange to insist on calling them "French", because *reasons*.
It is beyond any doubt that people living in the ERE called themselves "Roman". And it's not that they suddenly started larping as Romans in, let's say, X century. No - it goes back to the original, undivided "classical" Roman Empire. And then it just continues, uninterrupted, to the XX century.
All free peoples in the former empire were Roman after edict of Caracalla. In 500 AD there were Romans in Gaul, Spain, Italy, Thrace, Egypt and Africa. People still existed en masse. But there was one extremely important difference there - only in the ERE the continuity of the statehood was uninterrupted. Germanic kingdoms like the kingdom of Ostrogoths, Visigoths, Vandals or Franks were not "Roman", because their rulers were not Roman. They were imitating Roman style, but they were not Romans themselves, but foreign invaders (who did not came alone but with hundreds of thousands of their people) who founded their own states. In time their subjects intermarried and also stopped considering themselves Romans as new cultures emerged.
At the same time in ERE Roman state still existed. Nothing changed since the "classical Rome". Roman emperor was sitting on his throne in Constantinople. Be it VIth, VIIIth, Xth, XIIIth century - he's still there. Dynasties changed like in all monarchies, but the Roman Emperor was still there, all the time. Roman bureaucracy was still working. Roman law was still enforced. For people living in ERE nothing really changed in 476 AD.
Why its citizens shouldn't call themselves "Romans" if their fathers, grandfathers, great-grandfathers and so on called themselves like that since the reign of emperor Caracalla? Just because they spoke Greek? Majority of Roman citizens in 212 AD did not speak Latin. The empire was so huge and so multicultural the true "Romans" (Latins from the city of Rome) were always a minority. And no one has any problems with that when talking about the empire of Diocletian or Constantine the Great, but suddenly it becomes a huge problem when it comes to the empire of Palaiologos dynasty.
It's the same thing over and over again - it's not that there's something objectively wrong with the ERE or the "Byzantine" empire. It's not that there are some objective doubts about their legitimacy, their legacy and their uninterrupted existence. Some people, for some strange reason, simply will do anything, any mental flips and gymnastics just to never admit they were Romans and their empire was Roman. And if so - they will stubbornly look at the literal Roman Empire and say "no, it's not a Roman empire". We have holocaust deniers, we have flat-earthers, we have Moon landing deniers - and we have ERE-deniers. It's the same. The evidence is there, it's easily available, it points out at the only logical and true option - "Byzantine Empire" was Roman and its citizens were Romans - but they will always say "no", no matter what.
Rome is a city. A Roman Empire is an Empire of Rome. The moment the Byzantines lost Rome they lost the legitimacy of the Empire.
Just because you say so?
Why should its legitimacy be determined by the ownership of the city which stopped being its capital long time ago and wasn't even part of its administrative division?
Imagine this: there's a powerful state, the most developed one on the entire continent. Immensely weatlhy, its culture is unmatched. Traditions, laws, monuments dating back to times of Constantine and older. Uninterrupted continuity. It was THE Roman Empire for generations and now some guy comes around and says
"welp, we don't control the city of Rome directly, I guess we're something completely different now. Let's call ourselves "Byzantines" now."
You say that the people of the Byzantine called themselves Roman. What did the people of the actual city of Rome call themselves?
And you really think it has any impact on how the citizens of the Roman Empire should call themselves? Really?
What about Roman citizens who lived in Constantinople because they followed the court of Constantine there? Did they lose their "Roman" status because they were no longer living in the city of Rome? According to your "logic" every Roman citizen should automatically stop considering himself Roman if they moved outsie the city of Rome.
And what about the people of Latium? The core of the Roman empire? What about the people of Veii, Tusculum, Ostia or Praeneste, all within 50 km from Rome? They weren't Romans because they were not living in the city of Rome?
Seriously, this whole
"but only people of the city of Rome are Romans" is so naive... Yes, they are Romans. And yes - there were Romans living outside Rome. Plenty of them. In the fourth century more Roman citizens lived outside the city of Rome than in it.
As it was already pointed out in this thread - there are other "Romes" in the world. There is even a tiny village called like that in Poland. And you know what? They call themselves Romans. Because that's how they are called, it's absolutely normal. But... so what? It has literally no impact on the ERE legitimacy, just like the "bUt oNlY tHe pEoPlE oF rOmE" doesn't. At all. It's semantics, nothing more and has no impact on the reality of medieval states.
How can a Empire of the Romans not rule over the only ones who are indisputably Roman?
It's extremely simple - by ruling over those who were also indisputably Roman.