King Arthur of Camelot mate. This dude got a sword out of stone (or the lady in the lake version) and was made king. Think of dragons as Excalibur. Those with dragons typically rule, and one step down are those with valyrian steel swords, which happen to be old houses and/or powerful houses.
Ageon was met with dissension and was not a king, but a conqueror to many. You don't need their approval when you've got something that is better than words. Fire and steel is what keeps the kingdoms together or in the words of Robert Baratheon:
"One king, Seven Kingdoms! Do you think honor keeps them in line? Do you think it's honor that's keeping the peace? It's fear! Fear and blood!"
That being said, a valyrian steel sword such as Blackfyre would hold power, and not just physically.