If you read the OP you would see that I do suggest that.
In the system you propose Frisian and Bavarian would be a part of the same German language group and won't get any penalties. But in real life Frisian and Bavarian dialects are mutually unintelligible (as are many dialects of Scandinavian languages, Italian, and even English before Shakespeare), so people speaking such dialects were unable to hold conversation. If we apply language penalties when people are unable to communicate, surely we should apply them for distant dialects as well? But it won't happen in your system.