You've got it exactly the wrong way round.First of all, Vietnam was devided in 1954 due to the Geneva-Conference which ended the first Indochina-War
https://en.wikipedia.org/wiki/1954_Geneva_Conference
https://en.wikipedia.org/wiki/1954_Geneva_Conference#Provisions
https://en.wikipedia.org/wiki/State_of_Vietnam_referendum,_1955
The French-backed State of Vietnam, led by former Emperor Bảo Đại, provisionally held control south of the 17th parallel. Hồ Chí Minh's Viet Minh held the north under the Democratic Republic of Vietnam, which Hồ Chí Minh had proclaimed in 1945. The agreements stated that nationwide elections were to be held in 1956 to unify the country under a common government.
It was clear to all that the Vietminh would probably win those elections planned for 1956. So, Diem had a "Referendum" in 1955 and tereafter formed the Republic of Vietnam.
The "Referendum" was a total fraud
Though published counts showed Diệm winning the election with 98.2% of the vote, the referendum was widely marred by electoral fraud. In the capital, Saigon, Diệm was credited with more than 600,000 votes, although only 450,000 people were on the electoral roll.[1][2] He accumulated tallies in excess of 90% of the registered voters, even in rural regions where opposition groups prevented voting.
But, North Vietnam had no territory to reclaim in the South, since it never had legal control over vietnamese territory South of the 17th parallel in the first Place.
The failure of reunification led to the creation of the National Liberation Front (better known as the Viet Cong) by Ho Chi Minh's government. They were closely aided by the Vietnam People's Army (VPA) of the North, also known as the North Vietnamese Army. The result was the Vietnam War.
So, the presence of NVA-troops on South vietnamese soil was not justified. The US presence, on the other hand, was justified since the South vietnamese goverment had invited the US forces ( same as with russian troops in Syria today).
The US presence was legal, the North Vietnamese presence illegal.
There was only one Vietnam. This was true before the war of liberation against the French and it did not change due to the Geneva Accords, which only provided for a temporary administrative division, not a separation into two sovereign states. Crossing internal administrative lines is not an invasion. Especially not when the Vietminh only crossed after the other side broke the agreement that created the administrative line in the first place.
The South-Vietnamese regime that broke the agreement was not the government of a sovereign state but only one party in a civil war they could have avoided entirely by sticking to the agreement. A rebel group has no right to invite in a foreign power.
The best argument against the Vietminh is that it, too, was only a party in a civil war. It had, after all, not been recognized by the departing French or most of the international community. That would mean that neither party held sovereignty and neither had legal right to invite foreign participants. However, unlike the RoV, it could credibly claim the mantle of national liberation movement in light of its achievements against the French and Japanese. Moreover, the Vietminh did not abrogate the Geneva Accords, instead it reacted to the other side putting into place a policy that entailed a permanent rupture of the nation.