On the contrary, the legions were absolutely critical to the existence of the Roman Empire. Roman diplomacy relied upon a continuous aura of strength and intimidation to secure its boarders against the various peoples surrounding it. The interior of the empire was far richer than the barbarians surrounding it and it would be raided and dismembered at any show of weakness. As the crisis of the 3rd century and the collapse in 5th showed, when the Roman Empire displayed weakness it was attacked by virtually all its neighbours.
For more detail on Roman diplomacy I strongly recommend Rome and the Enemy: Imperial Strategy in the Principate by Susan Mattern.
There is a point that these "displays of weakness" were usually caused by the very thing that was supposed to keep the empire safe: IE: the Legions.