That's true, court of justice is a silver bullet against any form of organized criminal activity (helps in preventing and removing it). But even more critical than having a court of justice in a province (which just makes occurrence of thieves/robbers/smugglers less probable and removing faster), is to keep a regiment unmobilized in that province you have a problem in. Without this you can't broke the guild/ring/band.
Another tip is to avoid falling with your efficiency value to 50% or below - apart other negative consequences, you may have a "Major Criminal Gang Established" event launched which means you get all three types of organized criminal groups into a province in a single blow. (In fact, it's wise to keep your efficiency at least above 80%.)