You're not supposed to have levies raised unless you're using them; disbanding levies in peace time will drastically reduce the maintenance cost of your armies (since you'll only be paying for your retinue + mercs).
Additionally, I believe vassals contribute a drastically reduced (maybe zero?) percentage of their income if they aren't making a whole lot to begin with. It's to encourage you to spread your kokujin titles out instead of giving only one to each vassal.