In general, in shallow water (depths down to roughly 100m) sonar will be particularly affected by background noise (for active and passive sonar) and by reverberations (active only). Background noise is created by a range of organic and inorganic sources, ranging from the sea or seabed itself (ambient noise created by turbulence, effect of wind on the sea, etc), fish or other shipping or mechanical activity nearby, all covering a large range of frequencies. All this noise significantly reduces the effectiveness of passive sonar (the noise sources you want to detect are obliterated by the background noise) and active sonar (your sonar returns are similarly obliterated by the background noise). Reverberations occur on active sonar and represent the mass of unwanted echoes and backscattered sound from the seabed, particles suspended in water or the sea surface, and in shallow water the seabed clearly plays an increased role.
All this reduces the effectiveness of both active and passive sonar for surface ships and submarines in shallow water and so reduces the range at which you will detect objects. In terms of modelling, it is virtually impossible to do accurately as there is so much variability in the sources of this background noise in shallow water. The problem is some noise signatures are still audible despite all this background noise and that varies with classes and even individual ships, meaning they would be detected if the enemy had previously fingerprinted that ship and knew what they were looking for (and such information obviously wouldn't be publicly available so there is no way you'd know what ships were noisy and which weren't!). We haven't even got in to the effect of sound propagation in shallow water which
asdjames touched on and which varies for different times of the day, different seasons and different geographic areas.
Basically, and not very helpfully I'm afraid, all this makes it really hard to get an accurate sonar model in the game for shallow water, so I'd suggest a general reduction in detection ranges on active and passive sonar for both ships and submarines in shallow water (roughly less than 100m depth). Although it isn't going to be a perfect model (which is effectively impossible to achieve), it will represent the difficulties of operating sonar in shallow water without giving an advantage to either ships or submarines for gameplay.
Edit: Oh and I'll ask some 'friends' about detection of subs near the continental shelf or other areas where it gets deeper/shallower suddenly and get back to you
