Time for some experimentation!
Does it happen when your compressor is off?
If you dial in only gentle compression and threshold values does is still happen?
Is your compressor placed before or after the distortion? Try changing the order of the chain.
Can you tell I think your compressor is causing the problem?
One of the side effects of compressors is that they can increase the overall noise floor if set aggressively (i.e. they can turn apparent silence into a wall of noise). That is why a lot of compressors also have a built in noise gate - to stop the 'silence' turning into noise that can cause feedback.
Most distortion effects compress the signal pretty heavily on their own. Running a distortion pedal AND a compressor means you're probably getting a double dose of compression. I'd be surprised if the distortion sounded better with additional compression.