Possibly, but there are a number of different variations on color vision among humans. The most common one may be as you describe. The interesting thng is that color deficiency varies with luminosity. Color blind people can often distinguish colors better in bright light.
That makes sense. With greater illumination, there may be some enhancement in the stimulation of the photoreceptors, a decrease in the signal-to-noise ratio or even saturation of one receptor compared to the other. All three may affect color contrast. Also, don't forget that color is a matter of perception in the brain and not an actual physical property of a material.