No difference at all, except that ‘colored person’ is the old fashioned way to refer to black people, while ‘person of color is a more modern term, which also includes other minorities, such as latino, American Indian, arab and asian.
IOW, non-whites.