I tried something like that but I just wound up with a box of left socks so I had to throw them out and start over.
Now it doesnt make a difference what color they are, I match them by thickness.
.
That's how I match my nighttime fleece lined slipper socks - thickness.
Gotta keep my feet warm at night, ya know...