You do not know the reason(s) they avoided him. Your theory is possible but not determined. What was judged is they did not fulfill the law to love your neighbor as yourself.
It was a Levite and a priest that passed on the other side who both should have known the two greatest commandments but didn’t follow the spirit of those laws.
Let’s bring Luke 10:25-37 up for discussion and context:
The Parable of the Good Samaritan
25 On one occasion an expert in the law stood up to test Jesus. “Teacher,” he asked, “what must I do to inherit eternal life?”
26 “What is written in the Law?” he replied. “How do you read it?”
27 He answered, “‘Love the Lord your God with all your heart and with all your soul and with all your strength and with all your mind’[c]; and, ‘Love your neighbor as yourself.’[d]”
28 “You have answered correctly,” Jesus replied. “Do this and you will live.”
29 But he wanted to justify himself, so he asked Jesus, “And who is my neighbor?”
30 In reply Jesus said: “A man was going down from Jerusalem to Jericho, when he was attacked by robbers. They stripped him of his clothes, beat him and went away, leaving him half dead. 31 A priest happened to be going down the same road, and when he saw the man, he passed by on the other side. 32 So too, a Levite, when he came to the place and saw him, passed by on the other side. 33 But a Samaritan, as he traveled, came where the man was; and when he saw him, he took pity on him. 34 He went to him and bandaged his wounds, pouring on oil and wine. Then he put the man on his own donkey, brought him to an inn and took care of him. 35 The next day he took out two denarii[e] and gave them to the innkeeper. ‘Look after him,’ he said, ‘and when I return, I will reimburse you for any extra expense you may have.’
36 “Which of these three do you think was a neighbor to the man who fell into the hands of robbers?”
37 The expert in the law replied, “The one who had mercy on him.”
Jesus told him, “Go and do likewise.”
Notice that the legal expert wished to “justify himself” having felt the sting of the parable. Christ asked him which was the true neighbor and the “expert” answered “the one who had mercy on him.”
Now what was the reason the legal expert felt as though he needed to justify himself? If you studied the Mosaic code...priests and those of Levite class had to avoid anything of a bloody unclean nature that would taint their persons, yet there were never any laws against showing mercy and kindness. They hid behind a technicality to avoid a more selfless calling on their part. The legal expert was gently confronted by Christ regarding his reliance on the technicalities of the law to avoid the spirit of the law...which was the mercy, love, and justice flowing from the Spirit of God.
The “legal expert” quoted the two great commandments from a more narrowly interpreted, perhaps rotely memorized sense of those laws after what was supposed, the priestly order of Aaron. When Christ affirmed that the doing of those 2 laws would lead to eternal life,,,the “expert” realized he had been justly “fingered”. Just like Adam and Eve in the garden of Eden, the “expert” sought to justify himself. Christ’s parable would give the “expert” no cover. Even the “disgusting” Samaritan(Samaritans were estranged from polite Jewish society, kind of like yahoo hillbillies) who showed kindness would get eternal Life over the Priest and Levite. Christ’s parable was directed at the heart of the “expert” hence his use of the priest and levite in the parable.
Likely because they had strict regulations about what makes them unclean and they didn’t want to become unclean.