All these men had pretty low IQs. I am suspicious of the man that was on the side of the road. I think he scared them somehow, and they took off running.
They did have developmental issues of various degrees. Some were more functional than others. Some were probably not properly diagnosis at the time. Jack Madruga was the owner and diver of the car. Based on what I read about him he was mainly just very shy and withdrawn. Unfortunately, the man on who encountered them on the road was not a reliable witness. He changed his story more than once and he had a reputation for not being truthful. It does seem possible he had something do with them leaving the car and walking miles into the wilderness.