The Japanese are really big on not losing face. They are too far in to get out now.
I haven’t seen a specific poll, but apparently the Japanese people overall do not support holding them.