The Japanese were already in the war years before Pearl Harbor. The Rape of Nanking was 1937-38. The Japanese invaded and occupied Viet Nam before Pearl Harbor.
The "neutral" position appears to be in reference to America and the European theatre, not what attrocities Japan was committing in Asia