Understand, I'm not trying to convince (or "proselytize"? :)) you or anyone else either way. There're some language families, like Indo-European, where the correspondences are so clear and systematic that no one seriously questions the genetic relationship among the component languages. By contrast, the Altaic family is still often referred to as the "Altaic hypothesis," since it is an "umbrella" grouping of languages that are definitely related (Turkic languages), languages that are pretty clearly related (Mongolian languages like Khalkha, Buryat, Kalmyk), and languages that are very likely related (so-called "Tungusic languages," such as Evenki, Oroqen, and the extinct Manchu). The objections to this grouping center on claims that vocabulary correspondences between the groups are due to borrowing, and not to derivation from a "Proto-Altaic" parent language.
If Korean is part of the Altaic family, it is due to plausible affinities between it and the "Tungusic" branch of Altaic, in the main. And a large part of the case for Japanese as an Altaic language are its plausible links with Korean. Lot of room for differences of opinion.
But not as much as with the proposed "Eurasiatic" (often called "Nostratic"--although that term is eschewed now because it derives from Latin nostra--"our" languages---"exclusionary," y'know...) linguistic family, which would incorporate Indo-European, Altaic, and several other language families. Or in turn, that "Eurasiatic" super-family's possible affinity---even deeper in chronological time depth, obviously---with the "Amerind" languages, i.e., most of the languages of the Americas, except for Eskimo-Aleut (part of the "Eurasiatic" family under that hypothesis), and the Na-Dene languages like Navajo (ultimately related, according to an extremely imaginative and by no means established hypothesis, to languages like Chinese).