I don't think Belafonte was popular with white audiences because he was (in Johnny Carson's terms) "non-threatening." The Petula Clark incident you cite shows that any Black person (particularly men) could be seen as "threatening" given the systemic racism of the era. My opinion is that in his music and elsewhere Belafonte embodied authenticity to such a degree that he could overcome stereotypes and racist expectations.
BTW, as a thrift store shopper, I occasionally come across (and purchase) the occasional Belafonte album, like the Carnegie Hall performance. In general they seem to have been played and stored with great reverence.
Thank you for this important profile.