Interesting that you should "imagine" that....A one-act hit play by Douglass Turner Ward called "Day of Absence" posited just such a scenario -- all black people just disappearing from this racist land. It was done off-off Broadway in 1965. https://en.wikipedia.org/wiki/Day_of_Absence

Also, from the very first moment Africans were forced upon these shores, there has always been a "back to Africa" movement. Check out Paul Cuffee (sp?), Henry Highland Garnet, Dr. Dubois, Marcus Garvey and a century-long Pan Africa movment (among many others). Garvey's Universal Negro Improvement Assn. actually bought a ship, christened it "The Frederick Douglass," and made the crossing back to Africa (Liberia). I think there were about 300-plus blacks who sailed on that vessel.