The Congressional Black Caucus has refused to meet with President Trump because it says he has ignored its policy recommendations for the black community.