The U.S. population is getting older and more diverse, according to numbers released Thursday by the Census Bureau.