He didn't only "vote against it", Al Gore Sr. was the ringleader AGAINST voting for the 1964 Civil Rights Act.
Al Gore Sr., together with the rest of the southern Democrats, voted against the Civil Rights Act of 1964.
Gore attempted to send the Act to the Senate Judiciary Committee with an amendment to say "in defiance of a court desegregation order, federal funds could not be held from any school districts." The Gore Amendment was defeated by a vote of 74-25.
Later, Al Gore Sr. apologized for his 1964 vote and called it a "big mistake". Sure, it was just a "mistake"