The school was founded in 1897 and was then known as the Oklahoma Colored Agricultural and Normal University. It is the only historically black college in the state of Oklahoma and the westernmost historically black college in the United States.

Langston University is named for John Mercer Langston (1829–1897), civil rights pioneer, first African American member of Congress from Virginia, founder of the Howard University Law School, and American consul-general to Haiti. It was renamed Langston University in 1941.

John Mercer Langston is the great, great uncle of Langston Hughes.