The college was founded in 1509 by Sir Richard Sutton, a lawyer. It replaced an old house from the 13th century, and was initially funded by William Smyth. The unusual name of Brasenose college apparently came from a corruption of the words Brazen nose, describing the door handle on the college main gate, that was removed to the high table in the hall of the college after being stolen in the 14th century by a group of ex-students, and placed on the door of a house elsewhere in the city. The college was only able to retrieve the knocker by buying the whole house in the Victorian period. The name could also be a version of Brasenhuis, meaning brewery. The college has had many famous students, including Jeffrey Archer, Michael Palin, and Elias Ashmole.

There is a chestnut tree in Exeter college near the walls of Brasenose, and whenever the tree touches the wall, Exeter will beat Brasenose at rowing.