Ginsberg attended Columbia on a scholarship from the Young Men's Hebrew Association of Paterson, NJ. In 1945, he joined the Merchant Marine to earn money to continue his education.
While at Columbia he edited the Columbia Review, wrote for Jester and was president of the Philolexian Society. He was also winner of the Woodberry Poetry Prize and, reportedly, a Philolexian Prize as well.
He was friends with fellow Beat writer Jack Kerouac and future historian Fritz Stern. He served as Stern's debate partner on the College Debate Council and founded the Roosevelt for President club with him in 1944. Ginsberg was to prove influential in persuading Stern to enroll in humanities classes, which caused him to rethink his intended career in medicine.
He was placed on disciplinary suspension for one year from the College after writing the graffiti "Butler has no balls" (referring to University President Nicholas Murray Butler) in the grime of his dorm room window in Hartley Hall.