The Visitor's Center is located in 213 Low Library. It's immediately to the left of the bust of Athena when you enter.
All tour's of the university originate at the Visitor's Center. Though technically a 'University' visitor's center, it's clearly geared towards prospective undergraduates. The desk is generally manned by undergraduates, and Columbia College and SEAS bulletins are strewn liberally around the room. There are no GS bulletins.
Of interest to current students are the bits of Columbia history displayed around the room. Old badges, pins, yearbooks, postcards, pamphlets etc. give the room a feel of tradition and history that's more or less lacking in any actual student space on campus that isn't some administrators office. Interestingly there is a substantial amount of old GS and JTS paraphernalia on display.
Information for tours can be found at the Visitor Info website.
The Visitors Center was deliberately placed in Low Library by President Rupp in 1993 to integrate the building into Columbia life beyond just serving as an administrative hive.