Mount Olivet Cemetery was established in 1856 on former farmland to relieve overcrowding at the Old City Cemetery. It is said to have been modeled after the garden cemetery of Mount Auburn in Cambridge and Watertown, Massachusetts. The 206-acre Mount Olivet Cemetery is located about 2 miles east of downtown Nashville, and adjacent to the Catholic Calvary Cemetery.

Southern aristocracy, including planters, former governors, US senators and representatives, was buried in a separate section from common burials. Enslaved persons sometimes were buried near their owners, and visitors to Nashville were buried alongside paupers.

After the Civil War the women of Nashville purchased an area in the center of Mount Olivet to form Confederate Circle as a place to bury Confederate dead. The remains of about 1,500 soldiers who had died on nearby battlegrounds were moved here, and 7 Confederate generals were buried in or around the circle. These grounds were dedicated in 1869, and a 45-foot granite obelisk was erected in 1889.

Also, after the Civil War, many families removed their dead from the neglected old City Cemetery to the beautiful new Mount Olivet Cemetery.

In the 1870s a Gothic Revival–style chapel was built as an office. A bell at the rear of the chapel tolled for every funeral procession in the cemetery. In 2015 this chapel, listed on the National Register of Historic Places but sadly neglected, burned and was removed.

The cemetery is itself listed on the National Register, and contains many elaborate tombs and other interesting architectural works. Nearly 200,000 graves were known to be in the cemetery in 1988. A federal bankruptcy judge ordered the state to retain control of the cemetery until a buyer could be found. Stewart Enterprises purchased the cemetery in 1994.

Numerous people of note interred here include:

• Oswald Avery, scientist whose experiments proved DNA is the substance that carries genes
• William B. Bate, governor of Tennessee (1883–1887), and Civil War general
• Fannie Battle, Confederate spy and social reformer
• Aaron V. Brown, governor of Tennessee (1845–1847), US Postmaster General (1857–1859)
• George P. Buell, Union Army general
• Joseph Wellington Burns, US congressman and Speaker of the House
• John Catron, US Supreme Court Justice
• Anne Dallas Dudley, women's suffrage activist
• Winfield Dunn, 43rd governor of Tennessee
• Francis Furman, businessman during the Reconstruction era, whose tomb, styled after a Greek temple, is the largest in the cemetery
• Howell Edmunds Jackson, US Senator and Supreme Court Justice
• Hill McAlister, governor of Tennessee (1933–1937)
• Bruce Ryburn Payne, founding president of Peabody College, which is now part of Vanderbilt University
• James E. Rains, Civil War general killed in 1862 Battle of Murfreesboro
• Fred Rose, music publishing executive

Also, numerous mayors of Nashville, architects, clergymen, musicians, and others well known to Nashville and the nation.