William Henry Whitaker arrived in Sarasota Bay in the early 1840s. He built a cedar log cabin at “Yellow Bluffs,” part of nearly 200 acres he would acquire along the bay between Hog Creek and today's Indian Beach Road. Initially a fisherman, Whitaker later added cattle and a citrus grove to his homestead.

Whitaker married Mary Jane Wyatt from Manatee in 1851. They raised eight children to adulthood. During the Third Seminole War, Whitaker joined a local militia unit. After their home was burned by the Seminoles, the Whitakers built their second home adjacent to the family cemetery on today's 12th Street, east of the Tamiami Trail.

During the Civil War, Whitaker's growing cattle herd provided beef for the Confederate forces. His family experienced hardships from the Union naval blockade and occasional raids for food. After the war, Whitaker helped facilitate the escape of Confederate Secretary of State, Judah P. Benjamin, from Sarasota Bay to the Bahamas.

The Whitakers and other early settlers provided food and shelter for the colonists who arrived from Scotland in 1885 to create a new town of Sarasota.