Abel Hildreth, born in 1793, built the Hildreth Mansion as a home for himself and his family, probably in the 1820s. He was a carpenter and joiner and built the first toll bridge over the Georges River at the foot of Wadsworth Street in 1818.

The house still stands on its original site on the north side of Route One near the Mill River. The house is also referred to as the Benner house, after Mrs. Florence Benner, who lived in the house for a time.