Peleg Wadsworth, a Revolutionary war veteran, built a brick house in Portland on what was then known as Back Street (now Congress Street) in 1785-86. He and his wife, Elizabeth, raised 10 children in the house. The house is shown in a photograph taken on January 4, 1904.

Later, his daughter Zilpah and her husband, Stephen Longfellow, raised their family in the house. In 1815, they added a third story. One of their children was the poet Henry Wadsworth Longfellow. Their daughter Anne Longfellow Pierce lived in the house for many years and donated it in 1901 to the Maine Historical Society.