St. Peter's Episcopal Church traces its roots back to the Puritan Colony of Milford, when religious freedom and tolerance were evolving concepts in the colonies. In September 1764, Milford's first Church of England communion service was conducted. A year later, the Milford Episcopal Society attracted the attention of St. George Talbot of Westchester County, who had pledged his money and time to promoting the Church of England beyond New York City.

With Talbot's backing, the Society secured a deed from the town of Milford on January 2, 1765 for land on which to construct a church. Actual construction of the church did not begin on the site of the present St. Peter's Church until 1769. Named St. George's in honor of Talbot, the building's exterior was completed in 1772, the floor was put in place in 1774, and pews were added in 1775. Because there was no Colonial Bishop, the church could not be consecrated, but it was "set apart" and dedicated for Divine Service with solemn ceremonies in March 1775.

When the colonies began their struggle for independence from England, the church entered a period of uncertainty and dormancy. About three weeks after the Declaration of Independence was signed, the Connecticut clergy met and decided to close the Anglican churches. Parish records state: "From 1776 until 1786 there was seldom an assembling in the Church either for Prayers or Preaching; nor any new choice of Church Officers."

St. George's remained in a state of uncertainty, with financial problems for the half-century following the end of the Revolutionary War. Few records have been discovered to document activities from 1788 to 1805. It wasn't until the years of James Dixon Carder as rector from 1848 to 1861 that a transformation began. In 1849 the original wooden church was demolished and work began on the brownstone church that stands today. When the work was completed, an octave of services was scheduled beginning with the Feast of St. Peter on June 29, 1851. The festivities climaxed on July 2 with Thomas Church Brownell, Bishop of Connecticut, consecrating it "by the name of St. Peter's Church." The name change was formally adopted by the parish at its annual meeting in April, 1852 and authorized by the state's General Assembly in its May, 1852 session.