By the mid-1800's, the congregation of St. Mary's Church had outgrown their original building, and a new church was needed.
New St. Mary's Church, located across the churchyard from the old church, was designed by Richard Upjohn, architect of Trinity Church in New York. Built between 1846 and 1854 at a cost of $20,000, it is one of the first and best examples of the gothic revival style in American churches.
The church bells were cast in London and placed in the steeple, more than 160 feet high, in 1866. To this day, they are rung by hand. The church was gutted by a fire in 1976, leaving nothing but the stone walls standing. Services were held in Old St. Mary's while the interior of the new church was rebuilt.