The original house at 1 Wharf Lane consisted of only half of the house now standing there, the half closest to the river. In 1814 the other half was added to form its present configuration as a twin chimney colonial. The original house was probably built between 1742 and 1778 by John Swett III , the ferryman.
The name of John Swett Jr. (Sweat, Sweet), born in 1676, appears often in the early history of the village that grew up around “Holt’s Rocks”. Swett established a ferry across the river there in 1711. The town of Haverhill gave him permission to operate it for 40 years. He also had permission to operate a tavern, one of only two in the entire town. By some accounts there were only two houses in the village at that time. He died in 1725, leaving the ferry and his lands in Haverhill to his son John Swett III, and his homestead and other lands in Newbury to his son Benjamin Swett.
John Swett III (1699-1778) operated the ferry on the Haverhill side while his brother Benjamin ran it on the Newbury side. John purchased much of the land in the village from Zacharia Davis and his heirs, who had purchased it from William Holdridge who had been granted the Division 3 district land from the town of Haverhill in the late 1600’s. Phineas Nichol, who lived on River Rd, in later years said that in 1750 there were only four houses in the village, those of Joseph Burrill, Dr. Simeon Brown, John Swett, and his father, Joseph Nichols. Apparently before that time, the area was considered too dangerous for settlement due to the Indian presence.
The land Swett purchased in 1730 from the Davis family was 35 acres on the south side of E. Main St.:
Deed 60/159, 2/16/1731: Samuel Batchelder of Hampton, cordwainer, and wife Elizabeth (Davis) to John Swett 35 acres bounded by se on river, sw on 12-rod way, nw on road to Almsbury, ne by a highway by the side of Swett’s land. [This seems to be s of E. Main from 12 rod way to Wharf Lane.]
Swett owned land on both sides of E. Main in 1730 but sold the north side property in 1742. It is assumed he then lived on the south side. When John Swett died, he willed both the ferry and his dwelling house and barn to his grandson James White, along with 16 acres of land. In 1789 James White sold the house with one acre of land behind it to Dolly Ingersoll:
Deed 150/213 signed 9/23/1789 James White to Dolly Ingersoll singlewoman, house (where I now dwell) and barn on one acre beginning at a 2 1/2 rod way, e on a highway 8 rods to s&s then s by flats of river 18 rods to s&s , w on my land 10? rods 1/2 to s&s then n on my land 4 rods to s&s , then on a 2 rod way 4 rods then e half a rod to s&s, n 8 rods to s&s at start. Also acquit all the flats lying between the above land and the river.
Dolly Ingersoll’s brother, Zebulon, had been living in the village on River Road since 1782, at 21 River Rd, a house he had bought from Phineas Nichols. Dolly married Enoch Greenleaf of Newburyport in 1789, a few days after she bought the house (see deed above). A list of households of 1798 shows her still as head of the household with her brother Zebulon as a member of her household probably because Dolly retained ownership until 1803. The speculation is that Dolly moved with her new husband to Newburyport in 1789, leaving Zebulon and his family living at 1 Wharf. In 1803 Zebulon bought the house and land from Dolly :
Deed 173/67, signed 10/1/1803: Dolly Greenleaf a widow of Newburyport sells to Zebulon Ingersoll house with 1 acre of land, same that I bought of James White begin on 2 rod way run east on highway 8R, s by flats of river 19 R, w on Nathaniel Ladd land 10.5 R, n on Daniel Swan land 4 R, then on 2-rod way 4 R, e half a rod, n 8R to start.
The acre of land that originally accompanied the house is thought to have extended east-west from Wharf Lane to the river and north-south from the “Landing” back to include what is now #20 Wharf Lane. As was the custom in those days, the house was situated right on the street, in this case known as the Town Landing.
To return to Zebulon, he had fought in the Revolutionary War as a young man. In 1777 at age 18 he enlisted for three years in Capt. Samuel Carr’s company, Col. James Wesson’s regiment. Samuel Carr was from Newbury. The Sawyer-Carr house, probably his, is on Main St. in West Newbury on the left just before the library and the old Training Field.
Zebulon had married Ruth Moody, daughter of Benjamin Moody and granddaughter of Deacon Caleb Moody (see 1730 entry) and widow of Moses Pike. The Ingersolls began raising their family, producing six children in ten years. John Ingersoll was the second to youngest and the only male child, born in 1788.
Zebulon ran the brick store across Swett’s Lane until he sold it in 1815 to John Foot. May 13, 1800’s edition of the Newburyport Herald, states that items stolen from his store included Muslin handkerchiefs, fur hats, calico, and other cloth goods.
Zebulon packed fish and pickled them in barrels to be sent down the river in barges to Newburyport to be sold. There were wharves along the river and a cartway that ran in front of the house down to the river. Zebulon was a member of the organization that managed the bridge and collected the tolls. 
The photograph above must have been taken after 1826 since that’s when photography first appeared. But it is strange that it is labeled “Uncle and Aunt Johnson’s House”. Johnson was certainly living there in 1850 (Haverhill census). Since Johnson was married to Ruth Ingersoll, any of the children of Ruth’s sisters would have referred to her as ‘aunt’. The same is true of John Rollins’ children since he married Johnson’s sister, Betsy, although his only known offspring, John Johnson Rollins died in 1841. (And Betsy had died in 1820). Notice that the original chimney on the old half of the house has been removed, and the house on the 4-8 Wharf Lane land is present. It’s also possible that the Johnsons weren’t living in the house at the time the photo was taken, despite the labeling. In 1860 Johnson sold the house to his granddaughter Abby McQuesten (and he died in 1861). Indoor plumbing began in the U.S. in the 1840’s but took over a century to become widespread. It is a drastic measure to remove a chimney, as it was a structural member, but it was clearly done to make room for bathrooms and closets.
1804: from John’s day book: “Miss Pattee was at our hous today making me a great coat” and ” Mrs. Pattee was at our house making me a pair of pantaloons of light Kerseymere.” (21)
1804: From John’s Day book: “Sir had E&S Coleby to work for him trimming apple trees and ditching”, and ” We had Mr. Coleby and men to work for us all day at planting”. (21)
The Ingersoll girls all married locally, but Zebulon took on the responsibility of setting up his only son. He acquired several plots of land over the years (woodlots, saltmarsh, etc.), some of which he sold to John. One parcel is now the number 10 Wharf Lane property.
John taught school in the village and helped manage Zebulon’s brick store. Zebulon and John went to work building the “new” house, i.e. the other half that is closer to Wharf Lane in 1813 for himself and his new wife to be, Nancy Gage (b 1857. (Parents Uriah Gage and Hannah Tinney (m jan 1857) of Bradford). Samuel Estabrook, who lived on River Road, built the chimney of 14,000 brick, and did much of the plastering and other work on the house. That’s when the house took on its present twin-chimney form – completed in 1814. In 1813 Zebulon suffered a fall from a ladder while they were building the new part of the house. His head injury took over two months to recover from. And in November of that year, his wife Ruth died of an illness. Zebulon re-married on May 11, 1814, to Mary Emery, a widow.
John Ingersoll, Esq. married on Aug 27, 1814 but he went ‘to sea’ very soon thereafter. By June of 1815 John received a position on a ship and was expected to set sail for Savannah. Instead he contracted yellow fever while still in Boston. He is buried on Rainsford island in Boston harbor.
John’s new wife Nancy was pregnant with their first child when John died. Nancy seems to have lived for a while with her infant son in the ‘new half’ of the house. In 1821 there was a deposition (No. 4/224?) from James Kimball saying that Zebulon Ingersoll should be thrown in “gaol” unless he pays him over $900. The committee found real estate holdings of Zeb’s including the old half of One Wharf to use to pay off Kimball. It is unclear at this point where Zebulon and Mary would have lived.
Zebulon died intestate in 1829. Zebulon’s wife Mary administered the probate but the committee formed to pay off Zebulon’s debts assigned one half of the house to Mary as his widow – the probate says this is the west (new) half, but this is very strange because the west half was owned at that time by Nancy and her son John Ingersoll. The old half as shown above was owned by Kimball. The house and barn (not the present garage-barn) were assessed at $700. In the probate was also “one share in the Merrimack Bridge, $85.00.”
Many years later (1836) in deed 298/35, Jesse Kimball, James’ son, sold the old half of the house to John G. Ingersoll, Nancy’s son, for $100. (298/35 is recorded as John G. Ingalls but further along in the deed the name changes to John G. Ingersoll) John was by then about 22 years old. Since Nancy and John already owned the new half, that allowed them to turn around and sell the entire house and some land toward the river to John Johnson (612/270) for $400.
More information about the Ingersoll Family.
Col. John Johnson (b1774) ran an inn and tavern across E. Main St. for 40 years. It burned down in about 1920. Johnson was also a member of the organization that managed the bridge and collected the tolls. Johnson married Zebulon’s oldest daughter Ruth (b1782) on Jan 6, 1799.
Sometime around the 1930’s the main central staircase was completely removed along with the interior walls of the two front rooms in order to create one large living room. The original back stairs remain, and have a ‘butterfly’ structure on the second floor (known as “goodnight and good morning” staircase.
The white fence around the yard in front of the house was installed by the Sweeneys using material that they bought at a home show. The fence was not formerly at the Lord Timothy Dexter house on High Street in Newburyport, as many legends say.
1850 census
822-1040 Richard THOMPSON, 29, shoemaker, NH
Sarah ", 24
Sarah ", 8/12
Johnson and McQuestion are listed in the hotel at this point.
Richard Thompson b 1821 m Sarah Ann Morse b 1826 in 1847. Sarah is Dean Morse’s sister and child of Amos Morse. See Morse family page. In maps of 1881 and 1892 he is shown as living at 1 Wharf Lane.
1855 Census
John Johnson (82) Gentleman
Ruth Johnson (79)
Abbie McQuestin (31) born in Maine
John Murphy (18) Ireland
In 1860 John Johnson sold the house to his granddaughter Abigail P. McQuesten (also 612/270) – with a little more land directly behind the house along Wharf Lane. Rights to the use of a spring between the house and the river were also included.
The land behind the house (4-8 Wharf Lane) was often referred to as the land that Zebulon Ingersoll died ‘seized of’, and it seems to have passed on to Zebulon’s female heirs and their husbands. See 4-8 Wharf Lane page.
In 1865 Abby McQuesten sold the house to Samuel and George Elliott. The Elliotts, shoe manufacturers in Haverhill, owned a lot of property around Rocks Village, but probably neither of them ever lived in the house. George built a large estate on Elliot Road up by North Kenoza Lake. Sam owned land north of E Main and does appear in the censuses for 1850-1860 as residing there, which was probably also his shoe factory.
There were several other transfers of ownership and mortgages in 1868 (750, 38) (924,91) and 1875 (924,46) involving Isabella Currier (a relative of George and Samuel Elliott and daughter of Joseph Estabrook, see Estabrook family page) who seemed to live in the house at that time and Snow, Wentworth and Taylor. In 1872 a J. Estabrook is listed as residing at the house. This is probably John L. Estabrook, son of John R. and Ann Estabrook, and Isabel’s cousin. John L was married to Mary E. Elliott in 1865. In 1875 Snow-Wentworth-Taylor sold the house to Richard C. Thompson for $300 (932/128), although there was still a mortgage outstanding owed to George Elliott. 1881 and 1892 maps lists R. Thompson residing in the house. This could be Richard Thompson who married Dean Morse’s sister Sarah in 1847.
In 1899, since Isabella Currier had defaulted on her original mortgage from George Elliott, George was free to sell the property at public auction, as originally agreed in (750/38). The highest bid – of $100 -was from George Elliott (is this himself? He did have a son but his name was George C. Elliott). This is recorded in (1596/346).
1870 Census:
Joseph Estabrook (60) works in shoe factory
Emma Morse (18) no employment
Addie Morse (22) Keeps House
Charles Morse (6) attends school
Richard Thompson (45) works in shoe factory
Sarah Thompson (17) keeps house
Henry Thompson (14) works in shoe factory
Emma (6) attends School
In 1902 the house was sold by George and Mary Elliott to Francis True (1678, 383), still maintaining the rights to the spring.
In 1910 True sold the house to Alice Trussell of Lynn. Alice Trussell also bought the land behind the house (4-8 Wharf Lane), bringing the two parcels together again but only briefly. In 1911 Trussell sold the land behind the house to Fanny Hewson of Boxford (2057,185), and sold the house to William and Annie Mutch of Everett. (2057, 183). It would finally remain for Frank and Myrtle Sweeney to unite the parcels again in about 1931 when they bought the house from Mutch (2889/320) and the land behind from Charles Tuck(2889/430), who had bought it from the guardian of Fanny Hewson in 1920.
The Sweeneys lived in the house for many years and did extensive renovations. They are buried in a crypt behind the house. They acquired land from the city toward the street and the river (the old Town Landing) (2954/454) , and other properties around the village. They died without any heirs, the house eventually being sold to Steve and Patty Lyons of Newburyport in the late 1970’s. The Brown/Baileys, the current owners of the house, bought it from the Lyons in 1983.
1940 Census: considered 2 Wharf Lane: people living there refused to answer census
1941: Dr. Frank Sweeney
These appear to be unrelated properties in Rocks Village:
1975: John B. Ray and Brenda J. Ray to Frank Sweeney and Myrtle D’A Sweeney (5952/470) (City of Haverhill land due to collector of taxes in 1932)
1975: Frank Sweeney and Myrtle D’A Sweeney to Philip J. Lawrence and Audrey J. Lawrence(5987/745)Previously City of Haverhill property)
1978: Frank Sweeney and Myrtle D’A Sweeney to John B. Ray and Brenda J. Ray(5860/118)
References:
https://haverhill.pastperfectonline.com/photo/BA604344-CAB2-4E23-93F4-679557154831
https://haverhill.pastperfectonline.com/photo/970D0DB7-4B45-4F2B-BA3F-012015153432