Oil Company in Lloyd Harbor, NY.

In 1654, the Matinecock Native Americans sold 3,000 acres (12 km²) of what is now called Lloyd Neck to English settlers from Oyster Bay. The Matinecock referred to the region as Caumsett (“place by sharp rock”). In 1676, James Lloyd acquired the neck, which was then taken over by his son Henry. Henry Lloyd farmed the land and erected a house, which still survives in Caumsett State Park. After his death in 1763, his son Joseph built the Joseph Lloyd Manor House, which he was forced to abandon by the British during the Revolutionary War. The British built several fortifications in the neck, including Fort Franklin. Henry Lloyd IV was the last Lloyd to own the estate, in 1841. In the 1880s, it became a stop for steamboats coming from New York City, bringing tourists and wealthy New Yorkers.

In 1886, Lloyd’s Neck, which was then part of the town of Oyster Bay and had earlier been known as Queens Village, was set off and separated from Queens County and annexed to the town of Huntington in Suffolk County.

The 1900s ushered the era of the Long Island Gold Coast, and various wealthy families began to buy land and build seaside mansions and estates. These included William Matheson, Marshall Field III, Ronald Conklin, Harold Dimppel, Sr., Ferdinand Eberstadt and George McKesson. In 1926, Lloyd Neck and West Neck became incorporated as a municipality which was named Lloyd Harbor.