Besides the lack of protection, another reason why the Grassy Island fishery failed was due to the lack of product control. Much of the fish sent to Europe was not properly salted. Ships often arrived with a hold full of rotten fish. After the capture of Louisbourg in 1746, there was an influx to Louisbourg due to the newly established security of the English fishery there. Canso, in turn, suffered as the fishermen that usually worked in Canso migrated to Louisbourg instead. Tensions continued to mount though, and this was only amplified by the return of Louisbourg to the French in 1748 through the Treaty of Aix-La-Chappelle.

Very few settlers came to the area again before 1764 and ones that did often faced reprisals from the French and Mik'maq in the area. Canso became an outpost of sorts after 1764. Supplies were often shipped from Halifax to the growing population in the Cape Canso area. The fishery was taking some slow steps at rebuilding on the islands and mainland, including Grassy Island.

Louisbourg was again captured in 1758 and the ownership of the islands were once again questioned. In 1764, the issue was finally settled with the Treaty of Paris. In this treaty it was determined that St. Pierre et Miquelon would go to the French whereas the Canso Islands were officially transferred to British ownership.

In 1764, Charles Morris, Chief Surveyor, reported the fishery at Cape Canso Islands to be alive and well with over 13 fishing rooms, 30 schooners, and 15,000 quintals of fish. He encouraged protection of the area and drew up a town plan for the mainland called "Wilmot". One plan as late as 1781 shows lots on the island assigned by a Lord Colville.

The fishery as of 1770's was being revived but this resurgence would not last for long. Canso would not be able to evade the effects of the American Revolution. In the year 1776, privateer John Paul Jones (often accredited with the birth of the American Navy) attacked the Canso islands and systematically wrecked the fishery in the area for many years to come. After that time, the mainland became the focus of settlement. Grassy Island, from then on, lay dormant until 1975 when the first survey and archaeological testing was conducted on Grassy.