In 1694, L'Hermitte found himself a victim of an army cut-back, but he was lucky enough to be recognized by the Governor of Placentia, de Brouillan, as possessing catographic talents. As a result, he was appointed town major and engineer of Placentia in 1695.
Brouillan held L'Hermitte in high esteem and arranged to have him designated third in command, after the Governor and the King's Lieutentant. However, this made him far more enemies than friends. Though his engineering duties were demanding, he never the less had to perform the required tasks of being Placentia's town mayor. This included supervising the daily functions of the garrison.
Except for the occassional interlude that demanded a mapping of an expedition or the like, L'Hermitte spent 17 years trying desperately hard to keep the fortifications for falling into a chronic state of disrepair. It was a frustrating battle against surmontable odds, such as the harshness of the climate, the isolation of the area, the lack of sufficient replacements and an inadequate financial policy. Yet his efforts were recongized and he was appreciated as being a good draftsman, surveyor and cartographer. In 1698 he was sent to Arcida; later he went to Trois-Riveres where he was asked to map the Ile Royale and the surrounding area.
At the cession of Placentia in 1713 L'Hermitte played an important role in establishing the new colony of Ile Royale. However of age was beginning to make its effects felt and L'Hermitte was appointed the King's lieutenant of Trois-Riveres. In 1718, he was awarded the cross of the Order of Saint-Louis. Seven Years later he was drowned while aboard the ill-fated Chameau.