The most successful of Hamilton's early libraries, the Hamilton & Gore Mechanics' Institute, had its roots in the Mechanics' Institute movement which originated in England in 1823 to promote education for working-men, particularly in the newly-popular areas of science and mechanics. The inaugural meeting of the Institute was held in February 1839 and, in May of that year, its library opened for business. The Institute was incorporated in 1849, and a new building was constructed in 1853 to house its operations.

By the early 1880s, the Institute was in financial straits. Appeals for funding failed, and it was forced to close in 1882. That same year, the provincial government passed legislation which allowed communities to establish free libraries by a vote of taxpayers. Hamiltonians, for various reasons, did not take advantage of this legislation; for the next seven years, the only lending libraries available to the public were those attached to bookstores. In 1885 a by-law vote for a free public library finally was held, but was defeated. By 1888 the pro-library forces had regrouped and, on 7 January 1889, a new library by-law was ratified by voters. Subsequently, the Hamilton Public Library Board was established to oversee the organization and operation of the library; one of its first tasks was to hire Thomas Lancefield as chief librarian.

A site on the north side of Main Street, just west of James Street beside Centenary Methodist Church, was selected for the new library building, and William Stewart was named as architect. While the building was being constructed, a library collection was assembled both through purchases and through donations from local citizens; many of these books are still to be found among the present-day collections of the library. The cornerstone ceremony took place on 23 October 1889, and the library was opened to the public in February 1890. It was an immediate success, in that additional staff soon had to be hired and hours of service extended. At the official opening ceremony held on 16 September 1890, a field of dignitaries, including the Earl and Countess of Aberdeen, spoke of Hamilton's accomplishment in being the first city in Canada to erect a public building to serve as a library.

During the first decade of its existence, the library became indispensable to the citizens of Hamilton. The newest trends in librarianship were adopted, and collections flourished. Then, in February 1902, a major scandal occurred. After destroying the minute books and ledgers, Lancefield disappeared, leaving the Library Board to face a large overdraft. It soon was determined that, over the preceding two years, the chief librarian had defrauded the library of several thousand dollars. John Kenrick, a board member, was appointed acting chief librarian and managed to restore the library to a much sounder financial footing through careful management. Kenrick oversaw operations until his death in 1904 and was succeeded by Adam Hunter, who had considerable experience in the book trade.

Hamilton's first branch library opened on Barton Street in 1908. Soon, the Library Board began to plan for a new Main Library building. The Board contacted the Carnegie Corporation and, eventually, received a grant towards the construction of a new building. Tenders were called in 1911 for a structure to be located on the south side of Main Street, across the street and just west of the existing library. On 5 May 1913, the new library designed by architect Alfred Peene, was officially opened by the Lieutenant Governor, Sir John Morison Gibson. One month later, it was fully operational. This building served as the hub of library activity for many years and is remembered fondly by several generations of Hamiltonians.