Colville's military career began in 1942, when he enlisted in the Canadian Infantry. He served first in Canada, then in 1944 he was sent to London as an official war artist. He painted in Yorkshire. He took part in the landings of the Canadian Navy in southern France. After that, he was attached to the Third Canadian Infantry Division and followed it through Holland and Germany from 1944 to 1945. He was given the difficult task of depicting the horrors of the Belsen concentration camp on canvas. He mounted an exhibition of war art in Amsterdam after the war.

Colville returned to Canada and mounted his first individual exhibitions in the 1950s. He gave a workshop on painting and drawing at the University of Alberta in Edmonton, served as an artist in residence at the University of California in Santa Cruz, and spent time in Berlin as a visiting artist. A number of universities have awarded him honorary doctorates in recognition of his work, and he was named an Officer of the Order of Canada in 1967 and a Companion in 1982. Colville currently lives and works in Wolfville, Nova Scotia