Litvinoff's early years in what he frequently described as the Jewish ghetto in the East End of London made him very conscious of his Jewish identity, a subject he explored throughout his literary career. Litvinoff was born to Russian Jewish parents who emigrated from Odessa to Whitechapel, London, in 1915. His father was repatriated to Russia to fight for the czar and never returned: he is thought to have been killed in the revolution of 1917. Litvinoff was the second of nine children. One of his brothers was the historian Barnet Litvinoff and his half-brother was David Litvinoff who was born to his mother's second husband Solomon Levy.
Litvinoff left school at fourteen and, after working in a number of unskilled factory jobs, found himself homeless within a year. Drifting between Soho and Fitzrovia during the Depression of the 1930s, he wrote hallucinatory materials, since destroyed, and used his wits to survive.
Initially a conscientious objector, Litvinoff volunteered for military service in January 1940 on discovering the extent of the persecution suffered by Jews in Europe. He was commissioned into the Pioneer Corps in August 1942. Serving in Northern Ireland, West Africa and the Middle East, he rose through the ranks quickly, being promoted to major by the age of 27.
Litvinoff became known as a war poet during his time in the Army. The anthology Poems from the Forces, published by Routledge in 1941, included some of his poems, as did the BBC radio feature of the same title. Conscripts: A Symphonic Declaration appeared in the same year, and his first collection, The Untried Soldier, followed in 1942. A Crown for Cain, published in 1948, included his poems from West Africa and Egypt. Over the years, he contributed poems to many anthologies and periodicals, including The Terrible Rain: War Poets 1939–1945 and Stand, a magazine edited by Jon Silkin. Litvinoff was a friend and mentor to many younger poets. His poems were collected in Notes for a Survivor (1973).
Litvinoff was one of the first to raise publicly the implications of T. S. Eliot's negative references to Jews in a number of his poems, in his own poem "To T. S. Eliot". Litvinoff, an admirer of Eliot, was appalled to find Eliot republishing lines he had written in the 1920s about "money in furs" and the "protozoic slime" of Bleistein's "lustreless, protrusive eye" only a few years after the Holocaust, in his Selected Poems of 1948. When Litvinoff got up to announce the poem at a poetry reading for the Institute of Contemporary Arts in 1951 the event's host, Sir Herbert Read, declared, "Oh Good, Tom's just come in," referring to Eliot. Despite feeling "nervous", Litvinoff decided that "the poem was entitled to be read" and proceeded to recite it to the packed but silent room: