Joe Flaherty began his comedy career at The Second City in Chicago, where he wrote and performed in multiple Mainstage revues. He starred alongside Brian Doyle-Murray and Harold Ramis in The Next Generation, Justice is Done or Oh, Cal Coolidge, and Cooler Near The Lake. Three of his later revues, Cum Grano Salis, No, No, Wilmette, and 43rd Parallel or McCabre & Ms. Miller, also starred future comedic legend John Belushi. Flaherty later appeared on National Lampoon Radio Hour with Belushi, Bill Murray, Gilda Radner, Chevy Chase, and past collaborators Doyle-Murray and Ramis.

Flaherty moved to Toronto after seven years in Chicago to help develop the Second City Toronto in an original cast also including Radner. While in Canada, he also starred in SCTV, along with John Candy, Catherine O’Hara, and Martin Short, and was nominated for nine Primetime Emmy Awards in Outstanding Writing in a Variety or Music Program, winning twice in 1982 and 1983. Impressions he did on the show included Kirk Douglas, Richard Nixon, Art Garfunkel, Gregory Peck, Alan Alda, and many more. Original characters ranged from Big Jim McBob, Guy Caballero, and Count Floyd, the latter of which Alice Cooper gave a special thanks to on his album, Special Forces, in 1981.

Throughout his run at SCTV and beyond, Flaherty continued to act in both television and film. He had roles in many films that starred fellow Second City alum, such as 1941, Going Berserk, and Stripes. Additional films and television shows include Back to the Future II, Happy Gilmore, One Crazy Summer, National Security, Freaks and Geeks, Family Guy, The King of Queens, and more.

Flaherty is an advisory board member of the Harold Ramis Film School.