The 39-year-old presenter shot to fame on 'Saturday Night Live' but left the popular sketch show in 2004 in a bid to find success on the big screen and poor box office results left him questioning his career path.
He said: "I was probably drinking more than I should have been drinking.
"It wasn't, like, sitting and watching old tapes of me on 'SNL', with the screen flickering in front of me. But I was like,' 'I can't figure out what I want to do.'"
Jimmy was delighted when he was offered a job on 'Saturday Night Live' in 1999 as he had been such a huge fan of the show when he was younger, he insisted on watching it alone so no one could ruin his viewing experience.
He explained: "I just didn't want anyone ruining my experience. No small talk. I wanted to see the sketches, the new characters, what the angle was."
Though Jimmy - who has five-month-old daughter Winnie with wife Nancy Juvonen - had strict parents, they allowed him to drink beer while watching the show at home as they knew he wouldn't get into any trouble.
He told the new edition of Vanity Fair magazine: "As long as I wasn't doing anything at night, I'd just sit by myself, and I would have a six-pack of Pabst.
"I don't know if I made it all the way through the six, but I'd just sit there and watch the show. And tape it."