Rimer began his broadcasting career in Calgary with CFAC Radio. While at CFAC, he served as the play-by-play man for the Calgary Centennials of the Western Hockey League. Jeff would then spent over four years working for CBC Television in Edmonton. He was the network's gymnastics commentator during the 1976 Summer Olympics in Montreal. He also worked for eight years as the sports director and Montreal Canadiens pre- and post-game show host at CFCF Radio.
From 1983-1993, Rimer called Washington Capitals games on Home Team Sports and WDCA, first reporting from ice level and later the play-by-play action. He also served as the sports director at WBAL-Radio, where he hosted the Baltimore Orioles pre- and post-game shows, co-hosted syndicated college basketball shows with Billy Packer and Gary Williams, and was a sideline reporter for Maryland Terrapins football. He was awarded a regional Emmy Award for his play-by-play work and won a UPI National Broadcasting Award for his work at WBAL.
In 1993, Rimer became the TV voice of the expansion Florida Panthers. He remained with the team until the conclusion of the 2003–04 NHL season, when he left to join the Columbus Blue Jackets. While with the Panthers, he also co-hosted the internationally-syndicated "Inside the NHL" radio program with John Davidson.