As a true freshman at Duke in 2022, Rivers started six of 13 games and had 52 tackles and one interception. He started all 13 games his sophomore year in 2023, recording 58 tackles and one interception which was returned for a touchdown. The pick-six came in the 2023 Birmingham Bowl and he was named the games MVP. Rivers returned to Duke as a starter his junior year in 2024, recording interceptions in three straight games, tying a Duke record. He was named first-team All-ACC, and also a second-team All-American by The Athletic and College FootballNetwork.