When Kentucky's state highway system was originally established in 1929, KY 104's current routing was signed as Kentucky Route 181. KY 104 originally went from a junction with U.S. Route 79 at Guthrie to its current northern terminus. That alignment is now part of KY 181, and KY 104 moved to its current alignment at some time between 1939 and 1954.