If the drives are different brands you can check in the BIOS (press del to at startup) which drive is the primary master and which one is the secondary master. Then refer to the motherboard manual or look on the motherboard itself to see which cable is on the primary IDE controller and which is on the secondary IDE controller. Usually the sytem drive should be the primary master.
Note that it won't do any harm to your system if you remove the wrong one, you'll simply get a system disk error message at startup.
As Stu_computer said unplug the power cord before removing the drive, the motherboard remains powered when the computer is off and the power cable still attached.
Just remove the cable from one of the drives and then try to boot the computer, If it does not boot and gives an error that it can not find the hard drive then you have disconnected drive C: if it boots then you have disconnected the drive D: when you have found this mark them with a magic marker.
A forum community dedicated to tech experts and enthusiasts. Come join the discussion about articles, computer security, Mac, Microsoft, Linux, hardware, networking, gaming, reviews, accessories, and more!