Hi JodyO and welcome to TSF !
Have you tried pressing the caps lock key again or pressing the shift key to deactive the caps lock ?
It could also be the shift key that's stuck, try to repress it several times.
Go to control panel => accessibility options and check that the sticky keys are not enabled.
Lastly, try to reboot the computer, and unplug/replug the keyboard.