Anything plugged into the PCI or AGP bus is NOT hot swappable. USB and IEE1394 (Firewire) are both hot swappable. Ethernet can be connected and disconnected without powering off, but it's not actually "hot swappable", and the network won't always recover from swapping a new network connection. It will normally not have a problem if you unplug it and than plug it back into the same network.
Hot Swap usually refers to plugging or unplugging while power stays applied. This is a risky operation to do with any electronic device (for example what if, on insertion, the voltage pins make contact before the ground pins!), so it has to be specifically designed for it, e.g. USB and firewire peripherals as johnwill mentioned, or choke coils on the voltage lines of a bus slot, etc..
FYI with some large enterprise servers there is something called Online Add and Replace (OL/AR) or similar term. Part of this involves running a utility which lets you power down an individual I/O slot so you can remove or add an I/O card, then power it up. Meanwhile the rest of the server continues to fly. Power is off, so technically this isn't a "hot" swap but the end goal is still the same: change hardware without shutting down the box. Slot power is being killed to improve the overall chances of success of the operation.
Obviously this requires extensive support in both hardware and software, not just to control a slot's power, but also to recognize entire bus-based I/O adapters appearing and disappearing on-the-fly at runtime. This sort of thing is not generally available in PC-level platforms.
A forum community dedicated to tech experts and enthusiasts. Come join the discussion about articles, computer security, Mac, Microsoft, Linux, hardware, networking, gaming, reviews, accessories, and more!