Slow "high speed" internet

Hello all...any help with this would be great, thanks.

So here's the scoop. I moved into my current house on May 1, and after a few failed attempts to connect via wireless to the house internet, we gave up and plugged the cable into the back of the computer, and it worked. At first it was very slow, but then it picked up and was super fast. Then after about a day, it was extremely slow again. Then about a week later, it was really fast, and then again, slow. It seems to go from periods of about 2 or 3 weeks of being really slow, to a day or so of being extremely fast, back to super slow, and I'm getting extremely annoyed by the fact that I'm paying for part of this "high speed" internet, when I'm just getting "super slow" internet. We're using Bell Sympatico, and I'm wired through the wireless router of the guy in the basement - there's 5 others living here, but just me and the guy in the basement are here for the summer.

It shouldn't be my computer, because I recently finished an upgrade to an AMD X2 4400+, I've got 1 GB RAM, an nVidia 7200 video card...I don't have a proper networking card, so everything is run by my nVidia Networking Controller - is it possible that the problem is coming from that? Would a networking card possibly fix it?

The only other things I can think of are that it's something to do with the router downstairs, some settings on my computer not properly configured, or I've got a virus/spyware that's causing problems.