Sounds like a dns issue.
Your isp routes information in a certain way down certain channels. If that channel is dedicated to amazon then everyone accessing amazon prime on your isp then it would make sense as to why it keeps slowing down.
You may want to try a different primary and secondary dns.
There are how to guides, and even some free software to determine the best overall dns for you in your area. But I would suggest giving google's dns servers a try first.
Sadly I am typing on a tablet but you can quickly find google's dns ip address to use on your router and modem.
It doesn't cause any alarms and is easy to set up on a router.
The secondary problem that I see with the amazon prime service is its locality. In order for amazon prime to stream flawlessly they send out data mines that look like briefcases to hubs around the world to decrease the need to connect over vast miles of land to reach your home. Your dns may be routing you to a further location than it should. The software in amazon prime does a simple ping test to determine which data mine you should access. And sometimes a dns error will cause you to connect to the incorrect data mine.
A failure in the dns server can be a number of things including wires being down for maintenance, downed PowerLine, power outage at the data center, or sometimes the data mine itself is having its own internal issues such as chisels becoming complacent, hard drive failure, and other things. Basically there are a lot of mechanical points of failure with streaming services.
So again you may want to try a different dns server before purchasing any new modems, routers, or streaming devices.
You can also initiate a ping test on your main computer that is hooked up to your router to see if it is a dns issue simply by running a command prompt and typing in ping https://www.amazon.Com
and it will ping that website and tell you if it has packet loss or not. After you change your dns server run this ping test again and it should give you different results.
As a last ditch effort try getting a new ip address from your is. This one requires some tinkering with basic file settings with your Internet connection. Search for new ip address in Google and it should show you how to do it. You don't need to install anything to get a new ip address you just have to remember to reset everything once you are done.