Strange AT&T DSL Problem

BrentD · 01-29-2009, 04:43 PM

I've got a strange DSL problem that has AT&T DSL Support totally baffled.

I've got a Cisco 1700 router with a DSL modem card and a business DSL account witha static IP. I use the 1700 because I have to connect to a company VPN and need all computer at this particular site on the VPN.

My problem is that I am unable to browse certain websites. None of the microsoft or msn sites will show up and about 50% of other sites I try don't load. DNS resolves correctly. I've verified it against a working machine on another network.

I'm the IT manager for the company so I know that there is no content filtering on the network, no IP blacklists and no interference from the VPN. We've got 2 other offices also using AT&T DSL with this same setup and none of those locations have any problems.

Here is where things really get weird. If I change the username and password in our Cisco router from the company login to my Home DSL login with a dynamic IP, all sites load fine. If I switch over to AT&T's netopia router and use the company login to get the static IP, again all sites load fine.

I've been on the phone with AT&T every day this week and and can;t find a solution. We've tried changing the static IPs to a different range, changing the MTU in the router, disabling NAT and VPN, rebuilding the router config from scratch, copying in a config from a known working router, etc, etc and can't solve this problem. AT&T is insisting that it is the cisco router, but I don't understand how a known good router that works fine at another location with an identical config could be the source of the problem, nor why changing from a static IP to a dynamic IP would solve the problem unless it is a routing issue.

For what it's worth, I've also verified that that the hosts files on the computers are clean, there is no firewall software on the computers, and all of the other common problems I've seen in the past. I've been a sysadmin/network admin/IT manager for over 15 years and have never run into a problem quite like this.

Any help at all would be greatly appreciated.

Ulexus · 10-22-2009, 10:45 PM

I have for months been trying to tackle this very problem, and it has been progressively worse. This is affecting _all_ of my AT&T DSL customers in my region, not just one site.

Similar details:
- microsoft.com and several Microsoft subsidiary sites do not load; additionally, many other sites not seemingly related to Microsoft do not load. google.com DOES work
- DNS verified to not be the issue (same IP used on working system as non-working system)
- different accounts and different connections fix the problem (at least temporarily)

Differences:
- All of my sites use Linux-based routers, not Cisco routers
- I have a mix of static, dynamic, and dynamic with static blocks
- tearing down the PPPoE connection and reestablishing it would sometimes fix the problem (getting a different IP, I presumed). However, the frequency of this fixing the problem has gotten progressively worse over the months. My assumption is that the IP pool has become gradually more polluted. On sites with static IPs, there has been no workaround.

Additional findings:
- Requests originating from the router itself work for me. That is, if I set up a proxy server on the router or telnet to port 80 directly from the router, the connections work. From the masqueraded clients, they do not. I do not know if the Cisco 1700 IOS has telnet client support or not, but I'd be curious if that works for you, too.
- packet traces and telnet sessions show that ACKs are received, but the payload of the web request is not
- this affects both Linux and Windows clients, and I have used Live CDs to verify the cleanliness of the systems
- I had, in the past, found a high correlation between malware infections and the blocking of these certain sites. It used to be that I could go clean the machine(s), restart the DSL connection, and have it work. That is no longer the case. That lead me to the assumption that IPs in the DSL block were getting blacklisted. This may, in fact, be the case, but if so, the pools are getting to heavily blacklisted that they are nearly unusable.
- AT&T never acknowledges or admits to the issue; however, every time a technician has been dispatched to a site (which has not happened but a few times), the problem miraculously goes away (but returns later).

01-29-2009, 04:43 PM	#1 (permalink)
BrentD Registered User Join Date: Jan 2009 Posts: 1 OS: XP	Strange AT&T DSL Problem I've got a strange DSL problem that has AT&T DSL Support totally baffled. I've got a Cisco 1700 router with a DSL modem card and a business DSL account witha static IP. I use the 1700 because I have to connect to a company VPN and need all computer at this particular site on the VPN. My problem is that I am unable to browse certain websites. None of the microsoft or msn sites will show up and about 50% of other sites I try don't load. DNS resolves correctly. I've verified it against a working machine on another network. I'm the IT manager for the company so I know that there is no content filtering on the network, no IP blacklists and no interference from the VPN. We've got 2 other offices also using AT&T DSL with this same setup and none of those locations have any problems. Here is where things really get weird. If I change the username and password in our Cisco router from the company login to my Home DSL login with a dynamic IP, all sites load fine. If I switch over to AT&T's netopia router and use the company login to get the static IP, again all sites load fine. I've been on the phone with AT&T every day this week and and can;t find a solution. We've tried changing the static IPs to a different range, changing the MTU in the router, disabling NAT and VPN, rebuilding the router config from scratch, copying in a config from a known working router, etc, etc and can't solve this problem. AT&T is insisting that it is the cisco router, but I don't understand how a known good router that works fine at another location with an identical config could be the source of the problem, nor why changing from a static IP to a dynamic IP would solve the problem unless it is a routing issue. For what it's worth, I've also verified that that the hosts files on the computers are clean, there is no firewall software on the computers, and all of the other common problems I've seen in the past. I've been a sysadmin/network admin/IT manager for over 15 years and have never run into a problem quite like this. Any help at all would be greatly appreciated.

10-22-2009, 10:45 PM	#2 (permalink)
Ulexus Registered User Join Date: Oct 2009 Posts: 1 OS: Linux (Debian, Gentoo)	Re: Strange AT&T DSL Problem I have for months been trying to tackle this very problem, and it has been progressively worse. This is affecting _all_ of my AT&T DSL customers in my region, not just one site. Similar details: - microsoft.com and several Microsoft subsidiary sites do not load; additionally, many other sites not seemingly related to Microsoft do not load. google.com DOES work - DNS verified to not be the issue (same IP used on working system as non-working system) - different accounts and different connections fix the problem (at least temporarily) Differences: - All of my sites use Linux-based routers, not Cisco routers - I have a mix of static, dynamic, and dynamic with static blocks - tearing down the PPPoE connection and reestablishing it would sometimes fix the problem (getting a different IP, I presumed). However, the frequency of this fixing the problem has gotten progressively worse over the months. My assumption is that the IP pool has become gradually more polluted. On sites with static IPs, there has been no workaround. Additional findings: - Requests originating from the router itself work for me. That is, if I set up a proxy server on the router or telnet to port 80 directly from the router, the connections work. From the masqueraded clients, they do not. I do not know if the Cisco 1700 IOS has telnet client support or not, but I'd be curious if that works for you, too. - packet traces and telnet sessions show that ACKs are received, but the payload of the web request is not - this affects both Linux and Windows clients, and I have used Live CDs to verify the cleanliness of the systems - I had, in the past, found a high correlation between malware infections and the blocking of these certain sites. It used to be that I could go clean the machine(s), restart the DSL connection, and have it work. That is no longer the case. That lead me to the assumption that IPs in the DSL block were getting blacklisted. This may, in fact, be the case, but if so, the pools are getting to heavily blacklisted that they are nearly unusable. - AT&T never acknowledges or admits to the issue; however, every time a technician has been dispatched to a site (which has not happened but a few times), the problem miraculously goes away (but returns later).