Server 2003, Exchange 2003, and ESX 3.5 Madness!

alupis · 08-15-2009, 09:09 PM

I apologize if this is in the wrong section, I did not see any ESX or Virtual Machine related sections, so i posted in the next best thing, the server 2003 section.

We are a medium sized business that heavily relies on the use of MS Exchange 2003 for booking appointments and emails 24x7. I am part of the IT department that helped the decision to move the servers over to virtual machines running ontop of VmWare's ESX 3.5 a few months ago. We used the free virtual machine converter provided by VmWare to do this in real-time (it basicly makes a clone of your server's HDD and makes it into a VM on your ESX host machine). Ever since we initially moved the servers over, there has been nothing but trouble... mostly related to Exchange services not starting when the vm reboots, but there are other things as well. Yes the services are all set to Automatic, and yes Exchange is supposed to start automaticly when the server boots up. Some of the other problems are the virtual CPU maxing out at 100% randomly even though no service in task manager shows which one is doing this, and the client machines loosing access to the public folders which are when all of our shared calendars are located. When they loose access to them their machine will just say it is trying to retrieve information from the Exchange server... yady yadda... and will stay like that for, sometimes 30 minutes. It got so bad for a few days that we would have to reboot the exchange server entirely during the middle of the workday! We have been battling this for several weeks now and are very frustrated and fed up with this; it is costing us business at this point. A few of the other IT personell wish to ditch VmWare ESX all together and go back to physical machines, but i am a firm believer in the esx and vm platform, but i lack the complete expertise to diagnos and repair this problem. I do not want to move backwards to a physical server, so any help would be awsome! Has anyone else had problems with specifically Exchange servers once they were migrated over to an ESX host platform? Please Advise!

**bilbus** · 08-16-2009, 08:22 PM

Well, we have 20 exchange servers on 4x ESX 3.5U4 .. no porblems.

Sounds like you should move back to phyical server, and test your exchange servers under ESX before deploying it.

I would not have done a live migration like that. Should have taken the servers down, and P2Ved the servers.

Sounds like the proper testing and planing was not done.

What kinda host servers are you running? How many total Vms are on this server/servers

Is this OEM grade hardware or just some home built box?

alupis · 08-16-2009, 08:46 PM

Thanks for the quick reply!

THis is a Dell Poweredge 1800? tower with dual 3.2Ghz Xeon, 8GB ram, 1TB raid 5 array... the works... its not the newest server in town but its fine for what we need. we are only running 2 VM's on this setup. One is the Exchange server and primary domain controller, the other is a backup domain controller and has a few public shares on it... thats it. What is this "P2Ved" process of migrating over to the ESX about? How can we do it in the future? What kind of system are your 20 vm's running on? we had difficulties running both of those vm's on this setup (minus a couple gigs of ram and 1 cpu) and it took adding more specs to and fine tuning of the virtual specs to get everything running decently, then all of these problems started... lol! The only problem with testing before deplyment is that we can not effectively test the load on the exchange server before actually deploying it. We use Outlook far past what it was originally intended to be used for, and we are well aware of its limitations because of this... but we did not have any of these problems on the physical machines before the migration, and the physical boxes were very low end budget server builds...

alupis · 08-16-2009, 08:49 PM

btw, i just looked up P2V and got the VMware vCenter Converter Standalone software. This is the exact software we used for the migration. We kept the machines online (no users and the ipaddresses changed to prevent anyone from accessing anything) and just had it do what it does.

**bilbus** · 08-16-2009, 11:43 PM

You cant P2V domain controlers =( I bet thats your issue

Ram should not be an issue, just add more it's cheap enough.

I was told by many people that p2ving a DC will cause huge problems .. better to just rebuild the DC inside VM.

We have a cluster of 4 dell R900s (4x Quad core cpus (16 cores) .. 256GB ram) around 15k per box for hardware.

alupis · 08-17-2009, 05:07 PM

Nice setup!

Any idea's why moving a DC will cause problems? all the server knows is that it got new drivers suddenly...lol. Everything else was kept the same (ip's, ...etc... even used the same port on our switch! lmao!) how sure are you that this is a cause of our problems? Have you had any first hand experience with migrating a DC?

Thanks for the info! You rock! lol

alupis · 08-17-2009, 05:08 PM

about how many users are the 4 esx cluster supporting? thanks!

**bilbus** · 08-17-2009, 06:33 PM

around 10k users.

As for why P2Ving DCs is not recomended .. i don't recall .. vmware and microsoft said nopt to do it.

I rebuilt our DCs when we turned em into VMs, and decomed the phyical ones..

**bilbus** · 08-17-2009, 06:35 PM

http://kb.vmware.com/selfservice/mic...%200%207552177

http://support.microsoft.com/kb/875495

http://www.yellow-bricks.com/2008/12...n-controllers/

alupis · 08-17-2009, 08:13 PM

you are the best bilbus! thank you for all the helpful information.

Is your setup just hosted exchange for clients? how large do you allow the mailboxes to be?

Does this mean you are successfully running about 500 users per exchange server? and that would make 2500 users per physical box?

alupis · 08-17-2009, 08:14 PM

We are going to rebuild both servers from scratch on the esx host... are there any other things we should look out for since we are still relitively new at virtual servers and virtual exchange servers especially?

Any additional info would be awsome!

THank you!

**bilbus** · 08-17-2009, 09:55 PM

we run 2500 users per server, and 5000 per datacenter.

alupis · 08-17-2009, 10:06 PM

what type of pipe does that require?

**bilbus** · 08-17-2009, 10:12 PM

we have 10gb between DCs, i dont know what we have to the internet

alupis · 08-18-2009, 04:15 PM

Awesome! you have been very helpful bilbus. thank you so much!

08-15-2009, 09:09 PM	#1 (permalink)
alupis Registered User Join Date: Jun 2006 Location: CA, USA Posts: 87 OS: XP Pro, Server 2003, Vista, Ubuntu, Red Hat, OSX 10.5.4	Server 2003, Exchange 2003, and ESX 3.5 Madness! I apologize if this is in the wrong section, I did not see any ESX or Virtual Machine related sections, so i posted in the next best thing, the server 2003 section. We are a medium sized business that heavily relies on the use of MS Exchange 2003 for booking appointments and emails 24x7. I am part of the IT department that helped the decision to move the servers over to virtual machines running ontop of VmWare's ESX 3.5 a few months ago. We used the free virtual machine converter provided by VmWare to do this in real-time (it basicly makes a clone of your server's HDD and makes it into a VM on your ESX host machine). Ever since we initially moved the servers over, there has been nothing but trouble... mostly related to Exchange services not starting when the vm reboots, but there are other things as well. Yes the services are all set to Automatic, and yes Exchange is supposed to start automaticly when the server boots up. Some of the other problems are the virtual CPU maxing out at 100% randomly even though no service in task manager shows which one is doing this, and the client machines loosing access to the public folders which are when all of our shared calendars are located. When they loose access to them their machine will just say it is trying to retrieve information from the Exchange server... yady yadda... and will stay like that for, sometimes 30 minutes. It got so bad for a few days that we would have to reboot the exchange server entirely during the middle of the workday! We have been battling this for several weeks now and are very frustrated and fed up with this; it is costing us business at this point. A few of the other IT personell wish to ditch VmWare ESX all together and go back to physical machines, but i am a firm believer in the esx and vm platform, but i lack the complete expertise to diagnos and repair this problem. I do not want to move backwards to a physical server, so any help would be awsome! Has anyone else had problems with specifically Exchange servers once they were migrated over to an ESX host platform? Please Advise!

08-16-2009, 08:22 PM	#2 (permalink)
bilbus TSF Enthusiast Join Date: Aug 2006 Posts: 949 OS: OS2 Warp	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! Well, we have 20 exchange servers on 4x ESX 3.5U4 .. no porblems. Sounds like you should move back to phyical server, and test your exchange servers under ESX before deploying it. I would not have done a live migration like that. Should have taken the servers down, and P2Ved the servers. Sounds like the proper testing and planing was not done. What kinda host servers are you running? How many total Vms are on this server/servers Is this OEM grade hardware or just some home built box? Last edited by bilbus; 08-16-2009 at 08:23 PM.

08-16-2009, 08:46 PM	#3 (permalink)
alupis Registered User Join Date: Jun 2006 Location: CA, USA Posts: 87 OS: XP Pro, Server 2003, Vista, Ubuntu, Red Hat, OSX 10.5.4	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! Thanks for the quick reply! THis is a Dell Poweredge 1800? tower with dual 3.2Ghz Xeon, 8GB ram, 1TB raid 5 array... the works... its not the newest server in town but its fine for what we need. we are only running 2 VM's on this setup. One is the Exchange server and primary domain controller, the other is a backup domain controller and has a few public shares on it... thats it. What is this "P2Ved" process of migrating over to the ESX about? How can we do it in the future? What kind of system are your 20 vm's running on? we had difficulties running both of those vm's on this setup (minus a couple gigs of ram and 1 cpu) and it took adding more specs to and fine tuning of the virtual specs to get everything running decently, then all of these problems started... lol! The only problem with testing before deplyment is that we can not effectively test the load on the exchange server before actually deploying it. We use Outlook far past what it was originally intended to be used for, and we are well aware of its limitations because of this... but we did not have any of these problems on the physical machines before the migration, and the physical boxes were very low end budget server builds...

08-16-2009, 08:49 PM	#4 (permalink)
alupis Registered User Join Date: Jun 2006 Location: CA, USA Posts: 87 OS: XP Pro, Server 2003, Vista, Ubuntu, Red Hat, OSX 10.5.4	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! btw, i just looked up P2V and got the VMware vCenter Converter Standalone software. This is the exact software we used for the migration. We kept the machines online (no users and the ipaddresses changed to prevent anyone from accessing anything) and just had it do what it does.

08-16-2009, 11:43 PM	#5 (permalink)
bilbus TSF Enthusiast Join Date: Aug 2006 Posts: 949 OS: OS2 Warp	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! You cant P2V domain controlers =( I bet thats your issue Ram should not be an issue, just add more it's cheap enough. I was told by many people that p2ving a DC will cause huge problems .. better to just rebuild the DC inside VM. We have a cluster of 4 dell R900s (4x Quad core cpus (16 cores) .. 256GB ram) around 15k per box for hardware. Last edited by bilbus; 08-16-2009 at 11:46 PM.

08-17-2009, 05:07 PM	#6 (permalink)
alupis Registered User Join Date: Jun 2006 Location: CA, USA Posts: 87 OS: XP Pro, Server 2003, Vista, Ubuntu, Red Hat, OSX 10.5.4	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! Nice setup! Any idea's why moving a DC will cause problems? all the server knows is that it got new drivers suddenly...lol. Everything else was kept the same (ip's, ...etc... even used the same port on our switch! lmao!) how sure are you that this is a cause of our problems? Have you had any first hand experience with migrating a DC? Thanks for the info! You rock! lol

08-17-2009, 05:08 PM	#7 (permalink)
alupis Registered User Join Date: Jun 2006 Location: CA, USA Posts: 87 OS: XP Pro, Server 2003, Vista, Ubuntu, Red Hat, OSX 10.5.4	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! about how many users are the 4 esx cluster supporting? thanks!

08-17-2009, 06:33 PM	#8 (permalink)
bilbus TSF Enthusiast Join Date: Aug 2006 Posts: 949 OS: OS2 Warp	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! around 10k users. As for why P2Ving DCs is not recomended .. i don't recall .. vmware and microsoft said nopt to do it. I rebuilt our DCs when we turned em into VMs, and decomed the phyical ones..

08-17-2009, 06:35 PM	#9 (permalink)
bilbus TSF Enthusiast Join Date: Aug 2006 Posts: 949 OS: OS2 Warp	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! http://kb.vmware.com/selfservice/mic...%200%207552177 http://support.microsoft.com/kb/875495 http://www.yellow-bricks.com/2008/12...n-controllers/ Last edited by bilbus; 08-17-2009 at 06:37 PM.

08-17-2009, 08:13 PM	#10 (permalink)
alupis Registered User Join Date: Jun 2006 Location: CA, USA Posts: 87 OS: XP Pro, Server 2003, Vista, Ubuntu, Red Hat, OSX 10.5.4	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! you are the best bilbus! thank you for all the helpful information. Is your setup just hosted exchange for clients? how large do you allow the mailboxes to be? Does this mean you are successfully running about 500 users per exchange server? and that would make 2500 users per physical box?

08-17-2009, 08:14 PM	#11 (permalink)
alupis Registered User Join Date: Jun 2006 Location: CA, USA Posts: 87 OS: XP Pro, Server 2003, Vista, Ubuntu, Red Hat, OSX 10.5.4	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! We are going to rebuild both servers from scratch on the esx host... are there any other things we should look out for since we are still relitively new at virtual servers and virtual exchange servers especially? Any additional info would be awsome! THank you!

08-17-2009, 09:55 PM	#12 (permalink)
bilbus TSF Enthusiast Join Date: Aug 2006 Posts: 949 OS: OS2 Warp	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! we run 2500 users per server, and 5000 per datacenter.

08-17-2009, 10:06 PM	#13 (permalink)
alupis Registered User Join Date: Jun 2006 Location: CA, USA Posts: 87 OS: XP Pro, Server 2003, Vista, Ubuntu, Red Hat, OSX 10.5.4	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! what type of pipe does that require?

08-17-2009, 10:12 PM	#14 (permalink)
bilbus TSF Enthusiast Join Date: Aug 2006 Posts: 949 OS: OS2 Warp	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! we have 10gb between DCs, i dont know what we have to the internet

08-18-2009, 04:15 PM	#15 (permalink)
alupis Registered User Join Date: Jun 2006 Location: CA, USA Posts: 87 OS: XP Pro, Server 2003, Vista, Ubuntu, Red Hat, OSX 10.5.4	Re: Server 2003, Exchange 2003, and ESX 3.5 Madness! Awesome! you have been very helpful bilbus. thank you so much!