First Dell Blade Chassis, Part 2
Day 2 and 3
The M1000e has an option to use switch modules or ethernet passthrough modules for network connectivity. The one I am working on has 6 ethernet passthroughs to get the networking done. Each blade in the chassis has 6 NIC’s, 2 onboard, 2 on mezzanine B and 2 on mezzanine C. The question is, which is which and ethernet passthrough port do they connect to?
The passthroughs are numbered A1, B1, C1, C2, B2 and A2. Each has 16 ports which matches the total number of blades the M1000e can hold. The 16 ports on the passthrough module are numbered like this:
1 2
3 4
5 6
7 8
9 10
11 12
13 14
15 16
Passthrough A1 port 1 matches to onboard NIC 1 in Blade 1. Passthrough A2 port 1 maps to onboard NIC 2 in Blade 1. Passthough B1 port 1 maps to mezzanine 1 port 1 on Blade 1. Passthough B2 port 1 maps to mezzanine 1 port 2 on Blade 1. Passthough C1 port 1 maps to mezzanine 2 port 1 on Blade 1. Passthough C2 port 1 maps to mezzanine 2 port 2 on Blade 1. You get the picture and there will be more detail below.
When I installed ESX 3.5 Update 4 it numbered the NIC’s vmnic0, vmnic1, vmnic2, vmnic3, vmnic4 and vmnic5. Here is the mapping:
Onboard 1 => VMNIC0
Onboard 2 => VMNIC1
Mez 1, NIC 1 => VMNIC2
Mez 1, NIC 2 => VMNIC3
Mez 2, NIC 1 => VMNIC4
Mez 2, NIC 2 => VMNIC5
So, to bring it all together the mappings would look like this:
BLADE 1 (ESX -1)
A1-1 => VMNIC0 => Switch1 port 13
A2-1 => VMNIC1 => Switch1 port 14
B1-1 => VMNIC2 => Switch2 port 13
B2-1 => VMNIC3 => Switch2 port 14
C1-1 => VMNIC4 => Switch1 port 15
C2-1 => VMNIC5 => Switch2 port 15
BLADE 2 (ESX-2)
A1-1 => VMNIC0 => Switch1 port 16
A2-1 => VMNIC1 => Switch1 port 17
B1-1 => VMNIC2 => Switch2 port 16
B2-1 => VMNIC3 => Switch2 port 17
C1-1 => VMNIC4 => Switch1 port 18
C2-1 => VMNIC5 => Switch2 port 18
BLADE 3 (ESX-3)
A1-1 => VMNIC0 => Switch1 port 19
A2-1 => VMNIC1 => Switch1 port 20
B1-1 => VMNIC2 => Switch2 port 19
B2-1 => VMNIC3 => Switch2 port 20
C1-1 => VMNIC4 => Switch1 port 21
C2-1 => VMNIC5 => Switch2 port 21
I then configured the the vSwitches like this:
vSwitch0 (Front end Service Console/VMKernel)
vmnic0
vmnic2
vSwitch1 (VM Network / Load Balance IP Hash)
vmnic1
vmnic3
vSwitch2 (Back end service console/VMKernel / Load Balance IP Hash)
vmnic4
vmnic5
The 3750’s are configured like this:
Port-Channel 1
Uplink to Core
1/0/1
1/0/2
2/0/1
2/0/2
Port-Channel 10
ESX-1 VM Network
1/0/13
2/0/13
VLAN 100
Port-Channel 30
ESX-1 iSCSI/Backed VMKernel and Service Console
1/0/15
2/0/15
VLAN 500
The port-channels would be repeated as 11, 31, 12 and 32 for ESX-2 and ESX-3.
-mwm
Sphere: Related ContentFirst Dell Blade Chassis
This will be the first in a multipart series about the installation and configuration of a VMware ESX 3.5 cluster using a Dell M1000e blade chassis, Dell/Equallogic PS5000e SAN and 2 Cisco 3750G-24TS switch stack.
I am working on my first ESX cluster install on a Dell M1000e chassis with 4 m600 blades. 3 of the blades will be bound for ESX with on designated as a virtual center blade. I really wanted to install vSphere 4 in this case but the client purchased two Dell/Equallogic PS5000e 16tb iSCSI SAN’s with the intent of leveraging VMware SRM at a fiber connected warm site as well as possibility using View. View and SRM are net yet support on vSphere 4 and didn’t want to leave the client dependent on vapor ware. I’m sure VMware will get this done, just not sure when.
DAY 1
I arrived on site at about 1:00pm after a two and half hour drive. The first order of business was inventory of the hardware. The 3 ESX m600 blades can pre configured with dual quad core 2.8 GHZ processors, 32 gigs of RAM and mirrored 73gb drives. The m600 has 2 on board NIC’s as well as two mezzanine slots. As Dell has not yet released an iSCSI mezzanine card, these blades each had two additional dual port NICs for a total of 6. The 4th blade is an m600 with a single quad core proc, 2 gigs of RAM and mirrored 146gb SAS drives. Each SAN is a 16TB PS5000e from Dell/Equallogic with dual 3 port controllers.
For switching we specified dual Cisco 3750G-24TS (actually 4 including the 2 for the warm site). These are to be stacked using 2 stacking cabled for stack redundancy.
After inventory of all hardware it was time to rack it up. I did the SAN myself in the lower part of 4 post enclosed rack. Above the SAN I installed the 2 switches. I placed in the rear posts of the rack facing backwards. This will help conceal the massive amount of patch cables that will eventually run from the 6 m1000e Ethernet Passthroughs to the 3750 stack. Above the switches I installed the rails for the m1000e chassis. This thing is HEAVY. I rarely mount a device like this above waist height. This on was about chest high as the client requested. I found two other guys and walked the chassis about 20 feet to the rack and lifted it into place. With three of us it was a struggle, but it we got it in. I connected all power to the UPS and fired it all up to verify functionality. After successful power on, I powered it all off and left for the day.
DAY 2 will bring network configuration and some ESX Update 4 installs.
-mwm
Sphere: Related ContentOn to vSphere 4
So we got the M1000e from Dell. It came with 2 blades, Dual Quad Core with 32gb of RAM. Can’t beat that with a stick. Up to this point the only blade chassis I had spent any quality time with were the c3000 and c7000 from HP. We don’t have a SAN in the lab at this point but we do have a box full of disk running iSCSI Enterprise Target. Works out pretty well. We have 2 300gb VMFS volumes and the rest available for RDMs.
The vSphere installer looks very similar to the ESXi 3.5 installer. The steps have been streamlined a but and I found it quite easy and fast to install. We put vSphere on both blades and then built a Windows 2008 vm for vCenter 4. I didn’t do the vCenter install myself but the guy who did indicated it was similar and also quite similar. I did see the SQL part which was more visible than in previous versions.
Initial Feeling: The M1000e is ‘comparable’ market wise to the c7000. The Chassis Management Console (CMC) is kind of cheesy feeling but the remote console I actually like better that the HP iLO remote console. The blades to have a good positive click into the chassis. The bottom of each balde kind of hangs in breeze. Dell does not offer a half size chassis like HP does with the c3000, this will be a significant negative for Dell. My initial feeling is that the Dell is cheaper but it is not a good as the HP offerings in the blade market.
More to some as I continue to play with Data Recovery and Fault Tolerance.
-mwm
Sphere: Related ContentvSphere 4
vSphere 4 is here. I have yet to install it but rumor has it that a Dell M1000e is due to arrive at the office today. I typically install HP servers (DL, C3000…) but am looking forward to checking out the Dell offering. I’m not sure of the configuration but I am hoping to get two decent blades to work with. Most likely 2 vSphere hosts and a physical VC host. Gotta kick the tires before I attempt an install at a client location.
Data Recovery in vSphere is quite interesting. It runs as a VM on an ESX (vSphere) host. I have been usning vRanger and Backup Exec for a couple years to get the backups needed to ensure recovery. Sounds like vSphere data recovery might do the trick and potentially replace both of these tools. Gotta check it out soon.
I hope to have vSphere installed by early next week. I’ll be posting more about it then.
-mwm
Sphere: Related ContentStream to XBOX 360
So, as you can see in my previous, Vista is no longer a resident in my house. There have been a few stumbling blocks that I’d like to talk about. Nothing that I can’t overcome, just interesting.
The fall XBOX 360 dashboard update introduced better streaming support to the 360. Getting a file into the correct format has proven frustrating. In typical Microsoft fashion, only specifically encoded video files will play. I have gotten a few to work so far. using HandBrake seem to be the best fit although I cannot seem to get it to produce the output needed every time but I am sure this is a combination of me and the source files. Out the window with Vista went Windows Media Center. It was one of the better, cooler features of Vista. I am the lookout now for a comparable package for Linux.
Itunes. This was a major issue as we have an iPod and my wife manages it and our music via iTunes. No iTunes for Linux. I made a few attempts at running iTunes via Wine with varying degrees of success. None of the success enough to make it work though. I settled on Banshee. Banshee sees my iPod just fine and allows me to manage it. next I had to un-itunes my music collection. I did this using SoundTaxi. Works great. Having what sounds like bit rate issues with some conversions but overall it is great software.
I’ll post more as I get more and more stuff working.
-mwm
Sphere: Related ContentVISTA NO MORE!
Well, I did it. I blew VISTA away. If I could I would smash it into little tiny pieces and send it back to Microswift in a little coffin. It simply does not work and seems to be more of a documented bug than an operating system. If a major car company released a car that runs like VISTA, they would be closed down by the feds.
What did I do. LINUX! I dug around to find the best family oriented distro. I settled on UBUNTU 9.04. I needed something that the kids and my wife could use easily. My Linux distro of choice for years now has been Centos. Ubuntu just makes it a little easier for noobs. The only thing I lost along the way was my wifes email, although I did manage to save her contacts which proved to be more valuable to her than the email history anyway.
I was even able to get a UPnP media server running pretty quickly. I used uShare to get this done. It seems to be easiest to configure in conjuntion with the XBOX 360. Getting media into a format the Microswift XBOX 360 will play with is a bit of pain though. I ended up using HandBrakeGTK to convert video to a format the 360 would actually play. Still messing with the encoding settigns.
It’s time to get the rest of the world moving away from Winders. I dont see the OS getting any better any time soon. I am sick of the ‘It’ll be better in the next version’ crap. It never is. As soon as Vista was common place everyone was saying ‘Winders 7 will be better and fix a lot of these issues’. Not gonna happen. I have ZERO faith that Winders 7 will be any better than 3.1, 3.11, 98, Me… XP wasn’t bad but it took 3 major service packs (aka OS rewrites) to get there. What did they do when they got there? Released Vista!
Windows Vista sucks.
-mwm
Sphere: Related ContentVista Still SUCKS
So, Vista reports 5.00gb of RAM in System Properties. When I look at task manager I am reported 2.9gb of Physical memory.
WHY!?!?!
Is it PCI peripherals mapping below the 4gb limit? Dunno. I am really, really disgusted though. I am in the process of creating an Ubuntu VM using VMware Workstation 6.5. I gave the VM 3gb of RAM thinking it would leave the OS with 2gb. NOT! I am VERY close to just blowing Vista away entirely to get to Linux.
I want to give the family time to use it before I do it though, thus the VM aproach
-mwm
Sphere: Related ContentCitrix Presentation Server 4 to 4.5
I am currently working on a project with the follwoing goals:
- Migrate 10 CPS 4.0 servers to XenApp 4.5
- Install new licensing server
- Migrate Password Manager central store from AD to NTFS
- Migrate TS Profiles to new server
There are another minor goals but that is the general jist of it all. I have encountered a funky issue with the license server. The FARM license server setting is SERVER-A. This is fine. As I migrate a server to 4.5, I point the server to the new license server by overriding the farm setting at the server level. It finds the license server just fine and is happy. After a bit of time and perhaps a log on/off to the upgraded server, I get an error indicating the license server is not found. When I check the license server setting for the particular CPS server, the check box indicating to use the farm settings is once again checked!
I am just beginning the search into why but wondered if any readers had perhaps encountered this in the past. I’ll post a fix once I find one.
-mwm
Sphere: Related ContentVista and VMware Workstation
Crash! As I stated previously I have a decent home PC. 5gb of ram and a 64-bit processor running Vista Home Premium. I installed VMware Workstation 6.5 today and built an Ubuntu VM. I gave the VM 2048mb of RAM. Vista didn’t have CLUE how to run with only 3gb of RAM.
I fired up Windows Movie Maker to try to convert some avi files to wmv. I need mwv to use my XBOX 360 as a Media Center Extender. As far as I can tell wmv is the only format the 360 will play. Anyway, Movie Maker crashed 3 times. Each time it was before the application was even fully loaded. That is REALLY bad. Firefox ran terribly also. Just amazes me that Microsoft released on OS that simply sucks are really doesn’t work!
-mwm
Sphere: Related ContentGoogle Apps Migration
One of my latest projects is to migrate an organization of 650 users from an email environment consisting of a Cyrus IMAP server (most users), and two Exchange 2003 servers to Google Apps. The two Exchange 2003 servers are in seperate domains behind the same firewall. The internal mail routing alone is quite complex as the Cyrus server recieves all mail and then routes to either itself or one of the Exchange 2003 servers depening on the user. I started with getting the users exclusive to Cyrus moved to Google Apps (gMail).
The flow of mail had to allow for groups of users to be migrated in batches of 20 or 30 users. Mail destined for a migrated user had to get to that users mailbox at Google Apps. Mail destined for a non-migrated user had to get to the mail server at domain.com. Internal non-migrated users have to be able to email users that have been migrated. Here is what I came up with:
- Change DOMAIN.COM’s MX records to be those of Google Apps
This sends all mail for DOMAIN.COM to Google Apps - Configure DOMAIN.COM’s internal mail server to route mail for migrated users to Google Apps server.
This step required me to create a new, internet routable subdomain of domain.com. I chose MIG.DOMAIN.COM. When creating this subdomain I made it’s MX Records those of Google Apps. This means that all mail destined for user_a@mig.domain.com would be routed to Google Apps. - Configure Google Apps to accept mail for MIG.DOMAIN.COM and put the mail in the corresponding users mailbox on the Google Apps server. At Google this is called a Domain Alias.
- Configure Google Apps to send mail arriving at Google Apps for an unknown user to the old MX records for DOMAIN.COM.
This allows for mail sent to unknown_user@domain.com to be sent on to the internal mail server at DOMAIN.COM. This mail would include users that have not yet been migrated to Google Apps. - Configure internal Cyrus server to send mail to MIG.DOMAIN.COM for users that have been migrated to Google Apps.
This allows users still on the internal mail server at DOMAIN.COM to email users that have been migrated to Google Apps. - DOMAIN.COM’s internal DNS servers are authoritative over the internal domain name DOMAIN.COM. This means that internal mail sent to a user at MIG.DOMAIN.COM will not be deliverable as DNS will look at mig.domain.com as a host and not find it. I had to create a new internal DNS zone for MIG.DOMAIN.COM and put the Google Apps MX records in it.
**UPDATE**
I have had to alter routing sue to issues with Postini. I have set the domain MX records to Postini and then set postini to forward mail to the Google Apps MX records. After the migration is complete I’ll set the domain MX records to Google Apps.
