Tag Archives: DELL

EqualLogic Multipathing Extension Module – Installing

Last year I wrote a post on an issue attempting to install the DELL EqualLogic Multipathing Extension Module using VMware Update Manager.  I discussed an alternative method to VUM using the CLI to install the MEM.  The post has turned out to be fairly popular.  I’m guessing though that most people are more interested in how to install the EqualLogic MEM using VUM rather than my original workaround.  So I thought I would run through the steps using a version of MEM that now works.  The whole process of importing, attaching, and remediating came out a little longer than expected but I managed to capture all the steps in what I think is fairly easy to follow.

The version of MEM I am using is 1.1.2 (released Dec 2012).  You can obtain it from the EqualLogic support site (sign-in required).  The release notes state that the only change from version 1.1.1 is that it’s now compatible with Update Manager 5.1.  EqualLogic also state that if version 1.1.1 is installed 1.1.2 is not required.  At least this now explains why I had trouble with VUM and version 1.1.1

Using the vSphere Client under Solutions and Applications select Update Manager and click on the Patch Repository tab.

mem01a

Click on Import Patches.  Browse to the location of the patch.  Select the version you want.  In my case for ESX5

*Note: The ZIP file from the EqualLogic support site needs to be extracted prior to importing.  Once extracted there will be two zip versions.  An ESX4 and an ESX5 version.

mem01b

If the Upload is successful you’ll then be asked to confirm the Import.

mem01c

Once imported scroll to the bottom of the repository list and you should see the new Host Extension.

mem01d

With the extension imported into Update Manager we now create a new Baseline.  Click on the Baselines and Groups tab.

mem02

Click on Create to create a  new baseline.  Assign a name to the baseline and a description.  For the Baseline Type select Host Extension and click Next.

mem02a

Scroll to the bottom of the list and select the recently imported MEM patch.  Click the down arrow to Add the Extension and click Next.

mem02b

Confirm that the correct extension was selected and click Finish to create the baseline.

mem02c

With the patch imported and a new Baseline created for the Extension we now have to Attach the baseline.  This can be done at the top of the vCenter level or right down to the Host level.  In this case I just want to do a single host.  So I’m going to select the host and then select the Update Manager tab.  I’m then going to click Attach.

mem03a

Select the newly created baseline and click Attach.

mem03b

The baseline will now appear with a Question Mark beside it until a new scan is performed.  Click Scan, make sure Patches and Extension are selected and click Scan again.

mem03c

Once the scan is complete the Extension will now show up with a red cross signifying that it’s missing and needs to be Remediated.

mem03d

Click the Remediate button to start the process.  Select Extension Baselines on the left and the recently created Baseline on the right.  Then click Next.

mem03e

Omitted is a number of steps from the Remediate Wizard.  The options revolve around how the host and cluster will behave in Maintenance Mode.  The options are fairly straight-forward and the default options usually suffice. The last screen will summarise the options selected.  Make note what options have been selected and that the correct Baseline is selected.  Click Finish to start the Remediation.

mem03f

The host will now enter Maintenance Mode using the options you selected above.  Once complete we can select a datastore and select pathing where we can see a new pathing option and it’s selected by default. We will also see that all paths to the LUN are Active.

mem04

The whole importing and creating a baseline can seem a little tedious at first, but once done, all that’s needed is a scan and remediate on new hosts.

References

Link to original article EqualLogic MultiPathing Extension Module -- Alternative Install

Download the latest Extension module from EqualLogic Support Site

EqualLogic HTTP vs HTTPS vs Encrypt Communication

This week I realised that I had the option to log into an EqualLogic Web Management Portal with either HTTP or HTTPS.  It got me thinking what effect that has on the Encrypt Communication checkbox during login.

encrypt00

EqualLogic login prompt running Firmware 5.x

Under default configuration of an Equallogic array you have the option to use/select Encrypt Communication during login.  This can be changed and you can force the use of this option.

encrypt06

Under Group configuration select the Administration tab.  You will see that Web access is enabled and under GUI access that the checkbox for Allow only secure communication is unticked.  Ticking this box will force the use of Encrypt Communication during login.   You will then notice that Web access will change to Secure Only.

encrypt07

The above screenshot is running on Firmware 6.x.  On Firmware 5.x the checkbox is called Allow only secure SSL communication.  Oddly enough once enabled on either firmwares this won’t prevent the use of HTTP access to the Web Management Portal.

Now when attempting to login you will have to use Encrypt Communication.  Under Equallogic PS Series 5.x Firmware you have to select the checkbox.  If you don’t you will receive an error when attempting to login.

encrypt03

Under PS Series Firmware 6.x the checkbox will be selected by default and greyed out.  So you won’t get the above message.

encrypt04

As mentioned above, HTTP web access is still possible along with HTTPS.  So what’s going on here?!?!  Hence the reason for this post…

So I fired up Wireshark to watch  communication between my PC and the EqualLogic Array.  I first tried accessing the Web Management Portal with HTTPS and logging in  using the Encrypted Communication checkbox.  I then tried again but this time using Encrypt Communication.  No surprise here, both times all traffic was encrypted right from sign-in.

Next I accessed the Web Management Portal using HTTP,  not using Encrypt Communication, and signed in.  Looking through the Wireshark logs I could see my username and password in plaintext (certainly not recommended).  Again using HTTP to access the Portal I enabled Encrypted Communication and signed in.  This time looking through the Wireshark logs I could see my sign-in details were encrypted and all subsequent information as well.

From what I can see going on here is that the EqualLogic Web Management Portal is a Java Applet.  When loaded a connection is established over port 3002 on Firmware 5.x and Port 3003 on Firmware 6.x.  When Encrypt Communication is selected during sign-in, SSL encryption is handled by the Java Applet.  When not selected during sign-in SSL encryption is determined by whether you use HTTP or HTTPS and relies on the browser securing communication.

So if using HTTPS to access the Management Portal you’ve relatively sure your communication is secure but you can’t guarantee other admins are doing the same.  The safest thing to do is always enable the checkbox in the Administration tab Allow only secure communication.  By enabling this option you can be sure that whether administrators use HTTP or HTTPS all communication to the EqualLogic Array will be secure.

Enable EqualLogic Active Directory Authentication

If you’re using a DELL EqualLogic SAN you have the ability to turn on Active Directory authentication.  The benefit once setup is that you can control access to the SAN via AD groups rather than giving out the Group Admin account or maintaining local accounts on the EqualLogic group.

The process to turn on Active Directory authentication is quite simple.  Whether AD authentication is on or off you can still use Local authentication and locally created accounts.  So if you do lose connectivity to AD you will still be able to local on with the default grpadmin account or any other local accounts that you have made.

To begin, login to the EqualLogic Group Admin webpage with local Group Administrator permissions.  On the left select Group Configuration then navigate to the Administration Tab.  It should look similar to below with ‘Local Only’ set as the authentication type.

equal_ldap010

Select the Active Directory radio button.  A new window will appear similar to below.

equal_ldap02

Click Add and type in at least one IP address of an AD Server.  If you have more than one (which you should) you can click Add again and input multiple servers.  The EqualLogic san will connected against the first AD server in the list and if unable to connect will work its way down the list.

On the right you can leave the Secure protocol as none and Use Default Port.  If you’ve successfully put in the correct AD server IP addresses you should be able to click on Get Default and the Base DN should be automatically populated with you the root DN of your AD domain name.

For the User you will need an AD account.  Open up Active Directory Users and Computers.  Create a basic user with no special rights.  Set the account not to expire.  Make sure it has read access into AD (by default all user accounts will have this).  Back in the EqualLogic Group Administrator use this new account created.  Use the full domainusername format.

equal_ldap04

Click the Test AD Settings.  A new window will appear and make a connection to each AD Server you added and perform a test search using the User you just created.

Hopefully all green ticks will be returned and you can click Ok and return to the AD Settings window.  If you receive a red cross and a fail double check the IPs of the AD servers.

equal_ldap05

Click OK and one final window will open asking to join the EqualLogic group to the Windows domain.  You can choose to Cancel this step if you don’t wish to use Single Sign On.  If you proceed, for the username enter in an Administrator account that would have permissions to add workstations to the domain and click ok.

equal_ldap06

If successful you will see the Group name added as a computer in AD Users and Computers under the Computers OU.

The Administration tab should now look similar to below.  You will have a new Active Directory Status section with a couple green ticks indicating that you have successfully added in an AD server and the Group was successfully created as a computer object in AD for SSO.

equal_ldap03

That’s all there is to it.  You can now click on Add to add a new users or group from AD.  A window will appear giving the option to create a standard local user account but now the radio buttons to create an AD user or group are available.

The process is the same for a user or group.  Select the Add a new Active Directory user radio button.  Under General settings type in the username of the AD User omitting the domain name.  You can click on Check name which will verify with a green tick that the user does indeed exist in AD.

equal_ldap07

Click next and specify the permissions you want the account to have.

equal_ldap09

Click next again, verify the details and permissions you have set for the account then click finish.

equal_ldap08

If you choose to use an AD group.  I recommend first creating a Domain Local group first in AD.  Populate this group with the users you want to have access.  Then run through the steps above but select Add a new Active Directory group.

EqualLogic MultiPathing Extension Module – Alternative Install

UPDATE: If you’re looking for step-by-step instructions to install the EqualLogic Multipathing Extension Module click HERE to go to my most recent post on the topic.  

UPDATE: Note that the ESXCLI command below  —depot has a double dash.

I recently ran into an issue where vSphere 5.1 Update Manager scans and detects the latest Dell EqualLogic Multipathing Extension Module 1.1.1 as Not Applicable for ESXi 5.1 and will not select the patch during a Remediate.  Below I show how to install this extension via the ESXCLI.

If you’re running a DELL EqualLogic SAN with ESX you should be running DELL’s Multipathing Extension Module (MEM).  MEM is a Path Selection Plugin (PSP) driver for ESX.  In fact no matter what SAN you have you should investigate if they have a PSP driver for ESX.

ESX have three built-in pathing options, (Most Recently Used, Round Robin, and Fixed).  By installing EqualLogic’s MEM you get a fourth option called DELL_PSP_EQL_ROUTED.  EqualLogic PS Series SANs can run Active / Active pathing.  By installing the MEM, ESX can be made aware of this and can load balance appropriately.  This can all lead to increase in bandwidth utilisation and lower network latency.

Since ESX4 I’ve been installing the EqualLogic MEM using VMware Update Manager through the vCenter C# client.  I’ve never had any issues right up until and including ESXi 5 Update1.  The process is quite simple.

  1. Import the MEM as a patch.
  2. Create a new Baseline with a Host Extension and add the MEM extension that was imported. (EqualLogic have different versions for each ESX version so be mindful).
  3. Attach the baseline to the Host
  4. Perform a Scan and Remediate the ESX host.
  5. Reboot.

As mentioned in the beginning, Update Manager 5.1 saw the patch at Not Applicable.  I took this as an opportunity to try and install the patch through the ESXCLI.  To do this I used VMA.  If you’re not running or haven’t tried the VMware Management Assistant (VMA) it’s worth looking into.  It’s a nice convenient way to get CLI access to all your ESX hosts.

To install a PSP driver the host needs to be in Maintenance Mode.  So do this first or you’ll get a similar error to below.

Next transfer the EqualLogic Multipathing files to a location on the ESX host you want to install to.  In my case below I installed them to a folder on datastore1.

Back on the VMA use the ESXCLI and enter the following command, substituting the file location for your own, to install.

Esxcli —server=my_esx_host.domain software vib install —depot /vmfs/volumes/datastore1/EqualLogic-ESX-Multipathing-Module/dell-eql-mem-esx5-1.1.1.270268.zip

This is a kernel driver and a reboot is required for the PSP driver to successfully apply.  Once a reboot is performed the new PSP becomes the default selection.

Now selecting a datastore and selecting pathing you can see a new pathing option and it’s selected by default.  You will also see that all paths to the LUN are Active.

Appendix

Most recent step-by-step MEM installation article EqualLogic Multipathing Extension Module – Installing

VM Windows Cluster Volumes Offline in ESX

Windows Clustering on physical hardware is a pain at the best of times.  Just getting it to work can sometimes be a little try and effort… with a whole lot of luck.  Getting clustering to work in VMware is just cruel.

So when tasked to create a VM of a physical Windows Cluster for a test environment, boy was I excited! {Sarcasm sign}.

Actually creating the VM within ESX wasn’t that difficult.  Using Converter I created a VM of the OS.  Then using our DELL EqualLogic SAN I made clone copies of the cluster volumes.  I presented those volumes with the newly created VM as RDMs.  The process seemed to work really well until.  The OS booted up.  I could see all my presented volumes.  Issues began when I tried to start the Clustering Service and take it out of manual mode.  Out of the 6 volumes I had only one would ever become Online while all the others would (after some time) fail.

I spent days working through the issue (I’m pretty sure this is why I’m balding).  Articles seemed to lead me to DISKPART and trying to change the SAN Online Policy, manually online the disk, changing the READONLY attribute.  None of these seemed to work.  I’m assuming because there was an attribute that said the disk was Clustered and would prevent me making any changes.  Still, I thought I was on the wrong ‘path’ and began looking into a lower level issue at the ESX level.

The crux of my issue turned out to be a iSCSI multipathing problem.  DELL EqualLogic SANs run in an Active / Active pathing method where I/O is sent over all paths.  DELL has a third party Storage API plugin for ESXi that change the default behaviour of how mutlipathing works.  This is normally a good thing but for Windows Clustering in ESX… this is bad.

The solution is fairly simple to resolve.  The steps below is a rough outline of how to identify and change the multipathing policy.

Using vSphere vCenter, the changes are made within the Storage Adaptor.  In this case it’s the iSCSI Software Adaptor under the Configuration tab.

In the bottom pane select the paths view.  Expand the Target column and identify one of the cluster volumes with issues.  In this example I have a Dead path due to a recently removed SAN volume which is safe to ignore.  The one below is of interest as it’s one of the clustered volumes.  Remember the Runtime Name in the left column.

Change to the Devices view and locate the Runtime Name.  Right click on this device and select Manage Paths.  In this example DELL_PSP_EQL_ROUTED was selected as default.  Changing this to Most Recently Used (VMware) sends I/O only ever down one path.  The change is immediate.  As my volumes are offline I can safely make the changes.  On a working production volume I wouldn’t be making path selection changes during business hours.

Back over on the Windows Cluster VM I can now restarted the Clustering Service and have it correctly Online all the volumes.

MSCS is quite in depth and not for the faint hearted or something configured before you end home for the night.  Virtualising MSCS requires additional planning and thought in addition to regular planning.

Appendix

VMware -- Setup for Failover Clustering and Microsoft Cluster Service