Monthly Archives: June 2012

Encrypted / Password ZIP email attachments stripped

I recently migrated from an Exchange 2007 Edge server to an Exchange 2010 Edge server.  I used the opportunity to not carry over some of the legacy settings and start clean.  Everything appeared fine till a few weeks later.

A user complained that their ZIP files were being stripped in attachments.  Their attachments would arrive with a TXT extension on it.  Within that TXT file it would say “This attachment was removed.”.  Having never seen this before and able to successfully email myself ZIP attachments I put it down to the senders email filtering.  The recipient was adamant that it wasn’t the senders fault.  So I managed to get a hold of the questionable ZIP files and send them to myself via Gmail, sure enough the attachments failed to arrive.  So began my investigations…

Not doing any filtering on our HUB or Mailbox servers I was immediately able to eliminate those services.  I then inspected our AV / Spam provider and confirmed via their logs that these emails were successfully passing to our Exchange Edge network.  So I focused my attention here.

Comparing the decommissioned Exchange 2007 Edge to the 2010 Edge I ran the PowerShell command Get-TransportAgent.  This outputted the below on each server.  The difference being that the “Attachment Filtering Agent” was disabled on the old Exchange 2007 Edge.

Image1. Exchange 2010 Edge

Image 2. Exchange 2007 Edge

On the new Exchange 2010 Edge I ran Get-AttachmentFilterEntry and inspected what default Attachment Filtering Microsoft had specified.  ZIP attachments was not one of them.  Never the less, as a test, I disabled Attachment Filtering with the PowerShell command

Disable-TransportAgent -Identity “Attachment Filtering Agent”

I then resent myself the failed ZIP files.  To my surprise they were successfully received.  Doing some research online seemed to indicate that this was the solution many people took to resolve this same issue.  This seemed like a pretty piss poor solution that I wasn’t going to accept.  Not if it meant that I would have to disabled all attachment filtering just for ZIP files.

I re-enabled the Attachment Filtering with Enable-TransportAgent -Identity “Attachment Filtering Agent”

After quite a few hours of playing around I finally found a viable solution.  The ZIP file I had been working with turned out to be an encrypted / password protected zip file.  Because of this the Exchange Edge server was having issues identifying the type of attachment.  By modifying the EdgeTransport.exe.config file I managed to find a workaround while continuing to maintain attachment filtering.

Solution:

1.       Go to the Edge server

2.       Stop the Transport service.

3.       Locate the EdgeTransport.exe.config file. This file is located in the following path: drive:Program FilesMicrosoftExchange ServerV14Bin

4.       Add the following entry between the <appSettings> element and the </appSettings> element of the EdgeTransport.exe.config file:

5.       <add key=”AllowInvalidAttachment” value=”true” />

6.       Restart the Transport service.

VCP today, I think?!?!

-- Part 1 --

Finally after two years I’ve managed to make the effort to sit my VMware Certified Professional exam.  In itself, not really news worthy.  So why is it up on failsys?  Well… because of the process I went through.

Anyone can sit a VCP exam, but pass or fail, to become certified by VMware you need to attend certain authorised training courses first and meet VMware’s prerequisites.  Two years back I attended the vSphere: Configure, Install, Manage [V4] course.  Now don’t get me wrong, it was a great course.  The instructor was knowledgeable and knew how to sell, present, and generally get the most out of the course material.  After this four day course I expect to be able to sit the exam.  At the conclusion of the course speaking to the instructor, someone who had sat the exam and going through practice tests, I realise that I was nowhere near ready.  The exam went well beyond the high level overview of course material on how HA worked.  You needed to know more than just the concepts about SANs, iSCSI, FC.  Stuff that I thought I had but realised might not be good enough.

VMware provide a thorough Exam Blueprint document that is intended to provide all the objectives of the exam and links to VMware documentation that cover those areas.  The amount of documents and pages to read is quite disheartening.  Fortunately work paid for the course but I still felt a little cheated afterwards.

Fast forward 18 months later, vSphere 5 had just been released, and I had still not sat the VCP exam.  In between that time I had happily studied and passed other non VMware exams.  So when a new job asked what training I would like to do chose vSphere: What’s New [V5].

Image 1. Requirements as of November 2011

What’s New [V5], at the time and still is, solely not good enough to become certified.  In late 2011 you still needed to have sat the vSphere ICM [V5] course or during the certification grace period currently be a VCP4.  I would either have to sit the VCP4 exam or talk work into also sending me on the full vSphere ICM [V5] course.

Fast forward another 6 months.  The prerequisites have changed a few times and now allows a non VCP4 holder with the right qualifying course to take the VCP5 exam.  When I was offered an exam voucher at the end of last month I decided to cram and finally take this exam.

Image 2. Requirements pre June 2012 (notice the non specific VCP4 course requirements)

Image 3. Requirements on VMware Training website June 2012

So after two years I’ve finally sat the exam and believe I finally met the complex and ever changing VCP requirements.  From what I gather the VMware certification process after passing the exam can take as long as 6 weeks.  Seems like VMware lack Microsoft’s automation in this process.

I have mix opinions on this whole process.  I’m one of those people that like certifications.  They’re certainly not, ‘the be all to end all’, but they are a nice to have.  They show that you’ve gone that extra mile when you didn’t have to.  I’ve seen many resumes over the years pass my desk.  Many students, fresh from Indian University, and more certifications on their CV than I can count on my fingers.  Too many certifications can obviously be a bad thing when not put in context.

VMware’s approach of having to meet course requirements is a good start.  It is frustrating though when those courses have little relevance to the exams.  It feels like a token donation (a large donation at that) to the training provider to achieve certification.  When you have Indian training institutes selling budget courses due to the high turnover of students and other regions around the world paying high premiums it does seem unfair.

In all honestly I’d like the see the prices of courses come down, maybe 5 day courses shorten to balance the price reduction.  To counter this, courses and / or certifications could only be obtained with a corporate sponsor.  Now lets be completely honest here.  VMware courses are very specifically directed at the experience IT worker who has been in the industry at least 24 months.  Having a corporate sponsor (not a training institute) would dramatically cut down on paper certification university students fresh out of school.

Are there better ways?

Appendix

VMware VCP Homepage

VCP5 Blueprint (Login Required)

Part 2 -- VCP Today

Login failed, reason Success

Hmmm, you suck!

I’m still learning the ins and outs of vSphere Management Assistant (vMA).  From what I have seen so far I’m liking it.  vMA is a Vmware appliance that’s been around since vSphere 4.  Since then it’s gone through a couple revisions.

I originally installed vMA 5 while looking for a way to consolidate all my ESX host log files to an easy to access repository.  Researching vMA I came across vilogger.  I thought this was great and installed the appliance.  When I went to configure vilogger in vMA I found that the command was no longer available, vilogger had been deprecated in vMA 5.  Turns out that Syslog is the new ‘in’ way of logging.

All was not lost.  It turned out to be a nice convenient way to manage all my ESXi hosts.  It contains the vSphere command-line interface and the SDK for Perl.  It can store your credentials for all your hosts making it easy to script with.

The above screenshot I’m trying to run Remote ESXTOP on the vMA.  I know VMware has some of the sorriest error messages but this was too good not to post up.  Turns out it’s better to run Remote ESXTOP specifing the FQDN of the host and not just the hostname.  The actual root cause was because the vMA was using DHCP and had a different domain name then the ESX host.

Appendix

vMA Documentation and latest versions

vSphere Management Assistant Documentation (VMware Link)

VM Windows Cluster Volumes Offline in ESX

Windows Clustering on physical hardware is a pain at the best of times.  Just getting it to work can sometimes be a little try and effort… with a whole lot of luck.  Getting clustering to work in VMware is just cruel.

So when tasked to create a VM of a physical Windows Cluster for a test environment, boy was I excited! {Sarcasm sign}.

Actually creating the VM within ESX wasn’t that difficult.  Using Converter I created a VM of the OS.  Then using our DELL EqualLogic SAN I made clone copies of the cluster volumes.  I presented those volumes with the newly created VM as RDMs.  The process seemed to work really well until.  The OS booted up.  I could see all my presented volumes.  Issues began when I tried to start the Clustering Service and take it out of manual mode.  Out of the 6 volumes I had only one would ever become Online while all the others would (after some time) fail.

I spent days working through the issue (I’m pretty sure this is why I’m balding).  Articles seemed to lead me to DISKPART and trying to change the SAN Online Policy, manually online the disk, changing the READONLY attribute.  None of these seemed to work.  I’m assuming because there was an attribute that said the disk was Clustered and would prevent me making any changes.  Still, I thought I was on the wrong ‘path’ and began looking into a lower level issue at the ESX level.

The crux of my issue turned out to be a iSCSI multipathing problem.  DELL EqualLogic SANs run in an Active / Active pathing method where I/O is sent over all paths.  DELL has a third party Storage API plugin for ESXi that change the default behaviour of how mutlipathing works.  This is normally a good thing but for Windows Clustering in ESX… this is bad.

The solution is fairly simple to resolve.  The steps below is a rough outline of how to identify and change the multipathing policy.

Using vSphere vCenter, the changes are made within the Storage Adaptor.  In this case it’s the iSCSI Software Adaptor under the Configuration tab.

In the bottom pane select the paths view.  Expand the Target column and identify one of the cluster volumes with issues.  In this example I have a Dead path due to a recently removed SAN volume which is safe to ignore.  The one below is of interest as it’s one of the clustered volumes.  Remember the Runtime Name in the left column.

Change to the Devices view and locate the Runtime Name.  Right click on this device and select Manage Paths.  In this example DELL_PSP_EQL_ROUTED was selected as default.  Changing this to Most Recently Used (VMware) sends I/O only ever down one path.  The change is immediate.  As my volumes are offline I can safely make the changes.  On a working production volume I wouldn’t be making path selection changes during business hours.

Back over on the Windows Cluster VM I can now restarted the Clustering Service and have it correctly Online all the volumes.

MSCS is quite in depth and not for the faint hearted or something configured before you end home for the night.  Virtualising MSCS requires additional planning and thought in addition to regular planning.

Appendix

VMware -- Setup for Failover Clustering and Microsoft Cluster Service