Two local customers have ran into the same situation in the past month….both are running on a HP c-Class blade infrastructure leveraging Virtual Connect/FLEX-10 as well as on-board Broadcom 10G cards leveraging the bnx2x driver – this mixed with ESX(i) 4.1 is an almost lethal combination.

Random network drops, port flapping, dropped packets….not good for a virtual infrastructure. Luckily HP is aware of the issue and has released Advisory c02476622 which states;

The Broadcom bnx2x VMware ESX Driver Version 1.54 does not function with HP Virtual Connect Device Control Channel (DCC) and SmartLink features on ProLiant and Integrity server blades configured with the NC532m or the NC532i adapter running firmware version 2.2.6. After installing or upgrading VMware ESX/ESXi 4.1 the following functionality is either not installed or is lost:

1. New installation – DCC and SmartLink functionality is unavailable in an HP Virtual Connect environment with the NC532m or NC532i Network Adapters after installing VMware ESX/ESXi 4.1.
2. Upgrade installation – If the bnx2x Asynchronous Driver Update CD version 1.52 was previously installed on a VMware ESX/ESXi 4.0 host, DCC/SmartLink capabilities will be lost after upgrading to VMware ESX/ESXi 4.1, which will overwrite the bnx2x driver version 1.52 with version 1.54 that is included with the base VMware ESX/ESXi 4.1operating system.
3. Network failover – ProLiant and Integrity server blades hosting VMware ESX/ESXi 4.1 may lose network failover capabilities that use the VMware ESX NIC teaming failover policy (vSwitch setting) “Link Status only.”

There is a work-around that leverages VMware Beacon Probing but honestly this has been a hit/miss type work-around.  In my opinion, stick with ESX 4.0 until HP works out the issue with the firmware/driver.