SDX 16000 Model
The new NetScaler SDX 16000 appliance is a 2U appliance. This platform has two 16-core processors and 256 GB (16 x 16 GB DIMM) of memory. The appliance provides a total of eight 25G SFP28 ports and four 100G QSFP28 Ethernet ports. Quite nice hardware for running your VPX appliances.
Hardware is based on Supermicro motherboards. These new hardware NetScaler models finally feature the updated and reskinned IPMI Interface of Supermicro (BTW, I'm big fan of Supermicro's motherboards).
You can find the front panel look below. Additional information is available on Citrix doc.
IPMI High Availability
The new interface is quite nice and while we're here let's discuss the Supermicro's IPMI HA feature in the SDX context.
The IPMI HA Feature is controlled via the below 3 options (you can find them in Configuration > Network > General):
The options control how you access the IPMI Interface.
- Dedicated - (LOM PORT)
- Shared - (0/1 - Management PORT)
- Failover - Dynamic Detection
Dedicated: Always use the dedicated IPMI interface (LOM Port). This is the option you want if you're trying to have the simplest setup, at the expense of additional cabling and manual intervention required to restore the IMPI connectivity in case of hardwre failure.
Shared: Always use the 0/1 (Management Interface) interface. This is the option you want if you're trying to reduce your cabling to each server, and understand the tradeoffs. Under the covers, there's a virtual switch in hardware that's splitting out traffic to the IPMI card from traffic to the rest of the system; the IPMI card has a separate MAC address to differentiate the traffic. On modern Supermicro boards, you can also set the IPMI traffic to run on a different VLAN from the rest of the system, so you can tag the IPMI traffic. If the IPMI is on different network from the SVM/XEN, you'll need to tag the traffic. A failure of the 0/1 interface will mean losing access to the IMPI. 0/2 will still be available for SVM, VPX, XenServer access. Again manual intervention required to restore the IMPI connectivity.
Failover: On boot, detect if the dedicated IPMI interface is connected. If so, use the dedicated interface, otherwise fall back to the shared 0/1. You want to keep the this setting for the IPMI HA to work. The HA is formed between 0/1 and the LOM Port.
Configuring the IMPI interface can be done as described in Citrix docs.
Supermicro's default setting is "Failover", while Citrix defaults the appliance to "Shared" for LOM firmware version before 3.x.
Citrix doc.
In either case, if would like to change the settings of the IPMI to be different from the default (Shared), I would recommend using the GUI as the available IPMI tool in the XenServer requires input that's motherboard's specific.
The IPMI GUI is greyed out by default for the nsroot user.
If you want to unlock it, you should run the below command from XS/Dom0 shell:
/usr/sbin/sdx_bmc_unlock.sh
For different motherboard models the ipmi cli is diffrent. If for some reason you need to use the ipmi cli raise a case to Citrix or don't change the setting!
\\Motherboard: X8DTT-IBQF
\\Example commands (do not use):
\\Traffic goes through Dedicated LAN
ipmitool raw 0x30 0x70 0xc 1 1 0
\\Traffic goes through Onboard 0/1
ipmitool raw 0x30 0x70 0xc 1 1 1
\\Failover(default setting)
ipmitool raw 0x30 0x70 0xc 1 0 0
The above commands are taken from Supermicro for specific motherboard model. Citrix does not provide us motherboard model information!
AST2500 Block Diagram below illustrating the connectivity between 0/1 and LOM Port:
I hope this has been informative to you!