VMware Reservations and Avaya Aura Communication Manager

With great power comes great responsibility..... Avaya Aura applications can now be deployed onto multiple virtualization platforms from Google Cloud, AWS and VMWare to name a few options. With these new deployment options care still needs to be taken to ensure that the applications have the needed resources (CPU, Disk, Memory and Network) they need to run properly.

In this blog post we want to show what can happen to Communication Manager servers when we do not follow the recommended practice of leaving the memory and CPU reservations in place that are requested at the time of deployment.

An issue we received in our service desk
We had duplicated Communication Manager servers running on VMware that interchanged and became unstable after the interchange. The servers were running Avaya Aura Communication Manager 8.1.1 with the service packs shown below.

Output from the swversion command below

The issue occurred when the servers transitioned between active and standby, the transition times are shown below by looking at the restartcause command from Linux.



Knowing that the issue occurred at 2020-0324-06:57 we looked at the log file that was created before this time stamp in /var/log/ecs/
We saw the following in the file highlighted in red.



This would normally signal an issue with the network but these two Communication Manager servers are virtual machines connected by a virtual network so there is no real physical network between them. Due to the high speeds and low latency of these virtual networks, it is not very likely that an underlying network is truly the cause of our issue.

Cause of our issue?
Since these are virtual machines it is important that we check the reservations for CPU and memory that are given to the virtual machine. These reservations essentially set aside a certain amount of memory and CPU from VMware and allocate it to our virtual machines so it is always available. This way if the resources on the physical servers ever get scarce we can be assured that our Avaya Communication Manager Virtual Machines always have what they need.

Checking our resources
In order to check the resources available to each virtual machine, we can run the following commands on each Communication Manager server.

These commands should return a value that was determined at the time of deployment depending on what architecture was selected. 
For Communication Manager, you can review this link and go to page 17.

But in our case these commands returned zero so steps need to be taken to shut down the virtual machines, adjust the CPU and Memory reservations in VMware according to the referenced link and turn the virtual machines back on.
Verify with the vmware-toolbox-cmd commands after the VMware adjustments have been made to verify they are correct.

This will give the virtual machine the correct resources that it needs to operate correctly.