Improving the Scalability of the eG Manager

Where the eG manager has to support a large number of components or user connections, you can employ a few simple hacks to quickly improve the scalability of the manager and the system hosting it.

Increasing the Memory of the eG Manager

The eG manager runs as a Java process. Typically, 1/3rdof the total system memory is the maximum heap memory that can be allocated to the eG manager process.

Where a large number of components are to be monitored, you may want to allocate more memory heap to the eG manager process.

In such a case, follow the steps discussed below on an eG manager on Windows:

  1. Login to the eG manager host.
  2. Edit the <EG_INSTALL_DIR>\lib\setEnv.bat file.

  3. Search for the following entries:

    @set XMX=

    @set XMS=

  4. The XMX and XMS specifications govern the heap memory allocations to the eG manager. If you want to increase it to say, 8 GB (i.e., 8192 MB), change these specifications as indicated below:

    @set XMX=8192

    @set XMS=8192

  5. Finally, save the file.

On a Unix manager, follow the steps below to modify the heap memory allocation:

  1. Login to the eG manager host. Edit the catalina.sh file in the /opt/egurkha/manager/tomcat/bin directory.

  2. Search for the XMX parameter in the file. This parameter specifies the maximum memory allocation to the eG manager process. Increase the value of the XMX parameter to increase the memory cap of the eG manager process. For instance, if you want the eG manager process to use a maximum of 2 GB of memory, then your XMX specification will be as follows:

    XMX=2048m

  1. Next, search for the NEWSIZE parameter in the file. This parameter specifies the minimum size of young generation which is allocated at initialization of JVM. By default, this should be 1/4th of the maximum memory allocation. This means, if you increase the XMX value to say 2048 MB, you should increase NEWSIZE to 512 MB. In this case, the NEWSIZE specification should be as follows:

    NEWSIZE=512m

  2. Then, save the file.

  3. Finally, restart the eG manager.

While overriding the default heap memory allocations to the eG manager process, ensure that the allocated heap memory is not greater than the total memory capacity of the eG manager host.

Adjusting TCP Settings for Heavy Load on Windows Managers

The underlying Search architecture that directs searches across multiple physical partitions uses TCP/IP ports and non-blocking NIO SocketChannels to connect to the Search engines. These connections remain open in the TIME_WAIT state until the operating system times them out. Consequently, under heavy load conditions, the available ports on the machine running the Routing module can be exhausted.

On Windows platforms, the default timeout is 120 seconds, and the maximum number of ports is approximately 4,000, resulting in a maximum rate of 33 connections per second.

If this rate is exceeded, you may see failures as the supply of TCP/IP ports is exhausted. Symptoms include drops in throughput and errors indicating failed network connections.

Note:

This problem does not usually appear on UNIX systems due to the higher default connection rate in those operating systems.

To increase the scalability and performance of a Windows manager therefore, you should make sure that the TCP ports of the Windows system do not get exhausted frequently and the connection rate remains high. For this, it is recommended that you reduce the TIME_WAIT value, and increase the Keep Alive time for TCP connections and the port range. To achieve this, follow the steps below on the Winds system hosting the eG manager:

  1. Use the regedit command to access the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\ Services\TCPIP\Parameters registry subkey.

  2. Create a new REG_DWORD value named TcpTimedWaitDelay, if such a value does not pre-exisit in the registry.

  3. Set the value to 60.

  4. Next, create a new REG_DWORD value named MaxUserPort, if you do not find such a value in the registry.

  5. Set this value to 65535.

  6. Then, check for a REG_DWORD value named KeepAliveTime, under the Parameters registry subkey. If you do not find such a value, then create one.

  7. Set the value to 300000.

  8. Finally, stop and restart the Windows system.