Citrix ADC Pooled Capacity Validated Reference Design

Citrix ADC pooled capacity is a licensing framework that is comprised of a bandwidth pool and a virtual instance pool that is hosted on and served by Citrix Application Delivery Management (ADM). From this common pool, each Citrix ADC in a data center, checks out one virtual instance license and only as much bandwidth as necessary. It does this regardless of platform or form factor (except for the MPX-Z, which checks out only a bandwidth license). The license file and the bandwidth are not bound to Citrix ADC. When Citrix ADC no longer requires these resources, it checks them back in to the common pool–making the resources available to other ADCs that need them.

This licensing framework maximizes bandwidth utilization by ensuring the ADCs do not allocate excess unused bandwidth. The ability of Citrix ADC to check licenses and bandwidth in and out of a common pool enables users and administrators to automate instance provisioning. Users and administrators can increase or decrease the bandwidth allocated to an instance at run time without impacting traffic. What’s more, Citrix ADC licenses in the pool can also be transferred from one instance to another, and these licenses can be shared by all form factors (MPX, SDX, VPX, and CPX).

Pooled licensing components

Pooled capacity decouples software from the underlying hardware. This approach allows a licensing model that is transferrable from existing platforms to new platforms. Pooled capacity consists of four components:

1. Zero-capacity hardware which has no bandwidth, no instances, and no features.
2. A pool of bandwidth with the software edition (standard, advanced, and premium) that can be shared across all Citrix ADC form factors–including MPX, SDX, VPX, and CPX.
3. An instance pool, which is a pool of instances shared across software/virtual Citrix ADC form factors, including VPX running on SDX, stand-alone VPX, and CPX.
4. Citrix ADM, which is used for managing bandwidth and instance licenses. This function of Citrix ADM comes at no charge to the customer.

The components of pooled capacity are discussed in more detail later in this article.

How pooled capacity works

Description

Pooled capacity is a licensing framework that decouples software from the underlying hardware. This approach allows a licensing model that is transferrable from existing platforms to new platforms, and it maximizes bandwidth utilization by ensuring that ADCs are not allocated bandwidth in excess of their requirement. The ability of Citrix ADCs to check licenses and bandwidth in and out of a common pool also enables users to automate instance provisioning.

Users can increase or decrease the bandwidth allocated to Citrix ADC at run time without impacting traffic. Users can also transfer Citrix ADC licenses in the pool from one Citrix ADC to another.

License flow

Customers purchase Citrix ADC pooled capacity licenses and download them from My Account Login Page
.

Then, these licenses are imported to Citrix ADM.

Zero-capacity hardware

When managed through Citrix ADC pooled capacity, SDX instances are referred to as “zero-capacity hardware” because these instances cannot function until they check resources out of the bandwidth and instance pools. As a result, these platforms are referred to as SDX-Z appliances.

Similarly, MPX appliances are also referred to as “zero-capacity hardware” when managed through Citrix ADC pooled capacity because they cannot function until they check resources out of the bandwidth pool. As a result, these platforms are referred to as MPX-Z appliances.

Zero-capacity hardware appliances require a platform license to check out bandwidth and/or an instance license from the common pool. Users must first install a platform license manually by using the hardware serial number or the license access code.

Currently, the following zero-capacity platforms, running Citrix ADC software release 11.1 or later, support Citrix ADC pooled capacity for new purchases and upgrades:

• MPX-14000Z

• MPX-14000Z-40G

• MPX-15000Z

• MPX-15000Z-50G

• MPX-25000Z-40G

• MPX-26000Z

• MPX-26000Z-100G

• SDX-14000Z

• SDX-14000Z-40G

• SDX-15000Z-50G

• SDX-25000Z-40G

• SDX-26000Z-100G

Currently, the following zero-capacity platforms, running Citrix ADC software release 12.0 or later (MPX) and 11.1 or later (SDX), support Citrix ADC pooled capacity for new purchases and upgrades:

• MPX-14000Z-40S

• MPX-14000Z-40C

• MPX-14000 FIPS

• MPX-25000ZA

• MPX-26000Z-50S

• SDX-14000Z-40S

• SDX-14000Z-40C

• SDX-14000 FIPS

• SDX-25000ZA

Currently, the following zero-capacity platforms, running Citrix ADC software release 12.0 or later, support Citrix ADC pooled capacity for new purchases and upgrades:

• MPX-8900Z

• SDX-8900Z

Currently, the following zero-capacity platforms, running Citrix ADC software release 12.0 or later (MPX) and 11.1 or later (SDX), support Citrix ADC pooled capacity for upgrade only:

• MPX-115xx (11515 - 11542)

• MPX-89xx/80xx

• MPX-22xxx

• MPX-24xxx

• SDX-115xx (11515 - 11542)

• SDX-89xx/80xx

• SDX-22xxx

• SDX-24xxx

Currently, the following zero-capacity platforms, running Citrix ADC software release 13.1 or later, support Citrix ADC pooled capacity for new purchases and upgrades:

• MPX-9100z

• SDX-9100z

Currently, the following zero-capacity platforms, running Citrix ADC software release 11.1 or later, support Citrix ADC pooled capacity for new purchases only:

• VPX

• CPX

Standalone Citrix ADC VPX instances

Citrix ADC VPX instances running Citrix ADC software release 11.1 or later on the following hypervisors support pooled capacity:

• VMware ESX 6.0

• Citrix XenServer

• Linux KVM

Citrix ADC VPX instances running Citrix ADC software release 12.0 or later on the following hypervisors and cloud platforms support pooled capacity:

• Microsoft Hyper-V

• Amazon AWS

• Microsoft Azure

Note:

To enable communication between Citrix ADM and Microsoft Azure or AWS, an IPSEC tunnel must be configured. For more information, see Add Citrix ADC VPX instances deployed in cloud to Citrix ADM
.

Standalone Citrix ADC CPX instances

Citrix ADC CPX instances deployed on a Docker host support pooled capacity. Unlike zero-capacity hardware, CPX does not require a platform license. To process traffic, it must check out an instance license from the pool.

Bandwidth pool

The bandwidth pool is the total bandwidth that can be shared by Citrix ADC - both physical and virtual. The bandwidth pool is comprised of separate pools for each software edition (Standard, Advanced, and Premium). A given Citrix ADC cannot have bandwidth from different pools checked out concurrently. The bandwidth pool from which a Citrix ADC can check out bandwidth depends upon its software edition for which it is licensed. When checked out from the pool, a license unlocks resources such as CPUs/PEs, SSL cores, packets per second, and bandwidth.

Instance pool

The instance pool defines the number of VPX instances or CPX instances that can be managed through Citrix ADC pooled capacity or the number of VPX instances in an SDX-Z.

Note:

The Management Service of an SDX-Z does not consume an instance.

Citrix ADM

Citrix ADC pooled capacity uses Citrix ADM to manage pooled capacity licenses: bandwidth pool licenses and instance pool licenses. Users can use Citrix ADM to manage pooled capacity licenses without an ADM license.

When checking out licenses from a bandwidth and/or instance pool, Citrix ADC form factor and hardware model number on a zero-capacity hardware platform determines:

• The minimum bandwidth and the number of instances that a Citrix ADC must check out before being functional.

• The maximum bandwidth and the number of instances that a Citrix ADC can check out.

• The minimum bandwidth unit for each bandwidth check-out. Minimum bandwidth unit is the smallest unit of bandwidth that a Citrix ADC has to check out from a pool. Any check-out must be an integer multiple of the minimum bandwidth unit. For example, if the minimum bandwidth unit of a Citrix ADC is 1 Gbps, 100 Gbps can be checked out, but 200 Mbps or 150.5 Gbps cannot be checked out. Minimum bandwidth unit is different from the minimum bandwidth requirement. A Citrix ADC can only operate after it is licensed with at least the minimum bandwidth. Once the minimum bandwidth is met, the instance can check out additional bandwidth with the minimum bandwidth unit.

The following tables summarize the maximum bandwidth/instances, the minimum bandwidth/instances, and minimum bandwidth unit for all supported Citrix ADC platforms:

Minimum system requirement: MPX and SDX

The minimum bandwidth and instances for SDX applies to versions 11.1 64+, 12.1 54+, 12.0 63+, 13.0 41.X+ and later.

For Citrix ADC CPX models

For Citrix ADC VPX on Hypervisors and Cloud Services

The license requirement for different form factors

For more information on supported platforms, minimum bandwidth/instances supported, maximum bandwidth/instances supported and minimum bandwidth unit for supported platforms, see Citrix ADM License Server
.

Configuring Citrix ADC pooled capacity

Pooled capacity allows users to:

• Allocate licenses in the license pool to Citrix ADC on demand.
• Upload the pooled capacity license files (Bandwidth Pool or Instance Pool) to the ADM.
• Allocate the licenses from Citrix ADM according to the minimum and maximum capacity of the instance.

Citrix Application Delivery Management (ADM)

Users can configure the Citrix ADM as a license server for Citrix ADC pooled capacity. There are two ways for a Citrix ADC instance to get bandwidth and/or instance licenses:

• The first license checkout request should be initiated from Citrix ADC (SDX/MPX/VPX) to obtain its bandwidth and/or instance licenses.
• Users can initiate license checkout from Citrix ADC or Citrix ADM later.

Note:

Pooled capacity is displayed on Citrix ADM only if pooled licenses are added to the Citrix ADM.

Citrix ADM license pool status

• Allocated: The license state is fine.

• Grace: Citrix ADC instance is in the license grace period for 30 days.

• Sync in progress: Citrix ADM fetches information from Citrix ADC in a 2-minute intervals.

• Sync in progress: Synchronizing licenses between Citrix ADM and Citrix ADC might take as long as 15 minutes. Citrix ADM might have rebooted or ADM HAS failover is triggered.

• Partially allocated: Citrix ADC cannot accept the capacity allocated because it might be running at its maximum allocation. For example, Citrix ADC is running with 10 Gbps license pool capacity. When ADC reboots, the 10 Gbps is checked-in back to ADM license server. When Citrix ADC comes back online, it tries to check out the earlier allocated 10 Gbps automatically. Meanwhile, other ADCs may have checked out that bandwidth. Partially Allocated appears if the license pool does not have enough capacity to allocate complete 10 Gbps or even partial capacity to this ADC.

• Not managed: Citrix ADC is not added to ADM for manageability. This does not have impact on Citrix ADC licensing, but it can impact license monitoring from ADM.

• Connection lost: Citrix ADC is not reachable from ADM for manageability. For example, there are network connectivity issues, NITRO is not working, or Citrix ADC password mismatches. If NITRO is not working or Citrix ADC password mismatches, this does not have an impact on Citrix ADC licensing. However, it can impact license monitoring from ADM.

• Allocated: not applied on ADC: Citrix ADC might require reboot if license is checked-out or checked-in from ADC, but Citrix ADC hasn’t rebooted yet.

• Not allocated: License is not allocated in the ADC instance.

Citrix ADM license pool: Common issues

• ADC displays license server as Unreachable:

  • Connection to license server (ADM or ADM Service Agent) has severed for more than 15 minutes.

  • ADC is in Grace Mode.

• ADC displays license server status as Reachable but user attempt to change allocation has no effect:

  • Connection to license server has recently gone down but the ADC still hasn’t missed the second heartbeat. Therefore, it is not in Grace (yet).

  • Pressing “Change Allocation” will return 0 0, this may make it appear the configured capacity has been lost.

• ADC displays Capacity/Instance counts but the license server is Reachable/Unreachable:

  • Connection to license server got restored but the ADC is still to miss the second heartbeat/or send the Re-Connect Probe.

  • Pressing Change Allocation returns some numbers but does not account for configured capacity.

• ADC says Cannot connect to license server when configuring Pooled licensing with ADM Service:

  • Check firewall rules: 27000 and 7279.

  • Agent is not registered or ADM Service does not have license files uploaded (or it has the wrong files).

Use Case: Citrix ADM license pool usage reporting

Citrix ADM license pool usage reporting will Identify monthly peaks for customers to plan for increases in license usage and plan for the next license pool purchase.

• Polling:

  • License data is polled from the ADC every 15 minutes.

• Maintain only peaks per hour:

  • The maximum license usage requirements in an hour will be stored per device.

• Reporting:

  • GUI reports to be generated which show usage per device for a specified time range.

• Export:

  • The ability to export the metering data as CSV or XLS for a specified time range.

• Purging:

  • Purge jobs will run on the 1st of every month at 12:10 AM.

  • The purge period is configurable (the default period is 2 months).

To install license files on the Citrix Application Delivery Management (ADM)

1. In a web browser, type the IP address of the Citrix ADM. For example, http://192.168.100.1.

2. In the User Name and Password field, enter the administrator credentials.

3. On the Configuration tab, navigate to Networks > Licenses > Settings, and click Add New License.

4. In the License Files section, select one of the following options:

   • Upload license files from a local computer - If a license file is already present on the user’s local computer, the user can upload it to the Citrix ADM. To add license files, the user can click Browse to select the license file (.lic). Then click Done.

     Note:

     If the uploaded license files do not add the licenses in the Citrix ADC Pooled capacity, you can select the license files and click Apply Licenses to add the licenses to the pool.

   • Use License Access Code - Citrix emails the License Access Code (LAC) for the licenses that customers purchase. To add license files, enter the LAC in the text box and then click Get Licenses.

     Note:

     At any time, users can add more licenses to the Citrix ADM from the License Settings.

To allocate Citrix ADC pooled capacity licenses from the Citrix ADM

Prerequisite: Before users can manage their instance pool licenses through the Citrix ADM, they must register the Citrix ADC instance with the Citrix ADM. In the Citrix ADC GUI, navigate to System > Licenses > Manage Licenses and select the Register with Citrix ADM for manageability check box when adding the Citrix ADM IP.

Note:

If users have not registered the Citrix ADC instance with Citrix ADM, they can check out licenses from Citrix ADM. However, but they cannot allocate from Citrix ADM to the Citrix ADC pooled capacity enabled instance.

In the Username and Password fields, enter the Citrix ADM credentials.

This option doesn’t work if the Citrix ADC (SDX/MPX/VPX) password is not default.

After the instance is registered with the license server, allocate the licenses as follows

1. In a web browser, type the IP address of the Citrix ADM. For example, http://192.168.100.1.

2. In the User Name and Password field, enter the administrator credentials.

3. On the Configuration tab, navigate to Networks > Licenses > Pooled Capacity.

4. Click on the license pool to be managed.

5. Select a Citrix ADC instance from the list of available instances by clicking the > button.

6. If users want to change or release a license allocation, click Change allocation or Release allocation.

7. If users click Change allocation, a pop-up window with the available licenses in the license server appears.

8. Users can choose the bandwidth or instance allocation for the Citrix ADC instance by setting the Allocate drop-down options. After making desired selections, click Allocate.

9. Users can also change the allocated license edition from the drop-down options in the Change License Allocation window.

Citrix ADM with license pool High Availability (HA)

Previously, License Pool licenses were node locked and associated with the host-ID of the ADM primary node. Whenever failover occurred to the secondary node, Citrix ADC would go into a 30-day grace period to avoid any disruptions due to an ADM unreachable event. This allowed Citrix ADC to run for 30 days even if Citrix ADM was not reachable. However, new Citrix ADC instances would not be able to check out licenses from ADM License server if it was not reachable, which meant that there was no new license checkouts during the 30-day grace period. Customers had to generate a replica of the license file from the Citrix Licensing System to make this license work if the primary node did not come back and 30 days had gone past, which meant that they were generating new license files.

Solution

With the License Pool HA solution, customers do not have to generate new license files with ADM failover to the secondary node if the primary node does not come back. New license checkout continues to work post failover. License Pool licenses and ADM licenses are now associated with a Virtual host-ID that is shared across Citrix ADM primary and secondary nodes.

Virtual host-ID

Both the Citrix ADM Primary and Secondary nodes share the same virtual host-ID. The real host-ID of the Primary node or first Citrix ADM server in the HA deployment is used as the virtual host-ID. The virtual host-ID is auto-generated in the ADM deployment, and it is stored in the ADM database in encrypted format and cannot be changed by the customer. The virtual host-ID has preference over the real host-ID. License files are synced from the ADM Primary node to the Secondary node. Citrix ADC checks out licenses using the ADM floating IP address. On failover from the Primary node to the Secondary node, the license files and the virtual host-ID are synced from the Primary to the Secondary node along with the floating IP address.

Break HA behavior

If customers initiate the ADM break HA action, both ADM nodes retain the virtual host-ID and then initiate break HA workflow. Both Node 1 and Node 2 can continue to check out licenses. Existing Citrix ADC enters into a 30-day grace period since the floating IP address is removed from the ADM.

Split brain

Citrix ADM monitors availability of ADM HA nodes by sending heart beats at regular intervals. If heartbeats do not reach the other node due to network issues, both ADM nodes promote themselves as ADM primary. The license server is running on both nodes in this scenario. Citrix ADC can check out licenses from both nodes using ADM server node IP since both share the same virtual host-ID. Node 1 and Node 2 are promoted as ADM primary. The license server is running on both servers with same virtual host-ID. License capacity is doubled. Citrix ADM split brain related events and ADM HA grace period related events are generated.

Recovery from split brain

Citrix ADM can recover from split brain situation after the customer administrator finds and fixes network issues. Work flow for recovering from ADM split brain is as follows. Once the network has been restored, Citrix ADM automatically detects ADM node 1 as ADM primary. Citrix ADM initiates the join HA workflow from ADM Node 2. Citrix ADM Node 1 real host-ID is selected as virtual host-ID. Citrix ADM is restored to normal HA scenario, and the license files and the Virtual host-ID are synced to ADM Node 2.

Configuring pooled capacity on MPX-Z

MPX-Z is the Citrix ADC pooled capacity enabled Citrix ADC MPX appliance. MPX-Z supports bandwidth pooling for Premium, Advanced, or Standard edition licenses. MPX-Z requires its platform licenses before it can connect to the license server. Users can install the MPX-Z platform license by either uploading the license file from a local computer or by using the instance’s hardware serial number, or the License Access Code from the System > Licenses section of the Citrix ADC Instance’s GUI. If users remove the MPX-Z platform license, the pooled capacity feature is disabled and all of the checked-out licenses are checked in to the license server.

Users can dynamically modify the Bandwidth of the MPX-Z ADC without a restart. A restart is required only if users want to change the license edition.

Note:

When users restart Citrix ADC, it automatically checks out the pooled licenses required for its configured capacity.

Configuring pooled capacity on a VPX instance

A pooled capacity enabled Citrix ADC VPX instance can check out licenses from a bandwidth pool (Premium/Advanced/Standard editions). Users can use the Citrix ADC GUI to check out licenses from the license server.

Users can dynamically modify the Bandwidth of a VPX instance without a restart. A restart is required only if users want to change the license edition.

Note:

When users restart the instance, the instance automatically checks out the pooled licenses required for its configured capacity.

Allocating pool licenses to the MPX-Z or VPX instance

To allocate licenses:

1. In a web browser, type the IP address of the Citrix ADC instance. For example, http://192.168.100.1.

2. In the User Name and Password fields, enter the administrator credentials.

3. On the Configuration tab, navigate to System > Licenses > Manage Licenses, click Add New License, and select Use Pooled Licensing.

4. Enter the details of the license server in the Server Name/IP Address field.

5. If users want to manage their instance’s pool licenses through the Citrix ADM, select the Register with Citrix ADM for manageability check box and enter the Citrix ADM credentials.

6. Select the license edition and the required bandwidth, and then click Get Licenses.

7. Users can change or release the license allocation by selecting Change allocation or Release allocation.

8. If users click Change allocation, a pop-up window shows the licenses available on the license server.

   Note:

   A restart is not required if users change the bandwidth allocation, but a warm restart is required if users change the license edition.

9. Users can allocate bandwidth or instances to the Citrix ADC instance from the Allocate drop-down list. Then click Get Licenses.

10. Users can choose the license edition and the bandwidth required from the drop-down lists in the pop-up window.

    Note:

    Bandwidth allocation should be a multiple of the minimum bandwidth unit.

Configuring pooled capacity on SDX-Z

An SDX-Z instance is a pooled capacity enabled instance of Citrix ADC SDX. SDX-Z supports bandwidth pooling for Premium, Advanced, and Standard editions, and instance pooling. After users apply the SDX-Z platform license, the Management Service provides options for checking licenses out from and back into the licensing server, and for allocating bandwidth capacity to the Citrix ADC instances running on the SDX-Z platform.

Note:

Citrix ADC VPX instances running on SDX-Z cannot directly check licenses out from or in to the license server. This can be done by the Management Service in SDX.

Users can install the SDX-Z platform license by either uploading the license file from a local computer or by using the instance’s hardware serial number, or the License Access Code.

If users remove the SDX-Z platform license, the pooled capacity feature is disabled and all the licenses are checked back into the licensing server.

Note:

If users reboot the instance, the instance checks out the pooled licenses required for its configured capacity.

Pooled capacity on SDX

Instance pool

An SDX appliance can provision the same number of instances that are available in the instance pool of the SDX appliance.

Bandwidth pool

During Citrix ADC instance provisioning, bandwidth is allocated to the instance. Users can select the edition and required bandwidth to provision a Virtual Citrix ADC instance. The Management Service allows the provisioning to continue only if the instance has sufficient bandwidth for the requested edition. Users are notified if the bandwidth is insufficient.

Note:

Bandwidth modification does not require an instance to restart.

Allocating pool licenses to the SDX-Z instance

To allocate licenses:

1. In a web browser, type the IP address of the Citrix ADC SDX-Z instance. For example, http://192.168.100.1.

2. In the User Name and Password fields, enter the administrator credentials.

3. On the Configuration tab, navigate to System > Licenses, and go to Pooled Capacity.

4. Enter the details of the license server in the Server Name/IP Address field.

5. If users want to manage their instance’s pool licenses through the Citrix ADM, select the Register with Citrix ADM for manageability check box and enter the Citrix ADM credentials.

6. Users can change or release the license allocation by selecting Change allocation or Release allocation.

   Note:

   The checked out licenses are stored in a separate pool by the Management Service.

7. To change the license allocation for a specific VPX instance in the SDX-Z instance, select the instance from the Instances section, and click Change allocation. A new window displays the available licenses.

8. Users can change the bandwidth edition of the instance from the Feature License drop-down list, and the required bandwidth in the Throughput (mbps) field. Then click Done.

Note:

Bandwidth allocation should be an integer multiple of the minimum bandwidth unit of the corresponding form factor.

Configuring pooled capacity on a CPX instance

While provisioning the Citrix ADC CPX instance, users can configure the Citrix ADC CPX instance to use Citrix ADC Pooled Capacity. In the docker, users must provide the Citrix ADC Licensing Server (Citrix ADM) details. The Citrix ADC CPX instance checks out licenses from the instance pool.

Note:

By default, the Citrix ADC CPX instance checks out an instance license from the instance pool and the throughput is automatically set to 1,000 Mbps. Users cannot modify the 1,000 Mbps bandwidth allocated to the instance.

Users can download Citrix ADC CPX from the Docker App Store. On the Docker host, to download Citrix ADC CPX, run the following command:

docker pull store/citrix/netscalercpx:[version number]

To configure pooled capacity while provisioning the Citrix ADC CPX instance:

While provisioning a Citrix ADC CPX instance, define the Citrix ADC Licensing Server (Citrix ADM) as an environmental variable in the docker host, and then run the command as shown below:

docker run -dt -P -e LS_IP=<LS_IP_ADDRESS> -e LS_PORT=<LS_PORT> --name <container_name> --ulimit core=-1 -e EULA=yes -v <host_dir>:/cpx --cap-add=NET_ADMIN >REPOSITORY<:>TAG<

Where:

• <LS_IPADDRESS> is the IP address of the Citrix ADC Licensing Server (Citrix ADM).

• <LS_PORT> is the port of the Citrix ADC Licensing Server. By default, the port is 27000.

Best practices, corner cases, and FAQs

Upgrade SDX license - perpetual to pooled

When the license on an SDX is upgraded from perpetual to pooled licensing, the SDX does not require a reboot. Neither the SDX nor the VPX requires a reboot to move to pooled licensing. The SVM transitions one or more VPXs to pooled licenses automatically.

Users should ensure the following for a smooth transition:

• Ensure that the SDX has the proper zero capacity license.

• Ensure that Citrix ADM server has sufficient capacity for the license editions being used in the VPX instances in the SDX.

• Ensure that sufficient bandwidth capacity is checked out from the ADM in the SVM for all VPX instances.

  • For example: If the SDX has 10 VPX instances and together they consume 40 Gbps Premium and 20 Gbps Advanced, ensure this is checked out first via SVM so that VPX instances can get those licenses.

Citrix ADC instance operation during a 30-day grace period

If a Citrix ADC instance is disconnected from the Citrix ADM after receiving a license from the pool, it is allowed to continue to operate in a 30-day grace period while it attempts to reestablish a connection with the license server. Even if the Citrix ADC reboots, the license remains in the 30-day grace period and the instance continues to operate.

Client-side initiated check-in (randomized): Scenarios where Citrix ADM initiates connection

For client-side initiated check-in (randomized), are there scenarios where Citrix ADM would initiate this connection?

Citrix ADM (license server) and Citrix ADC (license client) exchange heart-beat packets to monitor the health of the connection established between the client and the server. This time period is randomized to avoid all Citrix ADC clients sending requests to the Citrix ADM License server at the same time.

If there is an issue with the license connection between the client and the server, the following actions are taken:

• If Citrix ADM does not receive a heart-beat packet from a Citrix ADC client, Citrix ADM server claims back licenses allocated to that specific Citrix ADC.

• If Citrix ADC does not receive the heart-beat packet, the Citrix ADC moves to a 30-day license grace period.

• If Citrix ADC receives a license server connection established signal with the Citrix ADM license server, Citrix ADC checks out the license again from Citrix ADM.

Pooled bandwidth allocation during Citrix ADC reboot

If bandwidth was allocated during a Citrix ADC reboot, would the pooled bandwidth license be partially distributed (up to the available bandwidth in the pool) or would no license be distributed?

Citrix ADC initially attempts to check out of user-configured pooled capacity. If this attempt fails, Citrix ADC attempts the check out of available pooled capacity in Citrix ADM.

Note:

This feature is only available for MPX and VPX. SDX attempts partial license checkout if Citrix ADM does not have enough capacity.

License mismatch alert (Citrix ADC receives partial or no license)

In the event of a mismatch (for example, Citrix ADC did not receive a license or received only a partial license), is Citrix ADM able to flag this situation for reconciliation?

In the event of a license mismatch where Citrix ADC receives no license or a partial license, Citrix ADM must flag this situation for reconciliation. License mismatch can occur in the following scenarios:

• If Citrix ADC reboots, Citrix ADC checks out license again after the reboot. This clears the pooled capacity mismatch event.

• If Citrix ADM reboots, Citrix ADC and Citrix ADM synchronize license information within a heart beat interval and this event is cleared.

• If Citrix ADC checkout fails after reboot/license server reconnection, there is no automatic recovery. The user needs to manually check out a license from the pool again.

High Availability (HA) failover of Citrix ADM

During HA failover of Citrix ADM, how does the licensing file sync and what failures could occur (such as SSL cert on Citrix ADC sometimes not being copied to a secondary node when updated on the primary node)?

Citrix ADM High Availability (HA) support for pooled licensing is available from software release 12.1-50.x onward. Citrix ADM periodically synchronizes files uploaded in the primary Citrix ADM to the secondary Citrix ADM. Therefore, a file sync is done before the HA failover event ever occurs. As a result, file sync failures are not likely to occur. For example, the SSL certificate on Citrix ADC that has been updated on the primary Citrix ADM and it hasn’t been copied to the secondary Citrix ADM.

Secondary Citrix ADM database health status checks

Are there health status checks in place for the secondary database issues? Does the secondary Citrix ADM validate that the information being shared is healthy to avoid replicating unhealthy information?

License information is maintained in license server memory (in Citrix ADM). This information is not synchronized to Citrix ADM secondary. All license check-outs/check-ins are performed against the in-memory information of the license server. Citrix ADM database is used only to store reports collected from the license server (in Citrix ADM) and Citrix ADC instance.

Citrix ADM synchronizes only the license files from Citrix ADM primary to secondary (beginning with software release 12.1-50.x onward).

During Citrix ADM HA failover, Citrix ADC checks out licenses from ADM after the heart-beat interval and license server memory is updated after the heart-beat interval.

Reverse grace period for license unavailability

Is there a reverse grace period for license unavailability that would allow the instance to stay licensed for a grace period as opposed to immediately shutting down? For example, Citrix ADC attempts to check in, and Citrix ADM states there is no valid license available.

A solution for this particular issue is being investigated. We will inform users when we have a proposed solution to this issue.

Configurable system ID for licensing on Citrix ADM

Is there support for a configurable system ID (as opposed to a MAC address based system) to use for licensing on Citrix ADM?

Support for a configurable system ID for licensing is not currently being planned.

File consistency checks or mechanisms

For files, including licenses that are replicated from the primary to secondary Citrix ADM, are there any consistency/corruption checks or mechanisms in place to ensure that the primary DB corruption does not replicate the problem to the secondary?

Citrix ADM maintains the license files in the file system and synced using RSYNC utility. As a result, database issues do not impact the license files.

Citrix ADM agent usage for license check-in/check-out

Note:

Currently only 1 Agent per given Tenant is supported for pooled capacity in public clouds.

BLX Pooled Licensing

BLX uses the same license pool as Citrix ADC VPX.

Overview of virtual CPU Licensing

Data centers are moving to newer technologies that simplify network functions while offering lower costs and greater scalability. Newer data center architecture must at least include the following features:

• Software-defined networking (SDN).
• Network functions virtualization (NFV).
• Network virtualization (NV).
• Micro-services.

Such a movement also demands that the software requirements be dynamic, flexible, and agile to meet the ever-changing business needs. Licenses are also expected to be managed by a central management tool with full visibility into the usage.

Previously, Citrix SW ADC licenses were allocated based on the bandwidth consumption by the instances. A Citrix SW ADC was restricted to use specific bandwidth and other performance metrics based on the license edition (Standard, Advanced, or Premium) to which it was bound. To increase the available bandwidth, users had to upgrade to a license edition that provided more bandwidth. In certain scenarios, the bandwidth requirement might be less, but the requirement was more for other L7 performance such as SSL TPS, compression throughput, etc. Upgrading the Citrix SW ADC license might not be suitable in such cases. But users might still have to buy a license with large bandwidth to unlock the system resources required for CPU-intensive processing. Citrix SW ADM now supports allocating licenses based on the number of virtual CPU (vCPU).

With the vCPU-based licensing feature, the license specifies the number of vCPUs to which a particular Citrix SW ADC VPX is entitled. The Citrix SW ADC VPX can check-out licenses dynamically from the license server for only the number of vCPUs on which the SW ADC can run. The vCPU licenses support all SW ADC form factors including VPX, CPX and BLX.

Similar to pooled license capacity and CICO (Check-In, Check-Out) licensing functionalities, the Citrix SW ADM license server manages a separate set of vCPU licenses. Here also, the three editions managed for vCPU licenses are standard, advanced, and premium. These editions unlock the same sets of features as those unlocked by the editions for bandwidth licenses.

There might be a change in the number of vCPUs or when there is a change in the license edition. In such a case, users must always shut down the instance before they initiate a request for a new set of licenses. Users must restart the Citrix SW ADC after checking out the licenses.

Configure licensing server in Citrix SW ADC VPX using GUI:

1. In Citrix SW ADC VPX, navigate to System > Licenses and click Manage Licenses.
2. On the License page, click Add New License.
3. On the Licenses page, select the Use remote licensing option.
4. Select CPU licensing from the Remote Licensing Mode list.
5. Type the IP address of the license server and the port number.

6. Click Continue.

   Note: Users must always register Citrix SW ADC VPX instances with Citrix SW ADM. If not done already, enable Register with Citrix SW ADM and type Citrix ADM login credentials.

7. In the Allocate licenses window, select the type of license. The window displays the total and the available vCPUs and also the CPUs that can be allocated. Click Get Licenses.

8. Click Reboot on the next page to apply for the licenses.

Note: Users can also release the current license and check out from a different edition. For example users are already running Standard edition license on their instance. They can release that license and then check out from Advanced edition.Note: Users must ensure that the correct amount of memory (2 Gb) is assigned per vCPU. Check the memory per vCPU allocations. If they are not correct, increase the memory and reboot the Citrix SW ADC VPX instance.

Configure licensing server in Citrix SW ADC VPX using CLI:

In the Citrix SW ADC VPX console, type the following commands for the following two tasks:

1. To add the licensing server to the Citrix SW ADC VPX:
   • Type the license server IP address. For example, http://192.168.100.1
     .
2. To apply for the licenses:
   • set capacity -vcpu - edition platinum
   • When prompted, reboot the instance by typing the following command: reboot -w

Managing vCPU licenses on Citrix SW ADM

1. In Citrix SW ADM, navigate to Networks > Licenses > Virtual CPU Licenses.

2. The page displays the licenses allocated for each type of license edition.

3. Click on the number within each donut (Standard, Advanced, Premium) to view the Citrix SW ADC instances that are using this license.

vCPU licensing for Citrix SW ADC CPX

While provisioning the Citrix SW ADC CPX instance, users can configure the Citrix SW ADC CPX instance to check out licenses from the license server depending on the CPU usage in the instance.

Citrix SW ADC CPX relies on the license server, running on Citrix SW ADM, to manage the licenses. Citrix SW ADC CPX checks out the licenses from the license server when it is starting up. The licenses are checked in back to the license server when the Citrix SW ADC CPX shuts down.

Users can download Citrix SW ADC CPX from the Docker App Store. On the Docker host, to download Citrix SW ADC CPX run the following command:

• docker pull store/citrix/netscalercpx:[version number]

There are three license types available for CPX licensing:

1. Virtual CPU subscription licenses supported for CPX and VPX

2. Pooled Capacity licenses

3. CP1000 licenses that support single to multiple vCPUs for CPX only

To configure vCPU subscription licenses while provisioning the Citrix SW ADC CPX instance:

Users need to specify the number of vCPU licenses that the Citrix ADC CPX instance uses.

• This value is entered as an environment variable through Docker, Kubernetes, or Mesos/Marathon.

• The target variable is “CPX_CORES.” The CPX can support from 1 to 7 cores.

To specify 2 cores, users can perform the docker run command as follows:

• docker run -dt -P –privileged=true –ulimit core=-1 -v:/cpx -e EULA=yes -e CPX_CORES=2

While provisioning a Citrix ADC CPX instance, define the Citrix SW ADC Licensing Server as an environmental variable in the docker run command as shown below:

• docker run -dt -P –privileged=true –ulimit core=-1 -v:/cpx -e EULA=yes -e CPX_CORES=2 -e LS_IP=-e LS_PORT=cpx:11.1

Where,

• is the IP address of the Citrix ADC Licensing Server.
• is the port of the Citrix ADC Licensing Server. By default, the port is 27000.

Note: By default, the Citrix SW ADC CPX instance checks out the license from the vCPU subscription pool. The CPX instance checks out “n” number of licenses if the instance is running with “n” CPUs.

To configure Citrix SW ADC Pooled Capacity or CP1000 licenses while provisioning the Citrix SW ADC CPX instance:

If users want to check out licenses for the CPX instance using the pooled licensing (bandwidth-based) or the CPX private pool (CP1000 or private-pool-based), users must provide the environment variables accordingly.

For example,

• docker run -dt -P –privileged=true –ulimit core=-1 -v:/cpx -e EULA=yes -e CPX_CORES=2 -e LS_IP=-e LS_PORT=-e PLATFORM=CP1000 cpx:11.1

CP1000. This command triggers the checkout from CP1000 pool (CPX private pool). The Citrix SW ADC CPX instance then retrieves “n” number of instances for “n” number of cores specified for CPX_CORES. The most common use case is to specify n = 1 for a checkout of a single instance. Multicore CPX use cases checks out “n” vCPUs (where “n” is from 1 to 7).

• docker run -dt -P –privileged=true –ulimit core=-1 -v:/cpx -e EULA=yes -e CPX_CORES=2 -e LS_IP=-e LS_PORT=-e BANDWIDTH=2000 cpx:11.1

Pooled capacity. This command checks out one license from the instance pool and consumes 1000 Mbps of bandwidth from the platinum bandwidth pool yet enables CPX to run up to 2000 Mbps. In Pooled Licensing, the first 1000 Mbps is not charged.

Note: Specify the corresponding number of vCPUs for the desired target bandwidth when checking out from the bandwidth pool as detailed in the following table:

Overview of Citrix ADM Service Pooled Licensing

Citrix ADM Service pooled licensing is a feature of Citrix ADM Service. Citrix ADM Service pooled licensing enables customers to use pooled licenses with ADM Service. With ADM Service pooled licensing, users are able to manage license allocations across multiple ADCs that are distributed across multiple data centers. Citrix ADM Service pooled licensing supports multiple Agents per data center. Licensing is managed by the ADM Service as a whole and not just one Agent. Thus, users should think of an Agent as a network proxy to a cloud based license server. Citrix ADM Service pooled licensing also supports license access codes to fetch licenses from the Citrix Portal. Citrix ADM Service provides a dashboard that helps users to manage capacity allocation and view license usage.

Licenses uploaded to the ADM Service should be of the pooled license type and they should be locked to the cloud virtual Host-ID. The Agent should have inbound ports 27000 and 7279 opened. If an Agent goes down, the ADCs connected to it will go into grace mode. If an Agent goes down, ADCs not connected to it will not reflect any configuration changes for about twenty minutes. They will continue to operate normally. Changing the license type on an ADC (or the license edition) requires a warm reboot. A capacity change for a license does not require a reboot.

Agents and ADCs have to be made known to the Citrix ADM Service through a registration/addition process before using any feature of the ADM Service. The ADM Service registration/addition workflow is as follows:

Note: The first three steps refer to the ADM context.

1. Register Agents with the ADM Service.
2. Add ADC instances to these Agents (on the ADM Service).

3. Upload licenses on the ADM Service.

   Note: The next three steps refer to the ADC context.

4. Choose remote licensing in the ADC GUI.
5. Enter the IP address of the Agent to which the ADC is registered.
6. Allocate (on ADC)

Note: ADM Service can now be used to monitor Pooled Licenses across all ADCs and also for changing allocation.