Citrix Universal Print Server load balancing in XenApp and XenDesktop 7.9 编辑

How we tested

All test scenarios were conducted in a real world fashion and the final end printing was the only simulated component. The test systems consisted of XenServer VMs; Windows 2012R2 for Infrastructure (DDC, StoreFront, and Universal Print Server) and XenApp VDA systems, Windows 10 for XenDesktop VDA systems; and Windows 8.1 for ICA launching through Citrix Receiver. Each ICA launcher was connected to 10 printers with no overlap of printers between ICA launchers, and custom scripts were used so that each XenApp VDA session was provided a random printer to use. The custom scripts also controlled the printing inside each session utilizing AutoIt to perform identical print actions. Finally, we used an internally developed tool to coordinate the ICA session launches and collect perfmon data for the tests.

Universal Print Server load balancing sizing

Just like every other component in your environment, sizing is vitally important for Universal Print Server load balancing to perform optimally. Since printing a large document is subjective depending on your needs, this article focuses primarily on the rate of printing and use what we consider to be an average job.

Through internal testing it has been found that you don’t want too many or too few Universal Print Server instances setup in load balancing. And while it is true you can increase printing with additional Universal Print Server instances, when you are talking about distribution of printers that is not necessarily the case. In particular, is the case where you have too many Universal Print Server instances configured. In this case there is a definitive skewing of the users to the first available Universal Print Server instances.

The graph references a 3,500 user test scenario across 48 XenApp servers utilizing 16 Universal Print Server instances. As can be seen above, the first 8 Universal Print Server instances took the majority of the connections and there wasn’t an even distribution of connections. This scenario also assumed a low printing rate which we will be discussed later.

In order to understand why this occurs we need to look at how the load balancing mechanism chooses a Universal Print Server instance. Let us say we have 16 Universal Print Server instances (numbered 1-16) in load balancing and a single XenApp server with 10 users maximum (a little exaggerated but you will see why). Let’s also say that the load balanced Universal Print Server instances are in numerical order in the Universal Print Server for load balancing policy.

When a user logs on and creates a session (in XenApp), the Universal Print Server instance that user receives is random. It could be any Universal Print Server instance that is available and no preference is given to any server. In this example, let us say they received server 16 for their connections. After that first user is logged on and their session creation is finished, another user logs in. This user will be utilizing the first Universal Print Server instance in the list from the policy, in this case server 1. Another user logs in and is now utilizing server 2. Continuing that trend will provide the server loading seen below. When all 10 users are logged in, Universal Print Server instances 1-9 and 16 will have connections while the remaining servers do not.

We now add an additional XenApp server to the mix, with the same maximum of 10 users. This server follows the same procedure as the previous example with the exception that the first user randomly receives server 4 instead of server 16. In this case the same process for each subsequent login occurs as before except that it skips over server 4 as that has a connection made currently. In this case servers 1-10 will have connections. You can see the additional users in orange in the below graph, and observe how it balances out.

Continue to increase the server count and the skewing that occurs can plainly be observed. In practice, you should have a quantity of XenApp hosts that are an equivalent or multiple of your Universal Print Server instances. It is also a good idea to keep the user session counts as multiples of the Universal Print Server instances you plan to use for more optimally loading. Looking at the above scenario from this point of view, with 2 XenApp servers and 16 Universal Print Server instances, we see that our user load should be a minimum of 16 users per XenApp server. Another way to look at the same problem is that if we are only supporting 10 users per XenApp server, then no more than 10 Universal Print Server instances are required. This will provide for a more balanced loading and will more effectively utilize the available resources.

The above is an overly simplistic look at a setup dealing with strictly user connection balancing. More complex setups with many more users per server would be the likely configuration seen. With the increase in users the printing rate that was mentioned earlier plays a larger role in Universal Print Server sizing. We recommend using the formula below to determine the required Universal Print Server instances for your environment. This will enable you to determine the number of Universal Print Server instances you need in load balancing based off of your required print rate. In the interests of helping to simplify the sizing, this formula can be used to provide guidance for a more ideal setup to provide a required print rate. In general you will be most interested in solving for N to determine your own print server count.

V\*P/N \< J

Where:

V = Number of VDAs that use LB

P = Average number of active network printing jobs per minute per VDA

N = Number of load balanced print servers

J = Maximum number of jobs per minute on Universal Print Server

P can be observed on 7.8 and newer VDAs by looking at the existing Universal Print Client perfmon counters on the VDA, more specifically by monitoring the average of the jobs Created per minute counter for network printers during a normal workday on a VDA that has Universal Print Server policy enabled and network printers mapped into sessions.

J should be a number between 50 and 100, depending on the hardware performance of the Print Servers and the size of the documents to be printed.

The above formula is generalized and heavily dependent on the requirements of your environment. It is important to fully understand the printing requirements of your environment before implementing Universal Print Server load balancing.

100 jobs per minute (JPM) testing

One of the largest improvements to come with the new Universal Print Server load balancing is the increase in concurrent print jobs per minute. The doubled print rate threshold of 100 jobs per minute now allows for even greater density on an individual Universal Print Server instance. Spread across multiple load balanced Universal Print Server instances makes this increase multiples more impactful. Below is perfmon output of the Jobs Created per Minute counter (the one referenced in the formula) averaging out to ~100 jobs per minute for an 18+ hour test cycle.

VDI randomization

The load balancing method used in a VDI environment is an implementation of the randomization function only. As has been stated before, the load balancing process occurs solely on the VDA, randomizing the first connection and then subsequent logins are load balanced across the list of Universal Print Server instances. Since a VDI implementation has a single user per VDA, the randomization function is the only one that applies.

To ensure that the randomization works, a 500 user XenDesktop test run using 16 Universal Print Server instances was conducted. This works out to roughly 31 sessions per Universal Print Server instance (in a perfectly load balanced world) which makes for a clear determination of the effectiveness of the randomization. Below is the printer connections created as a result of this testing. It can be easily observed that the connections are being randomly assigned to a Universal Print Server instance.

5000 user testing

XenApp is where the greatest benefit from Universal Print Server load balancing will be realized due to the density of the XenApp servers themselves. In order to determine how well Universal Print Server load balancing scales in a larger user count environment, it was decided that a 5,000 user test run was a sufficiently large to verify the load balancing.

The above is the result of the 5,000 user test, utilizing 48 XenApp servers with 16 Universal Print Server instances. This user count works out to roughly 100 users per XenApp server, or roughly 6.5 users per Universal Print Server instance per XenApp server. The results do exhibit the skewing that was identified earlier, as this is not an optimally designed test. Ultimately, this is intended to be a demonstration of the load balancing feature.

Universal Print Server load balancing failover

By default, a Universal Print Server instance will not be reported as failed for 180s MINIMUM, and can take as long as 360s to be considered failed. This timeout is important to understand as this causes failover to not happen the instant of a Universal Print Server instance failure. There will be time for the Universal Print Server instance to attempt to recover before the failover occurs. If a more immediate failover is required, then there will need to be modifications made based on your environmental needs. These modifications can be made through the Citrix Policy, and also through two registry keys.

Below is a demonstration of the failover in action. 2,304 users across 48 XenApp servers across 16 Universal Print Server instances were used to illustrate the initial load balancing and subsequent failover(s). The above values were selected so that it works out to 3 users per Universal Print Server instance per XenApp server, ideally.

All Universal Print Server instances are equally loaded and all users are logged in. Universal Print Server instances UPServer06 and UPServer11 are subjected to a Forced Shutdown (due to PING being used to determine the availability state of a Universal Print Server instance) at the hypervisor level so that they would be seen as completely failed. Below, the server connections affected are highlighted in orange. Next the failed server connections in orange are redistributed to the remaining Universal Print Server instances still up.

New connections are then made to the existing servers that are still available. Below it can be seen how the previously failed connections are redistributed to the existing servers.

As has been mentioned before, existing connection do not dynamically load balance. Load balancing occurs solely at the user session logon. As such, when the failed Universal Print Server instances become available again, no rebalancing or failback will occur. This can be seen below where the Universal Print Server instances that had failed are brought back online, but do not take any existing connections.

To illustrate that these Universal Print Server instances are again available and will accept connections, it is necessary to either logon additional users or to force a failure of further Universal Print Server instances. Below, Universal Print Server instances UPServer08 and UPServer10 were subjected to a forced failure, and their respective connections highlighted in orange.

With the failure it can be seen accordingly that the connections are migrated to other servers. In this case they will be migrated back to the two servers that were previously failed, now available to take connections again. As these servers are the least loaded (no load) they will take the bulk of the connections.

Adding multiple print servers

Multiple Universal Print Server instances can be added to the load balancing policy in two ways: through the Citrix Policy GUI or through PowerShell cmdlets. The Citrix Policy GUI is self-explanatory in its use. Below is a method of utilizing PowerShell to add multiple Universal Print Server instances to the load balancing policy in a more expedited fashion.

1. Add-PSSnapin Citrix.Common.GroupPolicy
2. New-PSDrive –PSProvider CitrixGroupPolicy –Name Site –Root \ -Controller localhost
3. CD site:\Computer
4. CD to the policy you want to edit (The policy name that contains your UPSLB policy)
5. CD Settings\ICA\Printing\UniversalPrintServer\LoadBalancedPrintServers
6. Use New-Item to add new printers to the list

There are a few caveats with the use of this method though. Firstly, be sure you are entering the information correctly. You may want to add a few printers manually to the policy and view how they are presented through the PowerShell display to ensure you are adding them correctly. Secondly, since you are sidestepping the policy UI, there is no validation of the print servers performed.

Universal Print Server counters

As mentioned in the sizing section there are new perfmon counters that can be utilized to determine information about the current conditions of printing. Unique counters exist on both the Universal Print Server instance and on the XenApp/XenDesktop systems.

Counters relevant to the overall Universal Print Server instance will be located on the corresponding Universal Print Server instance, such as the Jobs created per minute counter previously mentioned (these counters are for that Universal Print Server instance only, and are not cumulative across multiple instances). Counters relevant to Universal Print Server load balancing exist on each XenApp/XenDesktop system (load balancing occurs at the individual VDA level) such as the Current Connections counter.

UPClient (VDA component) specific counters can be configured to capture data for a specific Universal Print Server instance, all Universal Print Server instances or as a sum total for all Universal Print Server instances on a VDA. These counters can be seen either directly through perfmon or can be scripted through PowerShell. The following counters will be available underneath the Citrix Printing Load Balancing section of perfmon. These counters are further selectable by selecting a total (_loadbalancers_total) for the VDA or by selecting an individual Universal Print Server Instance (Universal Print Server Instance Name) available on that specific VDA.

Active Printer Connections Counter: Performance\Citrix Printing Load Balancer (SELECTION)\Active Printer Connections

Created Printer Connections Counter: Performance\Citrix Printing Load Balancer (SELECTION)\Created Printer Connections

Deleted Printer Connections Counter: Performance\Citrix Printing Load Balancer (SELECTION)\Deleted Printer Connections

As these are standard perfmon counters, the PowerShell built-in Get-Counter cmdlet can be used as below to obtain information from a specific VDA.

Get-Counter -Counter \\\\VDAName\\Citrix Printing Load Balancer(SELECTION)\\COUNTER

The above command will retrieve the desired COUNTER (Active/Created/Deleted Printer Connections) information from the desired VDAName (Name, FQDN, or IP address of the VDA) for the SELECTION (Universal Print Server instance name or _loadbalancers_total). This will provide out a full listing of the counter object.

If you are concerned with only the actual value you will need to pipe this command into a further command to retrieve the cooked value (or just the connections value). To accomplish this we will add this command to the end of the previous one:| Foreach-Object {$_.CounterSamples.CookedValue[0]}

Test environment

The testing environment consisted of three separate pooled sets of hardware running XenServer 6.2 and 6.5. There existed a single XenServer 6.2 pool which contained the Citrix infrastructure components (DDC, StoreFront, ICA Launchers, metrics collection) and two 6.5 pools containing the Universal Print Server instances and test RDS VDAs. Two separate centralized storage repositories were used for the testing (one for each pool version) and all test virtual machines were located there. All software used was the most up to date at the time of the testing conducted. All tests were conducted with the same driver and driver version to ensure results are consistent. Additional drivers were tested; individual results will vary based off the driver used.

XenServer 6.2 Physical Servers (x10)

• 2 x Intel Xeon E5620 @ 2.40 GHz (4 – core HyperThreaded) – 16 CPUs
• 64 GB Memory
• NFS Storage

XenServer 6.5 Physical Servers (x25)

• 2 x Intel Xeon E5-2640 @ 2.50 GHz (6 – core HyperThreaded) – 24 CPUs
• 256 GB Memory
• NFS Storage

Universal Print Server VM

• 16 vCPU (16 socket x 1 core)
• 16 GB vRAM
• 75 GB Storage
• Windows Server 2012 R2

RDS VM

• 16 vCPU (16 socket x 1 core)
• 16 GB vRAM
• 75 GB Storage
• Windows Server 2012 R2

ICA Launcher VM

• 2 vCPU (2 socket x 1 core)
• 4 GB vRAM
• 60 GB Storage
• Windows 8.1 x64 Enterprise

Citrix Universal Print Server Policies

ICA\Printing

• Universal driver preference – XPS;EMF;PCL5c;PCL4;PS
• Universal print driver usage – Use universal printing only
• Universal print server enable – Enabled with no fallback to Windows’ native remote printing
• Wait for printers to be created – Enabled
• Universal Print Servers for load balancing – List of Print Servers

Universal Print Server and RDS/VDA VMs were kept within the same hardware pooled physical servers to ensure that testing was performed on consistent hardware configurations. DDC and StoreFront servers were not included in the above as they do not have an impact on Universal Print Server load balancing, except for policy propagation from the DDC. Minimal policies were used in the Domain, and the XenDesktop/XenApp site was a default installation with default policies with the exception of the Universal Print Server and Load Balancing policies mentioned above.