EUC Weekly Digest – October 12, 2019

Last Modified: Oct 12, 2019 @ 8:11 am

Interesting EUC items from last week:

Citrix Virtual Apps and Desktops

Citrix App Layering

  • Download Citrix App Layering 1910 – new offload compositing process (currently Hyper-V only) for faster image creation and more feature compatibility

Citrix WEM and Profile Management

  • Updated the WEM Hydration Kit to now include Office 2016 ProPlus and Professional Plus as well as some FSLogix goodies – CTP James Kindon

Citrix Workspace app

Citrix ADM

VMware

Microsoft

For more immediate updates, follow me at http://twitter.com/cstalhood.

For a list of updates at carlstalhood.com, see the Detailed Change Log.

EUC Weekly Digest – October 5, 2019

Last Modified: Oct 5, 2019 @ 5:47 am

Interesting EUC items from last week:

Citrix Profile Management

  • Citrix Profile Management 1909 automatically syncs usrclass.dat* and automatically mirrors Packages and Caches

Citrix StoreFront

Citrix ADC

Citrix ADM

Citrix Gateway

Citrix Cloud

VMware

Microsoft

For more immediate updates, follow me at http://twitter.com/cstalhood.

For a list of updates at carlstalhood.com, see the Detailed Change Log.

EUC Weekly Digest – September 28, 2019

Last Modified: Sep 28, 2019 @ 6:34 am

Interesting EUC items from last week:

Citrix Virtual Apps and Desktops

Citrix VDA

Citrix Workspace app

Citrix ADC

Citrix ADM

VMware

Other

For more immediate updates, follow me at http://twitter.com/cstalhood.

For a list of updates at carlstalhood.com, see the Detailed Change Log.

EUC Weekly Digest – September 21, 2019

Last Modified: Sep 21, 2019 @ 7:09 am

Interesting EUC items from last week:

Citrix Virtual Apps and Desktops

Citrix WEM and Profile Management

Citrix Workspace app

Citrix ADC

Citrix ADM

Citrix Gateway

Citrix Cloud

VMware

Microsoft

For more immediate updates, follow me at http://twitter.com/cstalhood.

For a list of updates at carlstalhood.com, see the Detailed Change Log.

EUC Weekly Digest – September 14, 2019

Last Modified: Sep 14, 2019 @ 6:31 am

Interesting EUC items from last week:

Citrix Virtual Apps and Desktops

Citrix WEM and Profile Management

Citrix ADC

Citrix Gateway

VMware

For more immediate updates, follow me at http://twitter.com/cstalhood.

For a list of updates at carlstalhood.com, see the Detailed Change Log.

Citrix Health Check

Last Modified: Oct 14, 2019 @ 6:05 am

Navigation

💡 = Recently Modified

Change Log

Health Check Overview

Health Checks review an environment for configurations that might cause future problems, not necessarily existing problems. Health Checks tend to focus on non-functional qualities like the following:

  • Availability
  • Security
  • Manageability
  • User Experience
  • Performance
  • Reliability

The rest of this article is an incomplete list of health check assertions for Citrix environments.

StoreFront Load Balancing

  • Citrix connectivity infrastructure design is documented: StoreFront, Gateways, ADCs, multiple datacenters, Delivery Controllers, SQL, etc.
    • Separate test Citrix environment has identical architecture as production: multiple data centers, high availability for all components, etc. – enables testing changes, including HA/DR changes, before performing those changes in production. Some upgrades are performed differently for HA/DR than for single components.
  • The FQDN that users use to access Citrix (e.g. https://citrix.company.com) resolves to a Load Balancing VIP, not a single server.
    • The FQDN automatically fails over (e.g. GSLB) to a VIP in a different data center if the primary data center is down.
  • The certificate for the SSL Load Balancing VIP is valid: trusted, not expired, matches FQDN, no errors in Chrome, etc.
    • Someone is responsible for ensuring the certificate is not expired and receives pending certificate expiration notifications.
  • The Load Balancing VIP sends SSL traffic to two or more StoreFront servers in the local data center – for redundancy.
    • The ADC-to-StoreFront server communication is SSL/TLS encrypted, not HTTP – this traffic contains user credentials.
  • The ADC monitor for the StoreFront servers is type STOREFRONT, or does a GET request to /Citrix/Store/discovery – other monitors might not detect stopped services.
  • X-Forwarded-For is configured in the Load Balancing Services (or Service Group) for Client IP header insertion.
  • Load balancing persistence is SOURCEIP with a timeout that is as long as the Receiver for Web timeout – COOKIEINSERT doesn’t work on all client devices.

StoreFront Servers

  • If the StoreFront servers are on the same hypervisor cluster, then anti-affinity is configured to keep them on separate hypervisor hosts.
  • StoreFront server VMs do no have any old snapshots – slows down performance, and consumes disk space.
  • StoreFront version is an LTSR Cumulative Update version (e.g. 3.12.4000), or the latest Current Release version (e.g. 1906) – No other versions are supported.
    • Upgrades are performed in a separate test environment that has identical architecture as production before the updates are performed in production.
  • StoreFront configuration is propagated to other servers in the StoreFront Server Group.
  • No recent unknown errors in Event Viewer at Applications and Services -> Citrix Delivery Services.
  • StoreFront Base URL is an https URL, not http. The FQDN resolves to the Load Balancing VIP, not a single server.
  • SSL certificates are installed on each StoreFront server and bound to IIS Default Web site. The SSL certificates are not expired.
  • C:\Users does not contain a bunch of user profiles. Delprof2.exe should be scheduled to delete these profiles – caused by users changing expired passwords.
  • If HTML5 Workspace app is enabled, then HTML5 Receiver is up to date – New versions are released at least monthly.
  • If Workspace app is stored on StoreFront servers, then the local Workspace apps in C:\Program Files\Citrix\Receiver StoreFront\Receiver Clients is current.
  • If Favorites are enabled, then Favorites (aka Subscriptions) are replicated to a StoreFront Server Group in a different data center.
  • If Federated Authentication Service (FAS), then multiple FAS servers configured through Group Policy.
    • FAS Servers are the same version as StoreFront.
    • If the FAS servers are on the same hypervisor cluster, then anti-affinity is configured to keep them on separate hypervisor hosts.
    • FAS Get-FasAuthorizationCertificate shows registration certificate is OK and not MaintenanceDue.
    • FAS group policy .admx template is up to date in SYSVOL.
    • FAS User Rules restricts usage to just some StoreFront servers, some VDAs, and some users – not all
    • Auto-enrollment is not enabled on the FAS certificate templates..
    • The Certificate Authority database is not excessively large.
  • Task Manager shows sufficient CPU and Memory for each StoreFront server.
  • A monitoring tool alerts administrators of any StoreFront performance metric issue, availability issue (e.g. service stopped), and Event Log errors.
  • Logon Simulator runs periodically to verify that StoreFront is functional.
  • StoreFront Disaster Recovery procedure is documented and tested.

StoreFront Configuration

  • Only one store. Or every store but one is hidden – if multiple stores are advertised, then Workspace app will prompt the user to select a store.
  • Each Delivery Controller farm is configured with two or more Delivery Controllers – for redundancy.
    • Or Delivery Controller XML can be load balanced. If load balanced, then ADC monitor is of type CITRIX-XD-DDC – so ADC can detect Local Host Cache outages.
    • Prefer separate farms per data center instead of stretched single farms (with zones) across multiple data centers.
  • Transport Type for Delivery Controllers is https, not http – this traffic includes user credentials.
  • Receiver for Web Session Timeout is not too short for user experience or too long for security.
  • Citrix Gateway configuration in StoreFront console:
    • The STAs in StoreFront match the STAs configured on the Citrix Gateway Virtual Server on the ADC appliances.
    • Session Reliability is enabled.
    • Callback URL is only needed for SmartAccess and Citrix FAS. Callback URL should be removed if it’s not needed.
    • Internal Beacon is only reachable internally.
    • External Beacon does not include citrix.com – ping.citrix.com is OK
  • HDX Optimal Routing can send ICA traffic through the Citrix Gateway that is closest to the VDA (i.e. farm).

Delivery Controllers

  • In CVAD 1906+, Citrix Scout Health Check does not show any errors or warnings. 💡
  • If the Delivery Controller servers are on the same hypervisor cluster, then ensure anti-affinity is configured to keep them on separate hypervisor hosts.
  • Delivery Controller VMs do not have any old snapshots.
  • Delivery Controller version is an LTSR Cumulative Update version (e.g. 7.15.4000), or the two latest Current Release versions (e.g. 1906). No other versions are supported – Citrix Product Matrix shows support dates.
    • Delivery Controller Upgrades are performed in a separate test environment before performed in production.
    • Citrix upgrades or updates are performed around twice per year.
  • Run Get-BrokerDBConnection to see the SQL connection string. No SQL Express. For AlwaysOn Availability Group (AAG):
    • SQL String points to AAG Listener, not single node.
    • All AAG SQL nodes in one data center. For multiple data centers, prefer separate farms in each data center with local SQL.
    • SQL String contains MultiSubnetFailover.
    • Each SQL server has SQL Logins for all Delivery Controllers – SQL Logins usually don’t replicate between SQL nodes.
    • Prefer Synchronous Commit with Automatic Failover over Asynchronous replication.
    • AAG Dashboard in SQL Studio does not show any issues.
  • SQL databases for Citrix are not excessively large. Database Backup tool is truncating the database logs.
  • SQL Servers have sufficient CPU/Memory to handle the Citrix SQL traffic. Monitoring tool alerts SQL DBAs of any performance or availability issues.
  • SQL Server version is supported by Citrix. https://support.citrix.com/article/CTX114501
  • Local Host Cache is enabled on the Delivery Controllers. Run Get-BrokerSite to confirm.
    • Delivery Controller virtual CPU allocation is 1 CPU socket with multiple cores – SQL Express Local DB for Local Host Cache only runs on a single socket (up to four cores).
    • How are non-persistent virtual desktops handled during SQL outage?
  • SQL Disaster Recovery plan is documented and tested.
  • SSL Certificates are installed on Delivery Controllers to encrypt XML traffic from StoreFront.
    • SSL certificates are bound to IIS Default Web Site, or netsh http sslcert to perform binding. IIS Binding does not include hostname.
    • SSL certificate not expired.
  • Trust XML Requests is enabled for pass-through authentication, SmartAccess, FAS, etc. Run Get-BrokerSite to confirm.
  • Task Manager shows sufficient CPU and Memory for each Delivery Controller server.
  • A monitoring tool alerts administrators of any Delivery Controller performance metric issue, availability issue (e.g. service stopped), and Event Log errors.

Citrix Studio

  • Citrix Studio consoles installed on administrator machines are the same version as the Delivery Controllers.
  • Customer Experience Improvement Program is disabled in Citrix Studio > Configuration node > Product Support tab.
  • Licensing Model/Edition matches what you actually own.
  • Citrix Studio Administrators are periodically audited to ensure only authorized users are granted Studio access.
    • Administrators are added as Active Directory Groups, not individual users.
  • Applications are published to Active Directory Groups, not individual users.
  • If App Groups, applications are published to only App Groups. Applications are not published to both App Groups and Delivery Groups.
  • Hypervisor connection uses a service account, not an admin account.
    • Hypervisor permissions for the service account are the minimum permissions required (custom role), not full hypervisor administrator.
  • Each Hosting Resource only has one datastore selected, not multiple datastores – Citrix MCS does not have a datastore “Rebalance” option. More datastores means more copies of master image snapshots, which means longer time to push out an updated Master image.
  • MCS Memory Caching Option is not enabled unless VDA 1903 or newer – older VDA, including 7.15 VDA, has poor performing MCSIO driver.
  • StoreFront URLs are not assigned to Delivery Groups using Studio – instead use Workspace app group policy to assign StoreFront URL.

Citrix License Server

  • Citrix License Server is version 11.16.3.0 build 28000 or newer to resolve a security vulnerability. 💡
  • The licenses installed on Citrix License Server match the purchased licenses at http://mycitrix.com – some Citrix License Servers have too many licenses installed.
  • If multiple Citrix License Servers, installed license count across all License Servers does not exceed the purchased licenses shown at http://mycitrix.com
  • Administrators are not frequently clearing named user license assignments to simulate concurrent licensing – license assignments should only be cleared when the user permanently no longer uses Citrix.
  • Subscription Advantage dates are not expired – if expired, download new license files and install them.
  • Usage and Statistics tab is configured as intended in the Citrix Licensing Manager gear icon.
  • Citrix License Server Disaster Recovery procedure is documented and tested.

Remote Desktop Services (RDS) Licensing

  • If RDSH VDAs, two or more activated RDS Licensing servers.
  • RDS Licensing Server operating system version matches (or newer) the RDSH VDA operating system version – e.g. Windows 2019 RDS Licensing for Windows 2019 RDSH servers. Windows 2019 RDS Licensing also works with Windows 2016 RDSH servers.
  • In RD Licensing Manager, right-click server -> Review Configuration shows green checkmarks.
  • The combined licenses installed on all RDS license servers do not exceed the purchased licenses.
  • On RDSH VDAs, HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services\LicenseServers shows two servers.
    • LicensingMode = 4, which is Per User mode, which is not enforced.

Citrix Director

  • Director version matches the Delivery Controller version.
  • If multiple Director servers:
    • Hypervisor Anti-affinity is configured.
    • Director Saved Filters are relocated to a UNC path instead of local C: drive.
  • Director server VMs do not have old snapshots – slows down servers, and increases disk space.
  • SSL certificate is installed on Director servers.
    • Admins and Support teams always use https to access Director. IIS or load balancer redirects from http to https.
  • Director website is SSL load balanced.
    • SSL protocol, not http, between load balancer and Director servers – this traffic contains user credentials.
  • Director logon page auto-populates the domain name – for user convenience. Might have to reconfigure the domain name after every Director upgrade.
  • Citrix Policy Settings for Director:
    • Enable Process monitoring is enabled.
    • Enable monitoring of application failures is enabled.
  • If Citrix Virtual Apps and Desktops (CVAD) is Premium Edition:
    • Director Alerts are configured to email CVAD administrators.
    • Citrix ADM HDX Insight is integrated with Director. HTTPS protocol, not HTTP.
    • Probes are configured – Probe Agent version matches the Director version.
  • Help Desk knows how to use Citrix Director to support users.
  • Average logon durations are not excessive.
  • Repetitive issues (e.g. profile resets) are analyzed for root cause analysis and future prevention.

VDAs

  • Catalog design is documented – storage design, network design, multiple datacenters design, recovery design, etc.
  • VDA version matches the Delivery Controller version.
  • VDA Subnets are added to Active Directory Sites & Services.
  • DHCP is highly available. VDA IP Subnet router forwards DHCP requests to more than one DHCP server. DHCP scope is replicated to more than one DHCP server.
    • DHCP Scope has sufficient address availability for VDAs.
  • If KMS, slmgr.vbs /dlv shows a unique KMS CMID for each VDA machine – another option is Active Directory-based activation.
  • If persistent (dedicated) Catalogs:
    • The VDA version matches the Delivery Controller version – VDA updates should be automated (e.g. SCCM).
    • Dedicated Catalogs are created as Full Clones – Fast Clones cannot be moved to different storage or different hypervisor cluster.
    • Persistent desktops are backed up, replicated, etc. Recovery process is documented and tested.
    • Persistent desktop provisioning process is automated, preferably from a self-service portal.
  • No Personal vDisk – prefer SCCM-managed persistent desktops. Or App Layering User Layers if SCCM is not mature.
  • No double-hop – prefer master images with every application installed locally instead of double-hop to published application. Single hop have faster logins than double hop. Single hop is less complex than double hop since double hop requires Workspace app and icon management on the first-hop VDA machine. FSLogix can hide local applications.
  • Catalogs are upgraded to latest Catalog version available.
  • VDA registrations are somewhat evenly distributed across the Delivery Controllers.
  • ListOfDDCs registry value on VDAs has two or more Delivery Controllers.
  • Daily Health Check report shows registration status and maintenance mode status of every VDA machine.
  • RDSH Load Index Policy has not been modified from the default. CPU Metric is too volatile, and can cause a Denial of Service and uneven distribution of sessions. Current Load Index values should be almost the same on every RDSH VDA and not be anywhere near 10000.
  • In-guest monitoring agent shows VDA memory usage. Allocated VM Memory matches or exceeds memory Committed Bytes – Hypervisor monitoring can’t show actual VM memory usage.
  • RDSH VDAs are periodically restarted – net statistics workstation or net server statistics shows uptime.
    • In CVAD 1909+, MaxDelayMins is configured in Get-BrokerRebootScheduleV2. 💡
  • If Cloud-hosting of VDAs, PowerScale controls VDA power management.

VDAs – Hypervisor Hardware Clusters

  • Desktop VDAs are in their own hypervisor cluster that does not contain any Server virtual machines – avoids Windows Server licensing.
    • Hypervisor clusters with Windows Servers have proper Windows Server licensing.
  • Hypervisor admins don’t perform any hypervisor updates without first reviewing Citrix’s Supported Hypervisors article.
  • VDA vCenter is separate from non-VDA vCenter – allows non-VDA vCenter to be upgraded without affecting Citrix.
  • Hypervisor performance is monitored and alerted: CPU contention (aka CPU Ready Percentage), disk latency, CPU Usage, etc.
  • Capacity planning tool warns admins when more hypervisor hardware is needed.
  • If VMFS6 datastores, vSphere 6.7 Update 3 is installed – see release notes
  • NTP is configured and running on hypervisor hosts.
  • Hypervisor hosts have High performance BIOS settings.
  • NVIDIA vGPU software is current on hypervisor host and virtual machines.
    • The newest hypervisors can vMotion GPU-configured virtual machines.
    • NVIDIA in-guest vGPU Driver is installed before the VDA is installed – otherwise HDX 3D Pro will not work.

VDAs – Virtual Machine Hardware (vSphere)

  • Network Interface type is VMXNET3, not E1000.
  • devices.hotplug=false is configured in Virtual Machine Configuration Settings.
  • If disk space is a concern, virtual machine memory is reserved to reduce .vswp file size.
  • If Citrix App Layering:
    • Paravirtual controller is not added.
    • Boot firmware is BIOS, not EFI.
  • Windows 10 version is supported by Citrix VDA version, and supported by App Layering version.
  • VMware Tools version is current.

VDAs – Master Image Build

  • Master Image build process is documented.
  • Master Image virtual machine was built from scratch – not converted from a physical machine.
  • Security scan of the VDA Master Images shows compliance with enterprise security requirements.
  • Master Image updates:
    • Master Image maintenance is automated – e.g. SCCM can push updates to Master Images. A script can push Master Images to Catalogs.
    • Software Deployment team notifies the Master Image maintainers when applications or Windows require an update.
    • Master Image is sealed before shutdown – e.g. antivirus is generalized, SCCM Client is generalized – Base Image Script Framework (BIS-F) can automate this
    • Master Image updates are tested before deployed to production. QA testing. Canary testing.
    • Master Image snapshots are deleted after a period of time.
  • Antivirus is installed. Antivirus is optimized for non-persistent machines (aka VDI).
  • Other IT agents (e.g. software auditing, SCCM Agent) are optimized for non-persistent machines.
  • Local Groups:
    • Administrators group does not contain any non-administrators.
    • Direct Access Users group only contains authorized RDP users.
  • Citrix Optimizer or similar has removed Windows 10 Store Apps.
  • Windows Default profile was not modified – instead use group policy to control Windows appearance.
  • Windows Updates are current (i.e. last install date is within the last 60 days).
  • C: drive permissions are changed so Users can’t create folders on root of C: drive.
  • Power management is set to High Performance with no sleep timers.
  • If Citrix Provisioning:
    • Pagefile is shrunk so it fits on PVS cache disk – there’s no need to move the pagefile since PVS will move it for you. Just make sure it’s small.
    • Event Logs are moved to PVS cache disk.
  • Customer Experience Improvement Program is disabled in VDA registry.
  • Office 365 Shared Computer Activation is enabled.
    • FSLogix is implemented for Outlook search roaming.

Citrix App Layering

  • Prefer automated (e.g. SCCM) Master Image updates over manual App Layering layer updates – if SCCM is mature, then there’s no need for App Layering.
  • Prefer SCCM-managed dedicated desktops over User Layers – SCCM is a known technology. User Layers are proprietary to Citrix and might not support every application.
  • Enterprise Layer Manager (ELM) version is current – ELM updates are required to support newer Citrix Virtual Apps and Desktops (CVAD) and newer Windows 10. There’s no LTSR version of ELM.
  • Citrix Provisioning Agent version matches the ELM version.
  • Directory Junction Bind account is a service account, not a regular user whose password expires.
    • LDAP is Secure (Use SSL).
  • Administrator role membership is periodically audited to ensure only authorized users are granted access.
  • ELM is backed up. Or layers are periodically exported from ELM.
  • Group Policy controls membership of local groups in VDA machines – e.g. add Domain Admins to local Administrators group.
  • Antivirus is configured properly for Layering.
  • Hypervisor Connector uses a service account with limited permissions.
  • Connector cache is enabled to speed up layering operations.
  • File servers hosting Elastic Layers and User Layers are monitored for performance issues and capacity planning.
  • User Layers are backed up, replicated, etc.

Citrix Provisioning

Provisioning Servers:

  • Provisioning Servers version matches the Delivery Controller version.
  • Multiple Provisioning Servers for High Availability.
    • Hypervisor Anti-affinity is configured.
  • Sufficient RAM for vDisk caching in memory – around 2-3 GB of memory per active vDisk.
  • Only one NIC per Provisioning Server – simplifies the configuration.
  • Server Bootstrap has multiple Provisioning Servers listed.
  • Threads times Ports are sufficient for the number of target devices.
  • vDisk Boot Menu is disabled in the registry – enables maintenance mode Target Devices to automatically boot from maintenance mode vDisks.
  • Antivirus has exclusions for Citrix Provisioning.
  • Provisioning Server performance metrics are monitored and alerted.
    • NIC throughput is not saturated.

Provisioning Farm Properties:

  • Offline database is enabled.
  • Auditing is enabled.
  • Administrators list only contains authorized administrators, preferably from an Active Directory Group.
  • Customer Experience Improvement Program is disabled.
  • For AlwaysOn Availability Group, MultiSubnetFailover is configured in the database connection string.

vDisks:

  • If local storage, vDisk files are identical on all Provisioning Servers.
  • vDisk files are VHDX, not VHD – faster version merging.
  • vDisks are sized dynamic, not fixed – Saves disk space. Standard Mode vDisks don’t grow so no performance impact.
  • vDisk files are defragmented.
  • vDisk files are backed up.
  • vDisk updates are automated.

Target Devices:

  • Target Device Boot Method is highly available – Target Devices on same subnet Provisioning Servers. Or DHCP Option 66 with TFTP Load Balancing. Or Boot ISO/Boot Partition has multiple Provisioning Server addresses.
    • DHCP is highly available. Subnet’s router forwards DHCP requests to multiple DHCP servers. Replicated DHCP scope.
  • vDisk Write cache is configured for Target Device RAM with overflow to disk – health check script should periodically verify this.
  • WriteCache folders on Provisioning Servers are empty – no server-side caching.
  • If KMS, slmgr.vbs /dlv shows a unique KMS CMID for each Target Device machine – another option is Active Directory-based activation.
  • Target Devices are evenly distributed across multiple Provisioning servers.
  • System Reserved Partition is removed from inside vDisk.
  • VMware Tools in Target Devices (vDisks) is up to date.
  • Target Device Software version matches the Citrix Provisioning version.
  • Target Device status shows low number of retries.

Group Policies and Active Directory

  • VDAs are placed in VDA-only OUs, no users – group policies apply to VDAs without affecting physical endpoints.
    • Separate OUs per Delivery Group – different group policies apply to different Delivery Groups.
  • Master Images are located in VDA OUs – computer-level GPO settings apply to the Master Images to avoid GPO timing issues on linked clones.
  • Block Inheritance OUs and Enforced GPOs are minimized.
  • .admx templates in SYSVOL > PolicyDefinitions are current – Windows 10 templates, Office templates, Citrix templates, etc.
  • Group Policy Loopback Processing Mode is enabled.
  • Duplicate, conflicting GPO settings are minimized – e.g. Group Policy Loopback Processing Mode is sometimes enabled in several GPOs.
    • Group Policy Results shows the actual GPO settings that applied to a specific session.
  • Lockdown GPO applies to non-administrators that log into VDA machines. Lockdown GPO doesn’t apply to administrators.
  • Remote Desktop Session Host (RDSH) session timeouts are configured in a Microsoft GPO.
  • AppLocker or similar prevents users from running unauthorized executables (e.g. ransomware).
  • Initial application configuration is automated using group policy – e.g. auto configure application database connections, remove first time usage prompts.
  • Group Policy changes are tested in separate Test GPOs and separate Test VDAs before applying to production.
  • Monitoring tool shows group policy processing duration during logon.

Citrix Policies

  • Citrix Policies are configured in a Group Policy Object, not in Citrix Studio – a GPO can apply to multiple Citrix Virtual Apps and Desktops (CVAD) farms in multiple datacenters. Citrix Studio is single farm only.
    • Citrix Policies are not configured in both Citrix Studio and Group Policy – avoids confusion over which setting wins
  • Citrix Group Policy Management plug-in on GPMC machines is same version included with CVAD ISO.
  • Unfiltered policy is on the bottom of the list (lowest priority) – most specific filters on top, least specific filters on bottom.
  • Client drive mapping, client clipboard, client printing, and client USB are disabled when connecting from external (e.g. SmartAccess) – only enabled by exception.
  • Client printing is set to Use Universal Print Driver only – avoids installing print drivers on VDA machines.
  • Audio is set to Medium quality – High Quality uses more bandwidth than Medium Quality.
  • Time zone redirection is configured in both Citrix Policy and RDSH Microsoft Group Policy.
  • For HDX Insight, ICA Round-Trip Time policy is enabled.
  • Visual quality and video codec settings are not modified from the defaults.
    • Legacy Graphics Mode is disabled.
  • Adaptive Transport (EDT) is enabled – it’s default disabled in 7.15. MTU might need to be decreased.
  • Session Reliability is not disabled.
  • RDSH Session Timers are configured in Microsoft GPO, not Citrix Policy – Citrix Policy setting description shows if setting applies to Server OS or not.

Citrix Workspace Environment Management (WEM)

  • Prefer Group Policies over WEM – WEM requires extra infrastructure, extra learning, extra administration, and extra support. Some WEM user settings are per-machine (per configuration set) only. WEM can’t replace group policies since there’s currently no .admx support.
    • Citrix Profile Management and Microsoft Folder Redirection are configured using Microsoft Group Policy, not WEM – Group Policies are well known. WEM is proprietary to Citrix and requires WEM skills to troubleshoot.
  • WEM is within two versions of the latest – there’s no LTSR version of WEM.
    • WEM Consoles and WEM Agents match WEM Server version.
  • Multiple load balanced WEM Servers for High Availability.
    • If multiple WEM servers are on the same hypervisor cluster, then Hypervisor anti-affinity is configured for the multiple WEM servers.
    • WEM Agents point to WEM Server load balanced FQDN, not individual server.
    • WEM Console points to single WEM Server, not load balanced FQDN.
  • WEM Brokers are close the VDAs – WEM configuration can be exported/imported into WEM implementations in multiple data centers.
  • WEM Database is hosted on an AlwaysOn Availability Group or other Highly Available SQL solution.
    • SQL database is backed up. SQL database recovery is documented and tested.
  • In WEM 1909+, Infrastructure Service Enable performance tuning for Windows Communication Framework is enabled and set to the number of concurrent WEM Agents that will be connected to this one WEM server. Maximum value is 3000. 💡
  • Antivirus exclusions are configured for Citrix WEM.
  • WEM .admx group policy template in SYSVOL > PolicyDefinitions is updated whenever WEM Servers are updated.
  • Settings are in WEM, or Group Policy, but not both – helps troubleshooting. Reduces confusion.
  • Bypass ie4uinit Check is enabled (Advanced Settings > Service Options) – for faster logons.
  • Drive mappings and printer mappings are moved to WEM and processed asynchronously (Advanced Settings > Agent Options).
  • Check Application Existence is enabled (Advanced Settings > Agent Options) – doesn’t create shortcut unless application exists
  • CPU Optimization is enabled – Memory management trades memory for disk; which is cheaper? Process exclusions might be needed.
    • In WEM 1909 and newer, CPU Spike Protection = Auto instead of Customize. 💡
  • Fast logoff is enabled.
  • Unused action types are disabled from processing (Advanced Settings > Main Configuration) – speeds up logons.
  • Run Once enabled for Actions and scenarios that support it – speeds up logons.
  • WEM Agent Offline mode is enabled.
  • Computer startup script refreshes WEM Agent cache on each VDA reboot.
    • Script has correct Agent installation path and correct service name since they changed in 1909 and newer. 💡
  • WEM Logs are reviewed for problems – enable debug logging. Look for Active Directory timeouts.
  • WEM Server performance is monitored for metric thresholds and future capacity issues.
  • WEM Server recovery is documented and tested.

Citrix Profile Management and Folder Redirection

  • No mandatory profiles on Windows 10 – benchmarks show slower performance.
  • Profile Management is configured in Group Policy, not Citrix Policy or Citrix WEM – Group Policy is the most reliable and most well-known option.
  • Profile file share:
    • File server is close to the VDAs – users log into VDAs that are closest to the file server (aka home site).
    • File share is highly available.
    • No DFS multi-master replication. Single target only – neither Citrix nor Microsoft support merge replication.
    • Profiles are backed up and/or replicated. Recovery process is documented and tested.
    • Different profile folders for different operating system versions and/or different Delivery Groups.
    • NTFS permissions of individual user folders in the file share only grant access to the one user – no Users, no Domain Users, and no Authenticated Users.
    • Use TreeSize or similar to see profile size – adjust profile exclusions if too big.
    • Antivirus is not slowing down profile file transfer performance – time how long it takes to copy a profile folder to the local machine.
    • File servers are monitored for performance issues, including disk latency and free disk space.
  • Profile Management .admx file in SYSVOL > PolicyDefinitions matches the VDA version (or date).
  • Profile Management logs are stored on UNC share instead of local C: drive, especially if the VDAs are non-persistent.
    • Only Domain Computers have Modify permission to the Logs share – Users don’t need any permission.
  • Profile Management logs contain at least a few days of logons – if only a few minutes, then too much information is being logged and Log Settings GPO setting should be modified.
  • Profile streaming is enabled – speeds up logons.
  • Active Write Back is disabled – places extra load on file servers for not much benefit.
  • Customer Experience Improvement Program is disabled.
  • Locally cached profiles are deleted at logoff from RDSH machines that don’t reboot often.
  • No Start Menu roaming issues – might need ResetCache registry value.
  • Microsoft FSLogix is implemented for Outlook Search roaming – better than UPM’s Outlook search roaming.

Folder Redirection:

  • Folder Redirection is configured in Microsoft GPO settings, not in Citrix Profile Management settings – Microsoft GPO configuration is most reliable, most known, and can migrate existing files.
  • No AppData redirection – slows down applications.
  • “Grant the user exclusive rights” option is unchecked – allows administrators to access redirected profile folders.
  • Folder Redirection file share:
    • File share is highly available.
    • No DFS multi-master replication. Single target only – neither Citrix nor Microsoft support merge replication.
    • Redirected Folders are backed up and/or replicated. Recovery process is documented and tested.
    • NTFS permissions of individual user folders in the file share only grant access to the one user – no Users, no Domain Users, and no Authenticated Users.
    • Antivirus is not slowing down folder redirection performance.
    • File servers are monitored for performance issues, including disk latency and free disk space.

Home Directories:

  • File server is close to the VDAs – users log into VDAs that are closest to the file server (aka home site).
  • File share is highly available.
  • No DFS multi-master replication. Single target only – neither Citrix nor Microsoft support merge replication.
  • Home Directories are backed up and/or replicated. Recovery process is documented and tested.
  • NTFS permissions of individual user folders in the file share only grant access to the one user – no Users, no Domain Users, and no Authenticated Users.
  • Antivirus is not slowing down file transfer performance – time how long it takes to copy a Home Directory folder to the local machine.
  • File servers are monitored for performance issues, including disk latency and free disk space.

Endpoint Devices

  • Prefer Windows 10 endpoints over thin clients – thin clients don’t support all Citrix functionality (e.g. local printing, browser content redirection). ThinKiosk can lock down Windows 10 endpoints.
  • Newest VDAs and newest Workspace apps have better WAN performance than LTSR 7.15.
  • Browser Content Redirection offloads video (e.g. YouTube) from VDAs to endpoint – reduces CPU consumption in the data center.
  • Receiver and/or Workspace app is periodically (e.g. twice per year) updated by endpoint management team.
  • Workspace app (or Receiver) ADMX templates in SYSVOL > PolicyDefinitions are current.
  • Group Policy adds StoreFront URL to Local Intranet zone.
  • Group Policy pushes StoreFront URL to Workspace app (or Receiver) – so users don’t have to enter the URL.
  • Pass-through authentication is enabled for internal PCs – SSON Configuration Checker can verify proper configuration.
  • HKCU\Software\Citrix\Dazzle\Sites\store\type shows DS, not PNA – store added as Delivery Services (StoreFront), not PNAgent (legacy).
  • Internal Beacon at HKEY_CURRENT_USER\SOFTWARE\Citrix\Receiver\SR\Store\#\Beacons\Internal\Addr0 is internally reachable only – not reachable externally.
  • External Beacon at HKEY_CURRENT_USER\SOFTWARE\Citrix\Receiver\SR\Store\#\Beacons\External does not include citrix.com – ping.citrix.com is OK.
  • EDT protocol (aka Adaptive Transport) is enabled. Director shows HDX protocol as UDP – Remote Display Analyzer can analyze problems with the graphics/codec.
  • HDX Insight: Newest VDAs and newest Workspace app have less AppFlow CPU impact on ADC than LTSR 7.15 VDAs.

Citrix ADC

  • ADC Admins have subscribed to Citrix Security Bulletins at https://support.citrix.com/user/alerts
  • ADC firmware build is not vulnerable to TLS vulnerability.
  • ADC firmware updates are tested on separate test ADC appliances before performed in production. Test ADC appliances have test VIPs – application owners can test their VIPs on test ADC before firmware is upgraded in production.
  • ADC VPX on vSphere:
    • ADC NICs are VMXNET3, not E1000.
    • If ADC 12.0 or older, no vMotion/DRS (not supported).
    • In 12.1 and newer, DRS Cluster Anti-affinity is configured for the VPX appliances in the same HA pair.
    • CPU/Memory are reserved. Yield CPU is not enabled.
  • ADC license does not expire any time soon.
  • Physical ADC:
    • LOM port is connected and configured.
    • LOM nsroot password is changed from the default.
    • No VLAN is connected to multiple active interfaces unless those interfaces are in a port channel.
  • ADC nsroot password is not nsroot. nsroot password is managed by Privileged Identity Management tool. Admins don’t use nsroot to login.
  • Management authentication is configured for external authentication server, typically LDAP.
    • LDAP is load balanced instead of multiple LDAP Policies to individual LDAP servers – avoids premature account lockout.
    • LDAP is encrypted: LDAPS on port 636.
    • LDAP Bind account is a service account – not a regular user whose password expires.
    • LDAP Search Filter only allows ADC Admins Active Directory Group to authenticate.
  • If TACACS, firmware is 12.0 build 57 or newer to prevent TACACS Accounting from blocking AAA.
  • nsroot account has external authentication disabled.
  • No local ADC accounts except nsroot.
  • NTP and Time Zone are configured.
  • Syslog is configured to send logs to external SIEM, especially if ADC is performing authentication.
  • SNMP Traps are sent to Citrix ADM appliance.
    • Thresholds are configured for CPU and Memory alarms.
  • Customer Experience Improvement Program (CUXIP) is disabled.
  • Recommended TCP Profile Settings are configured.
  • Drop Invalid HTTP requests is enabled in HTTP global settings.
  • Secure Access Only is enabled on all NSIPs and all management-enabled SNIPs – check both nodes of High Availability pair.
    • Management certificate has no certificate errors.
  • Networking:
    • If Dedicated Management Network, Policy Based Routes (PBR) are configured for NSIP reply traffic and NSIP-initiated traffic.
    • Unused network interfaces are disabled.
    • ADC instance is connected to only one security zone – if connected to multiple security zones, then a firewall is bypassed.
    • Default route should be Internet facing, or a data VLAN – not NSIP VLAN.
    • Only one default route – extra default routes can come from HA pairing or hardware migration.
  • Unused ADC configurations are removed – unused server objects, unused policies, etc.
  • Citrix ADM monitors and backs up the ADC appliances.
  • ADC Dashboard shows that CPU, Memory, and Throughput have not exceeded appliance capacity or appliance licensing.
  • /var/core and /var/crash do not have recent crash dumps.

ADC High Availability Pair

  • Firmware build is identical on both nodes.
  • Installed Licenses are identical on both nodes.
  • NTP and time zones are configured on both appliances – Configuration node shows System Time.
  • Unused interfaces are disabled.
  • HA is synchronizing without error.
  • Both HA nodes are set to ENABLED – not STAYPRIMARY and/or STAYSECONDARY.
  • Fail-safe mode is enabled.
  • “show ha node” shows heartbeats across all interfaces – no “interfaces on which heartbeats are not seen”.
  • High Availability failover has been tested, including RADIUS authentication, which might come from a different source IP.

ADC SDX

  • LOM port is connected and configured.
    • LOM nsroot password is not nsroot.
  • No hardware problems shown on SDX SVM dashboard page.
  • SDX firmware is current – should be same or newer than the VPX firmware.
  • SDX SVM nsroot password is not nsroot. nsroot password is complex. Admins don’t use nsroot to login.
  • Management authentication is configured for external authentication server, typically LDAP.
    • LDAP is load balanced instead of multiple LDAP Policies to individual LDAP servers – avoids premature account lockout.
    • LDAP is encrypted: LDAPS on port 636.
    • LDAP Bind account is a service account – not a regular user whose password expires.
    • LDAP Search Filter only allows ADC SDX Admins Active Directory Group to authenticate.
  • No local accounts except nsroot.
  • No certificate errors when accessing SVM management using htttps.
    • HTTPS is forced in System Settings – HTTP is not allowed.
  • Multiple DNS servers are configured in Networking Configuration – initial setup only asks for one DNS server.
  • Channels are created at SDX SVM instead of inside VPX instances.
  • NTP is configured and enabled.
  • Syslog is configured.
  • SNMP traps are sent to Citrix ADM.
  • The number of SDX instance licenses installed matches what’s owned at http://mycitrix.com/
  • SDX SVM Backups are configured with External Transfer – or download periodically – or ADM.
  • VPX Instances:
    • Platinum Edition license is assigned to instances.
    • SSL Chips are assigned to VPX instances.
    • All SDX hardware is allocated to VPX instances – If not, why not?
    • Production instances typically have Dedicated CPU cores. Test/Dev instances typically have Shared CPU.
    • VLANs are specified inside VPX instances instead of at instance properties on SDX Management Service – avoids reboot if you need to change the VLAN configuration.
    • No VMACs in instance interface settings.

ADC Load Balancing and SSL

  • Load Balancing configurations are documented.
  • Monitors do more than just telnet – e.g. LDAP monitor performs LDAP query.
    • LDAP monitor bind account uses service account, not domain admin.
    • LDAP monitor is filtered to cn=builtin – to reduce result size.
    • RADIUS monitor looks for response code 2 or 3.
  • If multiple Virtual Servers for multiple ports on the same VIP, configure Persistency Group – e.g. Horizon Load Balancing.
  • Rewrite policies remove web server header information (Server, X-Powered-By, etc.)
  • SSL Labs SSL Server Test shows A or A+ grade for all Internet-facing SSL vServers.
  • Redirect Virtual Servers are UP (Responder method) instead of DOWN (Backup URL method).
  • Custom (non-default) ciphers are bound to every SSL Virtual Server – see Citrix Networking SSL / TLS Best Practices.
  • SSL v3 and TLS v1.0 are disabled on every SSL Virtual Server.
  • SSL Renegotiation is set to NONSECURE: configured globally, or in SSL Profiles (including default profile).
  • Root certificate is not linked to intermediate certificate.
  • Certificates are not expired.
  • ADM alerts ADC administrators when certificates are soon to expire.
  • ADM Analytics is enabled for the HTTP Virtual Servers.
    • ADM Web Insight is viewed.
  • Bot Management (13.0 build 41+) and/or Web App Firewall are configured if ADC Premium Edition.
    • ADM Security Insight is enabled and viewed.

Citrix ADM

  • ADM exists and manages all ADC appliances.
    • Prompt credentials for instance login is enabled in ADM System Settings – if ADM does Single Sign-on to instances, then all instance changes are logged as nsroot instead of ADM user.
  • ADM firmware version is 12.1 build 54.13 and later or 13.0 build 41.20 and later – security vulnerability – many bugs are fixed in ADM 12.1 build 53 and newer. 💡
    • ADM Agents and DR nodes have same firmware version as ADM – check /var/mps/log/install_state
  • Two DNS servers are configured in ADM Network Configuration – initial setup only asks for one DNS server.
  • Two ADM appliances in High Availability mode with Floating IP – provides redundancy.
  • Every High Availability node and DR node has same disk size.
  • ADM nsroot password is not nsroot. nsroot password is complex. Admins don’t use nsroot to login.
  • Management authentication is configured for external authentication server, typically LDAP.
    • LDAP is load balanced instead of multiple LDAP Policies to individual LDAP servers – avoids premature account lockout.
    • LDAP is encrypted: LDAPS on port 636.
    • LDAP Bind account is a service account – not a regular user whose password expires.
    • LDAP Search Filter only allows ADM Admins Active Directory Group to authenticate.
  • No local accounts except nsroot.
  • No certificate errors when accessing ADM management using htttps.
    • HTTPS is forced in System Settings – HTTP is not allowed
  • Time zone is configured.
  • NTP is configured and enabled.
  • ADM Database is not full. Sufficient disk space.
  • Sufficient ADM CPU/Memory – verify at System > Statistics or System > Deployment.
  • SSL Dashboard alert notifications are enabled to warn of upcoming certificate expiration.
  • Event Rules are configured to email ADC administrators of Critical or Major ADC alarms.
  • ADC Instance Backup settings on ADM:
    • Number of ADC instance backups retained is sufficient for restoring from history.
    • ADC Backups are transferred to external SFTP, SCP, or FTP server.
    • ADC Restore process is documented and tested.
  • VIP Licensing:
    • Installed license count on ADM matches the licenses owned at http://mycitrix.com.
    • Licenses are assigned to Virtual Servers that need Analytics (e.g. HDX Insight) or Applications tab.
    • AppFlow/Insight is enabled on Gateway and HTTP Virtual Servers.
    • License expiration notifications are enabled.
  • Private IP Blocks are configured for geo mapping of ADC instances and Analytics sessions.
  • Analytics Thresholds are configured – e.g. ICA Latency threshold.
  • Session Reliability on HA Failover is enabled on ADC instances in ICA Parameters – if not enabled, then sessions drop on failover.
  • ADM HDX Insight is linked to Director Premium Edition using https protocol, not http protocol.

Citrix Gateway ICA Proxy

Gateway Virtual Server:

  • SSL Labs SSL Server Test shows A or A+ when it scans the Gateway external FQDN.
  • If ICA Only is unchecked on the Gateway Virtual Server, then System > Licenses shows sufficient Maximum Citrix Gateway Users Allowed.
  • Gateway Virtual Server Maximum Users is 0, which means unlimited.
  • TCP Profile is configured with Recommended TCP Profile Settings.
  • DTLS is enabled on the Virtual Server for EDT protocol.
    • UDP ports are open on firewall from Internet and to VDAs.
    • Director Session Details shows HDX protocol as UDP.
  • ICA Connections shows port 2598 (Session Reliability enabled), not 1494.
  • Gateway communication to StoreFront is https protocol, not http.
  • Gateway communication to StoreFront is load balanced to multiple StoreFront servers – not a single StoreFront server.
  • STAs on Citrix Gateway matches StoreFront configuration.
  • If EPA is used for SmartAccess, then Endpoint Analysis Libraries are updated.

Gateway Authentication:

  • Encrypted LDAP:
    • LDAP is load balanced instead of multiple LDAP Policies to individual LDAP servers – avoids premature account lockout.
    • LDAP is encrypted: LDAPS on port 636.
    • LDAP Bind account is a service account – not a regular user whose password expires.
    • LDAP Search Filter only allows authorized remote users in an Active Directory group to authenticate.
  • Two-factor authentication – RADIUS:
    • For Workspace app or Receiver, password fields are swapped.
    • Both factors are required to login. Can’t bypass second factor.
    • RADIUS tested from both High Availability nodes (perform failover).
  • SAML Authentication:
    • Prefer RADIUS over SAML so that ADC will have access to the user’s password to facilitate Single Sign-on to the VDA machines.
    • If SAML response does not provide user’s password, then Federated Authentication Service (FAS) is deployed .
    • For Workspace app support of SAML, SAML is configured in nFactor (AAA), not Gateway – requires ADC 12.1 and newest Workspace app.
    • SAML iDP Signing certificate is not expired. ADC administrators know how to update the Signing certificate.
  • Native OTP:
    • OTP Active Directory attribute is encrypted

Citrix ADC GSLB

  • If a DNS name resolves to multiple IP addresses, then the DNS name should be GSLB-enabled for automatic failover.
  • DNS Records are delegated to two or more ADC ADNS services, usually in separate data centers.
    • NS records and SOA records are added to ADC for delegated domain names and/or delegated sub zones.
  • All ADC nodes that have ADNS listeners for the same DNS name have identical GSLB configuration.
  • Public GSLB Services have monitors that verify remote Internet connectivity – don’t give out IP if users can’t reach it.
  • Separate ADC appliances for public DNS and internal DNS – If both are on one appliance, then how are the DNS configurations separated?
  • RPC nodes for Metric Exchange Protocol (MEP) should have Secure enabled.
  • Firewall should only allow the MEP endpoints to communicate over 3009 – don’t open to whole Internet.
  • If Static Proximity:
    • Static Proximity database is current.
    • GSLB Services show correct geo location.
    • Custom Entries are added for internal subnets.
  • If DNS Views, DNS Views are configured on all GSLB Services – if GSLB Service doesn’t have a DNS View, then that GSLB Service might not function correctly.
  • If Active/Active GSLB load balancing, then site persistence is functioning correctly.
  • DNS security options are configured to prevent ADNS Denial of Service.

EUC Weekly Digest – September 7, 2019

Last Modified: Sep 7, 2019 @ 6:37 am

Interesting EUC items from last week:

Citrix Virtual Apps and Desktops

Citrix StoreFront

Citrix ADM

Citrix Endpoint Management

Citrix Cloud

VMware

Microsoft

For more immediate updates, follow me at http://twitter.com/cstalhood.

For a list of updates at carlstalhood.com, see the Detailed Change Log.

Self-Service Password Reset (SSPR) – Citrix ADC

Last Modified: Sep 1, 2019 @ 11:14 am

Navigation

SSPR Flow

The SSPR feature in Citrix ADC is completely unrelated to the SSPR in Citrix StoreFront. They are separate registrations.

SSPR is available in Citrix ADC 12.1 build 51.16 and higher.

SSPR requires nFactor, which is only available in Citrix ADC Advanced Edition and Premium Edition. You can use nFactor with Citrix Gateway.

SSPR Registration doesn’t work in Citrix Workspace app.

The first factor is a Login Schema that has an optional Forgot Password link. The Login Schema file named SingleAuthPasswordResetRem.xml has this link. There are other built-in Login Schemas for Username Only and Dual Auth that also have the link.

If the user logs in normally, then ADC checks if the user has been enrolled in SSPR or not. If not enrolled, then the user is prompted to select questions and provide answers. This is a custom Login Schema that you can create from the ADC GUI as described later.

If the user clicks the Forgot Password link, then the user is prompted to answer two of the previously enrolled questions.

  • Optionally, the user might be prompted to retrieve a One-time password (OTP) sent to an Alternate Email address. The user might not have access to corporate email, so the OTP should be sent to the user’s personal email address. The user enters the alternate email address during SSPR registration.

  • Finally, the user is prompted to enter a new password.

Here’s what a sample SSPR nFactor configuration looks like:

# ** nFactor Visualizer 
# ** ------------------ 
# ** AAA vserver: AAA-SSPR
# **    Login Schema Policy = lschema_password_reset
# **       Priority = 20
# **       Rule = true
# **       Login Schema XML = "/nsconfig/loginschema/LoginSchema/SingleAuthPasswordResetRem.xml"
# **    Adv Authn Policy = LDAP_Password_Reset
# **       Priority = 10
# **       Rule = "aaa.LOGIN.VALUE(\"passwdreset\").EQ(\"1\")"
# **       Action = ldapAction named LDAP-Corp_NoAuth
# **       Goto if failed = NEXT
# **       Next Factor if Success = kba_validation
# **          Login Schema Profile = LSCHEMA_INT
# **          Adv Authn Policy = kba_validation
# **             Priority = 10
# **             Rule = true
# **             Action = ldapAction named LDAP-Corp_NoAuth
# **             Goto if failed = NEXT
# **             Next Factor if Success = email_validation
# **                Login Schema Profile = LSCHEMA_INT
# **                Adv Authn Policy = email_validation
# **                   Priority = 10
# **                   Rule = true
# **                   Action = emailAction named EmailOTP
# **                   Goto if failed = NEXT
# **                   Next Factor if Success = password_reset
# **                      Login Schema Profile = LSCHEMA_INT
# **                      Adv Authn Policy = LDAP-Corp-Adv_NoAuth
# **                         Priority = 100
# **                         Rule = true
# **                         Action = ldapAction named LDAP-Corp_NoAuth
# **                         Goto if failed = NEXT
# **    Adv Authn Policy = LDAP-Corp-Adv
# **       Priority = 20
# **       Rule = true
# **       Action = ldapAction named LDAP-Corp
# **       Goto if failed = NEXT
# **       Next Factor if Success = check_for_sspr_registration
# **          Login Schema Profile = LSCHEMA_INT
# **          Adv Authn Policy = switch_to_kba_register
# **             Priority = 1
# **             Rule = "AAA.USER.ATTRIBUTE(\"kba_registered\").EQ(\"0\")"
# **             Action = NO_AUTHN
# **             Goto if failed = NEXT
# **             Next Factor if Success = kba_registration
# **                Login Schema Profile = lschema_KBARegistration
# **                Adv Authn Policy = first_time_login_forced_kba_registration
# **                   Priority = 1
# **                   Rule = true
# **                   Action = ldapAction named LDAP-Corp
# **                   Goto if failed = NEXT
# **          Adv Authn Policy = NoAuth
# **             Priority = 100
# **             Rule = true
# **             Action = NO_AUTHN
# **             Goto if failed = END

Active Directory User Attribute

Citrix ADC SSPR stores its configuration in an Active Directory User attribute.

The attribute must be dedicated to SSPR. You can’t use the same attribute that you might already be using for Citrix ADC One-time passwords (OTP).

  • In Active Directory Users and Computers, enable Advanced Features mode (from the View menu), double-click a user, switch to the Attribute Editor tab, and find a String attribute that is not being used. When you double-click an attribute, look for String in the window title.
  • Open the Active Directory Schema console. Under Attributes, find your candidate attribute and double-click it. Make sure the Maximum length is high (e.g. 32767). The extensionAttributes default to 1024 or 2048, which is not long enough.
  • You might want to uncheck the bottom boxes (e.g. Attribute is copied when duplicating a user) since each user has their own SSPR configuration.
  • ADC needs permissions to set this Attribute and reset user passwords. You can use the Delegate Control wizard to grant these permissions.

    1. Consider creating a group for SSPR Admins and delegating permissions to that group. You can add your LDAP Bind Account and your Service Desk accounts to this group.
    2. In the Tasks to Delegate page, change it to Create a custom task to delegate and click Next.
    3. In the Active Directory Object Type page, change the selection to Only the following objects in this folder and select User objects. Click Next.
    4. In the Permissions page, on the top of the page, check the box for Property-specific.
    5. In the Permissions list, check the box next to Reset password. Scroll down.
    6. Find the attribute you are using for SSPR and check the Read and Write boxes. Click Next.

Help Desk will need to be trained to reset a user’s SSPR registration by double-clicking the attribute and clicking the Clear button. The next time the user logs in, the user will be prompted to enroll for SSPR again.

ADC uses a certificate to encrypt the contents of the Attribute. It currently is not possible to configure the certificate from the GUI, so you’ll need to SSH to the ADC and run the following command:

bind vpn global -userDataEncryptionKey MyCertificate

The ADC SSPR configuration needs two LDAP Policy/Servers: one with authentication enabled, and one without authentication. In both LDAP Servers, in the Other Settings section, enter the Attribute name into the Alternate Email field and in the KB Attribute field. The same attribute can be used for both. However, you cannot use the same attribute for the OTP Secret field.

SSPR KBA Registration Login Schema

When creating the KBA Registration Login Schema you will be prompted to choose from a list of questions. This list is configured in the file KBQuestions.json located in /nsconfig/loginschema/LoginSchema.

  1. In ADC GUI, on the left, expand Security, expand AAA – Application Traffic, and click Login Schema.
  2. On the right, switch to the Profiles tab.
  3. On the right, click the button labelled Add KBA Registration Login Schema.
  4. Give the Schema a name. ADC will create a file with the same name.
  5. Configure at least two questions. Move questions from the left to the right. Note: the XML file that it creates has size limits so don’t choose too many questions.
  6. On the right, you can optionally change the labels for Question Field and Answer Field. If you don’t enter anything, then the labels are Question, and Answer, as shown earlier in this article. Scroll down.
  7. Near the bottom are two More links you can click.
  8. The first More link lets you allow users to enter their own questions.
  9. The second More link lets you enable Register Alternate Email.
  10. Click Create when done.
  11. ADC creates a file in /nsconfig/loginschema/LoginSchema that matches the Schema name you entered earlier.
  12. ADC also adds a Login Schema Profile with the same name you entered earlier.

LDAP Servers

You’ll need two LDAP Servers: one with Authentication enabled, and the other with Authentication unchecked. Other than the Authentication checkbox, they are identical. Configure them like normal LDAP Servers.

The NoAuth server has Authentication unchecked.

Configure both LDAP Servers with the SSPR attribute as detailed earlier.

Email Action and Policy

You can optionally configure an Email action to send the user a OTP before allowing the password reset.

  1. In the ADC GUI, the easy way to find Email Actions is to enter “email” in the top left search box.
  2. On the right, click Add.
  3. Give the Email Action a name.
  4. In the Username box, enter the From email address. The emails will look like they come from this address.
  5. Enter the SMTP password for this From address.
  6. In the Server URL box, enter a smtps URL with port number similar to smtps://mail01.corp.com:587. Gmail’s URL smtps://smtp-relay.gmail.com:587 also works.
  7. In the Content box, enter a string containing $code. ADC replaces $code with the OTP.
  8. In the Type box, enter SMTP.
  9. In the Email Address box, enter aaa.user.attribute("alternate_mail"). This expression pulls the destination email address from Alternate Email attribute you defined in your LDAP Server.
  10. Click Create.
  11. It does not appear to be possible to use the ADC GUI to create an Email Authentication Policy. SSH to your ADC and run something similar to the following command:
    add authentication Policy email_validation -rule true -action EmailOTP

SSPR nFactor Configuration

AAA vServer and Login Schema

The first factor will be a Login Schema that has the Reset Password link. This section creates the following portion of the nFactor configuration:

# ** AAA vserver: AAA-SSPR
# **    Login Schema Policy = lschema_password_reset
# **       Priority = 20
# **       Rule = true
# **       Login Schema XML = "/nsconfig/loginschema/LoginSchema/SingleAuthPasswordResetRem.xml"

Do the following:

  1. Create a AAA vServer.
  2. It can be Non Addressable.
  3. Optionally bind a certificate to the AAA vServer so its status will be Green instead of Red.
  4. On the right, in the Advanced Settings column, click Login Schemas to move it to the left.
  5. On the left, in the Login Schemas section, click where it says No Login Schema.
  6. Next to the Select Policy field, click Add to create a new Login Schema Policy.
  7. Give the Login Schema Policy a name to indicate that this Login Schema includes a link for Self-Service Password Reset.
  8. Next to Profile, click Add to create a new Login Schema Profile.
  9. Give the Login Schema Profile a name.
  10. In the Authentication Schema field, click the edit icon.
  11. In the Login Schema Files section, click the LoginSchema folder.
  12. Highlight one of the PasswordReset files (e.g. SingleAuthPasswordResetRem.xml) and then click the blue Select button on the top right. You can also optionally modify the built-in files.
  13. The Create Authentication Login Schema window collapses and the Authentication Schema field shows the selected file. Click Create when done.
  14. Click Create to finish creating the Authentication Login Schema Policy.
  15. Click Bind to bind the Login Schema Policy to the AAA vServer.

SSPR Registration

Let’s start with the SSPR Registration flow. This section creates the following snippet of the full nFactor configuration:

# ** AAA vserver: AAA-SSPR
# **    Login Schema Policy = lschema_password_reset
# **       Priority = 20
# **       Rule = true
# **       Login Schema XML = "/nsconfig/loginschema/LoginSchema/SingleAuthPasswordResetRem.xml"
# **    Adv Authn Policy = LDAP-Corp-Adv
# **       Priority = 20
# **       Rule = true
# **       Action = ldapAction named LDAP-Corp
# **       Goto if failed = NEXT
# **       Next Factor if Success = check_for_sspr_registration
# **          Login Schema Profile = LSCHEMA_INT
# **          Adv Authn Policy = switch_to_kba_register
# **             Priority = 1
# **             Rule = "AAA.USER.ATTRIBUTE(\"kba_registered\").EQ(\"0\")"
# **             Action = NO_AUTHN
# **             Goto if failed = NEXT
# **             Next Factor if Success = kba_registration
# **                Login Schema Profile = lschema_KBARegistration
# **                Adv Authn Policy = first_time_login_forced_kba_registration
# **                   Priority = 1
# **                   Rule = true
# **                   Action = ldapAction named LDAP-Corp
# **                   Goto if failed = NEXT
# **          Adv Authn Policy = NoAuth
# **             Priority = 100
# **             Rule = true
# **             Action = NO_AUTHN
# **             Goto if failed = END

Do the following:

  1. In the AAA vServer, scroll down to the Advanced Authentication Policies section and click where it says No Authentication Policy.
  2. In the Select Policy field, select your normal LDAP policy. The Login Schema collects username and password and verifies it with the LDAP Policy that you select here.
  3. Change the Priority to 20 or similar. This binding must be a higher number than the Password Reset flow that we configure later.
  4. Next to the Select Next Factor field, click Add to create a Policy Label which will check if the user is registered or not.
  5. Name the Policy Label to indicate that this Policy Label checks if the user is registered or not.
  6. Leave the Login Schema set to LSCHEMA_INT (i.e. no schema), and click Continue.
  7. Next to the Select Policy field, click Add to create an Authentication Policy.
  8. Name this Authentication Policy to indicate that it does nothing more than check if the user is registered or not.
  9. In the Expression field, enter the following expression:
    AAA.USER.ATTRIBUTE("kba_registered").EQ("0")
  10. Click Create.
  11. Back in the Policy Label Policy Binding window, next to the Select Next Factor field, click Add to create another Policy Label.
  12. Name the Policy Label to indicate that this factor performs the SSPR Registration (i.e. Knowledge-Based Authentication registration).
  13. In the Login Schema field, select the KBA Registration Login Schema that you created earlier.
  14. Click Continue.
  15. Next to the Select Policy field, click Add to create an Authentication Policy.
  16. Name the Authentication Policy to indicate that ADC uses this LDAP Server to store the KBA registration information in the Active Directory User attribute.
  17. Change Action Type to LDAP.
  18. Change Action to your normal authenticating LDAP Server that has the KB Attribute defined.
  19. For Expression, enter true.
  20. Click Create.
  21. Back in the Policy Label Policy Binding page, click Bind.
  22. Click Done to finish creating the Policy Label for KBA Registration.
  23. Back in the “Check for SSPR Registration” Policy Label Policy Binding page, click Bind.
  24. To handle users that are already registered, click Add Binding.
  25. Next to the Select Policy field, click Add to create an Authentication Policy.
  26. Name the Policy to indicate that no authentication is occurring. This policy always returns true and is necessary to avoid errors for users that are already registered.
  27. Change Action Type to NO_AUTHN.
  28. In the Expression box, enter true.
  29. Click Create.
  30. Back in the Policy Label Policy Binding screen, click Bind.
  31. Click Done to close the “Check for SSPR Registration” Policy Label.
  32. Click Bind to bind the LDAP Policy and “Check for SSPR Registration” factor to the AAA vServer.

Password Reset

This section creates the following snippet of the full nFactor configuration:

# ** AAA vserver: AAA-SSPR
# **    Login Schema Policy = lschema_password_reset
# **       Priority = 20
# **       Rule = true
# **       Login Schema XML = "/nsconfig/loginschema/LoginSchema/SingleAuthPasswordResetRem.xml"
# **    Adv Authn Policy = LDAP_Password_Reset
# **       Priority = 10
# **       Rule = "aaa.LOGIN.VALUE(\"passwdreset\").EQ(\"1\")"
# **       Action = ldapAction named LDAP-Corp_NoAuth
# **       Goto if failed = NEXT
# **       Next Factor if Success = kba_validation
# **          Login Schema Profile = LSCHEMA_INT
# **          Adv Authn Policy = kba_validation
# **             Priority = 10
# **             Rule = true
# **             Action = ldapAction named LDAP-Corp_NoAuth
# **             Goto if failed = NEXT
# **             Next Factor if Success = email_validation
# **                Login Schema Profile = LSCHEMA_INT
# **                Adv Authn Policy = email_validation
# **                   Priority = 10
# **                   Rule = true
# **                   Action = emailAction named EmailOTP
# **                   Goto if failed = NEXT
# **                   Next Factor if Success = password_reset
# **                      Login Schema Profile = LSCHEMA_INT
# **                      Adv Authn Policy = LDAP-Corp-Adv_NoAuth
# **                         Priority = 100
# **                         Rule = true
# **                         Action = ldapAction named LDAP-Corp_NoAuth
# **                         Goto if failed = NEXT

Do the following to create the Password Reset flow:

  1. In the AAA vServer, in the Advanced Authentication Policies section, click where it says 1 Authentication Policy
  2. Note the Priority number of the existing binding. Our new binding will need a lower priority number so it is evaluated first. Click Add Binding.
  3. Next to the Select Policy field, click Add to create an Authentication Policy.
  4. Name the Authentication Policy to indicate that this policy only applies if the user clicked the Forgot Password link in the Login Schema.
  5. Change Action Type to LDAP.
  6. Change Action to the your LDAP Server that has Authentication unchecked (i.e. not enabled).
  7. In the Expression box, enter the following expression that returns true if the user clicked the link:
    aaa.LOGIN.VALUE("passwdreset").EQ("1")
  8. Click Create.
  9. Back in the Policy Binding page, change the Priority to a number that is lower than the existing Policy Binding.
  10. Next to the Select Next Factor field, click Add to create a Policy Label (i.e. factor)
  11. Name the Policy Label to indicate that this factor will verify the user’s questions and answers.
  12. Leave the Login Schema set to the default LSCHEMA_INT (i.e. no schema) and click Continue.
  13. Next to the Select Policy field, click Add to create an Authentication Policy.
  14. Name the Authentication Policy to indicate that this policy verifies the user’s KBA questions and answers.
  15. Change Action Type to LDAP.
  16. Change Action to your LDAP Server that does not require authentication.
  17. In the Expression box, enter true.
  18. Click Create.
  19. Back in the Policy Label Policy Binding page, if you wish to add email validation, then next to Select Next Factor, click Add to create another Policy Label (factor).
  20. Name the Policy Label to indicate that this factor will perform Email OTP verification.
  21. Leave the Login Schema set to the default LSCHEMA_INT (i.e. no schema) and click Add.
  22. In the Policy Label Policy Binding section, click where it says Click to select.
  23. Click the small circle next to the email policy that you created earlier from the ADC’s command line. It’s not possible to use the GUI to create the email policy.
  24. At the top of the screen, click the blue Select button.
  25. Back in the Policy Label Policy Binding screen, next to the Select Next Factor field, click Add to create another Policy Label (i.e. factor).
  26. Name this Policy Label to indicate that this factor performs the password reset.
  27. Leave the Login Schema set to the default LSCHEMA_INT (i.e. no schema) and click Continue.
  28. In the Policy Label Policy Binding section, next to the Select Policy field, click Add to create an Authentication Policy.
  29. Name the Authentication Policy to indicate that this is your LDAP Policy that has Authentication unchecked (not enabled).
  30. Change Action Type to LDAP.
  31. Change Action to your LDAP Server that has Authentication unchecked.
  32. In the Expression field, enter true.
  33. Click Create.
  34. That’s it for the AAA configuration. Now click Bind and Done several times to return to the AAA vServer.






  35. Click Close to finish the AAA vServer’s Authentication Policy bindings.

Bind to Citrix Gateway

If you are using nFactor configuration with Citrix Gateway, then configure the Authentication Profile like normal.

  1. Go to Citrix Gateway > Virtual Servers and edit a Gateway.
  2. On the right, in the Advanced Settings column, click Authentication Profile to move it to the left.
  3. On the left, in the Authentication Profile section, click Add.
  4. Name the Authentication Profile to indicate that this nFactor Flow does Self-Service Password Reset.
  5. In the Authentication Virtual Server field, click where it says Click to select.
  6. Click the small circle next to the AAA vServer that is configured with your SSPR flow.
  7. At the top of the page, click the blue Select button.
  8. Click Create.
  9. Click OK to select the Authentication Profile.

CLI Commands

Here’s a full sample ADC CLI configuration:

# Certs
# -----
add ssl certKey WildcardCorpCom.cer_CERT_KE_ic1 -cert WildcardCorpCom.cer_ic1 -passcrypt b217e266 -encrypted -encryptmethod ENCMTHD_3

add ssl certKey WildcardCorpCom.cer_CERT_KEY -cert WildcardCorpCom.pfx -key WildcardCorpCom.pfx -inform PFX -passcrypt 52e54af -encrypted -encryptmethod ENCMTHD_3 -expiryMonitor DISABLED

add ssl certKey CorpIntermediate -cert CorpIntermediate.cer -inform DER -passcrypt cc9675 -encrypted -encryptmethod ENCMTHD_3 -expiryMonitor DISABLED

# Cert Links
# ----------
link ssl certKey WildcardCorpCom.cer_CERT_KEY CorpIntermediate

link ssl certKey CorpIntermediate WildcardCorpCom.cer_CERT_KE_ic1


# AAA Global Settings
# -------------------
enable ns feature AAA


# LDAP Actions
# ------------
add authentication ldapAction LDAP-Corp -serverIP 10.2.2.11 -serverPort 636 -ldapBase "dc=corp,dc=local" -ldapBindDn ctxsvc@corp.local -ldapBindDnPassword 868d22c -encrypted -encryptmethod ENCMTHD_3 -ldapLoginName sAMAccountName -groupAttrName memberOf -subAttributeName cn -secType SSL -passwdChange ENABLED -nestedGroupExtraction ON -groupNameIdentifier sAMAccountName -groupSearchAttribute memberOf -groupSearchSubAttribute CN -KBAttribute extensionAttribute1 -alternateEmailAttr extensionAttribute1

add authentication ldapAction LDAP-Corp_NoAuth -serverIP 10.2.2.11 -serverPort 636 -ldapBase "dc=corp,dc=local" -ldapBindDn ctxsvc@corp.local -ldapBindDnPassword a4ffb4 -encrypted -encryptmethod ENCMTHD_3 -ldapLoginName sAMAccountName -groupAttrName memberOf -subAttributeName cn -secType SSL -authentication DISABLED -passwdChange ENABLED -nestedGroupExtraction ON -groupNameIdentifier sAMAccountName -groupSearchAttribute memberOf -groupSearchSubAttribute CN -KBAttribute extensionAttribute1 -alternateEmailAttr extensionAttribute1


# Email (SSPR) Actions
# --------------------
add authentication emailAction EmailOTP -userName admin@corp.local -password 7868e -encrypted -encryptmethod ENCMTHD_3 -serverURL "smtps://mail01.corp.local:587" -content "Your OTP is $code" -emailAddress "aaa.user.attribute(\"alternate_mail\")"


# Advanced Authentication Policies
# --------------------------------
add authentication Policy LDAP-Corp-Adv -rule true -action LDAP-Corp

add authentication Policy kba_validation -rule true -action LDAP-Corp_NoAuth

add authentication Policy LDAP_Password_Reset -rule "aaa.LOGIN.VALUE(\"passwdreset\").EQ(\"1\")" -action LDAP-Corp_NoAuth

add authentication Policy NoAuth -rule true -action NO_AUTHN

add authentication Policy switch_to_kba_register -rule "AAA.USER.ATTRIBUTE(\"kba_registered\").EQ(\"0\")" -action NO_AUTHN

add authentication Policy email_validation -rule true -action EmailOTP

add authentication Policy first_time_login_forced_kba_registration -rule true -action LDAP-Corp

add authentication Policy EmailOTP -rule true -action EmailOTP

add authentication Policy LDAP-Corp-Adv_NoAuth -rule true -action LDAP-Corp_NoAuth


# Login Schemas
# -------------
add authentication loginSchema lschema_KBARegistration -authenticationSchema "/nsconfig/loginschema/LoginSchema/lschema_KBARegistration.xml"

add authentication loginSchema lschema_password_reset -authenticationSchema "/nsconfig/loginschema/LoginSchema/SingleAuthPasswordResetRem.xml"


# Login Schema Policies
# ---------------------
add authentication loginSchemaPolicy lschema_password_reset -rule true -action lschema_password_reset


# Authentication Policy Labels
# ----------------------------
add authentication policylabel kba_registration -loginSchema lschema_KBARegistration

bind authentication policylabel kba_registration -policyName first_time_login_forced_kba_registration -priority 1 -gotoPriorityExpression NEXT

add authentication policylabel check_for_sspr_registration -loginSchema LSCHEMA_INT

bind authentication policylabel check_for_sspr_registration -policyName switch_to_kba_register -priority 1 -gotoPriorityExpression NEXT -nextFactor kba_registration

bind authentication policylabel check_for_sspr_registration -policyName NoAuth -priority 100 -gotoPriorityExpression END

add authentication policylabel password_reset -loginSchema lschema_noschema

bind authentication policylabel password_reset -policyName LDAP-Corp-Adv_NoAuth -priority 100 -gotoPriorityExpression NEXT

add authentication policylabel email_validation -loginSchema lschema_noschema

bind authentication policylabel email_validation -policyName email_validation -priority 10 -gotoPriorityExpression NEXT -nextFactor password_reset

add authentication policylabel kba_validation -loginSchema lschema_noschema

bind authentication policylabel kba_validation -policyName kba_validation -priority 10 -gotoPriorityExpression NEXT -nextFactor email_validation


# Authentication Virtual Servers
# ------------------------------
add authentication vserver AAA-SSPR SSL 0.0.0.0

bind authentication vserver AAA-SSPR -portaltheme RfWebUI

bind authentication vserver AAA-SSPR -policy lschema_password_reset -priority 20 -gotoPriorityExpression END

bind authentication vserver AAA-SSPR -policy LDAP_Password_Reset -priority 10 -nextFactor kba_validation -gotoPriorityExpression NEXT

bind authentication vserver AAA-SSPR -policy LDAP-Corp-Adv -priority 20 -nextFactor check_for_sspr_registration -gotoPriorityExpression NEXT


# Authentication Profiles
# -----------------------
add authentication authnProfile SSPR -authnVsName AAA-SSPR


# NetScaler Gateway Global Bindings
# ---------------------------------
bind vpn global -userDataEncryptionKey WildcardCorpCom.cer_CERT_KEY


# NetScaler Gateway Virtual Servers
# ---------------------------------
add vpn vserver gateway2 SSL 10.2.4.220 443 -Listenpolicy NONE -tcpProfileName nstcp_default_XA_XD_profile -httpProfileName http2 -deploymentType ICA_STOREFRONT -authnProfile SSPR -vserverFqdn gateway3.corp.com

bind vpn vserver gateway2 -portaltheme RfWebUI

bind vpn vserver gateway2 -policy PL_OS_10.2.4.120 -priority 100

bind vpn vserver gateway2 -policy PL_WB_10.2.4.120 -priority 100


# SSL Virtual Servers
# -------------------
bind ssl vserver gateway2 -certkeyName WildcardCorpCom.cer_CERT_KEY

bind ssl vserver AAA-SSPR -certkeyName WildcardCorpCom.cer_CERT_KEY

EUC Weekly Digest – August 31, 2019

Last Modified: Aug 31, 2019 @ 5:44 am

Interesting EUC items from last week:

Citrix Virtual Apps and Desktops

Citrix App Layering

Citrix WEM and Profile Management

Citrix Workspace app

Citrix ADC

VMware

Other

For more immediate updates, follow me at http://twitter.com/cstalhood.

For a list of updates at carlstalhood.com, see the Detailed Change Log.

EUC Weekly Digest – August 24, 2019

Last Modified: Aug 24, 2019 @ 6:55 am

Interesting EUC items from last week:

Citrix StoreFront

Citrix ADC

VMware

Other

For more immediate updates, follow me at http://twitter.com/cstalhood.

For a list of updates at carlstalhood.com, see the Detailed Change Log.