I’ve deployed a lot of 2 factor authentication products with Citrix NetScaler Gateway in my career but the one I’ve always liked a lot is Microsoft Azure Multi-Factor Authentication (MFA). I used to deploy this product years ago when it was called PhoneFactor. Microsoft purchased PhoneFactor in 2012 and I was worried that would be the end of the service. But Microsoft has really taken the product to the next level.
There are several steps involved in deploying it now that PhoneFactor is integrated with Microsoft Azure. I’m going to break it down step-by-step for you and cover the following in this article:
- Why You Should Deploy a 2 Factor Or Multi-Factor Authentication Solution
- Setting Up Azure Active Directory
- Setting Up Azure AD Connect
- Setting Up Azure AD Premium
- Setting Up Azure Multi-Factor Authentication Server On Premise
- Configuring Azure MFA Server For NetScaler Gateway
- Configuring NetScaler Gateway To Use MFA
- Modifying NetScaler Gateway Logon Page For MFA
- Troubleshooting MFA Authentication Issue
- Auditing & Reporting MFA Logins
- Azure MFA Hacking Attempt Notifications
- Consider Deploying the Azure MFA User Portal For Self Service
- Final Thoughts
WHY YOU SHOULD DEPLOY A 2 FACTOR OR MULTI-FACTOR AUTHENTICATION SOLUTION
There are many reasons why you should endeavor to deploy a 2 factor or multi-factor authentication system at any size organization. Privacy and security have never been so prevalent subjects of discussion in organizations as it has been in recent times. Pretty much any consumer level service worth mentioning has some form of 2 factor authentication available for users. I would almost call it a necessity these days.
A NetScaler is a very powerful device and you can setup all sorts of authentication methods on it (LDAP/LDAPS, RADIUS, SAML, OAuth, etc.) and the list keeps growing. But if someone were to guess your user’s ID and domain password they’re in and there’s nothing you can do about it unless you authenticate based on another factor. This is where the “something you have” part of 2 factor authentication comes in. A One Time Passcode/Password (OTP) usually in the form of a PIN number is provided to the user on something they have in their possession to verify they are who they say they are.
Years ago I used to deploy solutions using little tokens people had to carry around on their keychains like RSA SecurID. Nobody wanted to carry those little tokens around. If they loose it, there’s a cost to IT to have to replace it. It was tough for the Help Desk to manage them. And when the 2 factor system itself gets hacked like it did in 2011, you’re now having to face an organization wide token replacement dilemma: https://en.wikipedia.org/wiki/RSA_SecurID#March_2011_system_compromise
Then came phone apps like Google Authenticator, Symantec VIP, etc. with “soft tokens” that could be installed on the user’s phone. This has become more convenient for users but it still slows down the login process. It requires the user to have to transcribe a PIN code into the NetScaler Gateway logon page. One that changes every 60 seconds so if you have 10 seconds left on the countdown clock within the authentication app, type fast like it’s Mission Impossible or stand there and stare at your phone aimlessly for 10 seconds before the code changes and you can continue with the login process. People want things to be simple. They want to get in quickly. So this became a real hassle for them.
Citrix technologies are first and foremost about the User Experience. Whatever solution you choose needs to be easy on the users. If you don’t make technology easy and convenient to use they will reject the solution. They’ll circumvent it using an easier to use consumer level technology if they can. I covered this last year in article about User Experience and MDM. The same holds true for anything customer facing you deploy:
The two solutions that work quickly for the user are phone calls and SMS text messages. This is what many of my clients prefer now for their organizations. They are fast and convenient methods to verify a user’s identity. Azure MFA/PhoneFactor, SMSPasscode, Duo Security, etc. are all very well received in this space. With a phone call based system you simply answer the call and hit the “#” or other key you specify in the system and the user is in. With a text message you simply respond back to the text message or type in the PIN code presented to you and you’re in. Most all vendors support multiple paths to authenticate as fallback. Azure MFA for example has options like using a mobile app as well as a self service user portal website where the user can do a One-Time Bypass of MFA or enter security questions authenticate. Azure MFA even has support for OATH (Initiative For Open Authentication) tokens so it’s compatible with a variety of hard token manufacturers that support this standard.
You can get even more advanced than this. You can look at each login in real-time and compare several factors like the IP address for geo-location, company device vs. personal, time of day the login is occurring, etc. to analyze and react accordingly to the login. Let’s say a hacker in another country compromised the user’s cell phone carrier account and is forwarding calls and texts to their cell phone. They can get the One Time Password now but because their IP is coming from a different country or untrusted device, you can now react to this and deny access even though they have all the right credentials. Many of the services I have described can do this and you can also leverage some of these protection features on the NetScaler itself but that’s a whole other article. 🙂
SETTING UP AZURE ACTIVE DIRECTORY
I’m going to write this article assuming you have 0 presence in the Azure cloud right now. If you already have Azure AD up and running, skip to the next section. If you don’t, the first thing you’ll need to do is sign up for an Azure subscription followed by setting up AD Connect which will get you connected to the cloud. Even a trial subscription of Azure is fine:
Once that’s done, follow these steps to link your on premise Active Directory to Azure using AD Connect:
1. First login to to your Microsoft Azure account. For this example I’m using an MSDN subscription with a certain amount of monthly credit for Microsoft Azure included. Just sign in at:
and click “Activate Microsoft Azure” to activate your subscription if you haven’t done so already (will take about 3-4 min to get it setup). Otherwise you can go directly to the “Microsoft Azure Management Portal”:
2. In the Azure Management Portal (https://manage.windowsazure.com/), click the + New button in the bottom left:
3. You’ll be presented with a quick tour and then finally the All Items section where the only thing you’ll see is Default Directory. Go ahead and click on it:
4. First thing to do is create your domain. So go ahead and click “Add domain”. Please make sure this is a domain you already own by the way.
5. Add your domain name and click the check box saying “I plan to configure this domain for single sign-on with my local Active Directory”:
6. On the next screen it wants to take you to the Directory Integration page to verify domain ownership. Hit the check icon in the bottom right:
SETTING UP AZURE AD CONNECT
Note, AD Connect is not necessary if all you have is an on premise AD. I wanted to show you the whole cloud setup but if you only have an on premise Active Directory, then skip to the AD Premium setup in the next section.
7. You’ll notice you have 1 domain planned for single sign-on. So the next step is to run Azure AD Connect. Click the little blue cloud next to Users to get back to the Quick Start screen.
8. Download Azure AD Connect, it’s a 62 MB file called AzureADConnect.msi:
9. Run Azure AD Connect on a DC or a member server, both are supported. For production environments I always recommend have dedicated servers for everything. It does install a very light weight SQL Express LocalDB database on the server so I really recommend keeping it off a DC if you can. I know what you’re thinking, “What is this thing going to do to my Active Directory?”. Don’t be afraid. It’s just a simple sync tool and won’t do anything to your AD. Bill Mather wrote a great guide on integrating your on-premise Active Directory user IDs with Azure Active Directory here:
Hit Agree, then Continue
10. I chose Express Settings but you can Customize if you want to. You can point at a SQL Server, use a different service account, create custom sync groups, etc. You’ll want to do a Custom install if doing this in a production environment so you know everything it’s doing.
11. Type in your Azure AD credentials. Note, this account must end with “.onmicrosoft.com” and be a Global Admin in Azure. I just created a new user in Azure with the Global Administrator role for this purpose:
12. Next type in the credentials for an Enterprise Administrator in your local Active Directory
13. Click Install and it will immediately install and start syncing your Active Directory to Azure
14. Once it’s done hit Exit and confirm your local AD accounts are now in Azure AD.
15. You’ll notice all the Users are in the Users section and now under Directory Integration you are now ACTIVATED:
SETTING UP AZURE AD PREMIUM
Did you know there are presently 3 different version of Azure MFA? Microsoft allows you to set it up for Office 365 only, Azure Administrators only (comes with your Azure subscription), or what you probably came here for, any and all of your Active Directory users both for on prem and cloud authentication. You must be using that 3rd flavor to work with NetScaler Gateway. In order to get it, you must be using Azure AD Premium or the Enterprise Mobility Suite. Bill Mathers has a great article explaining the flavors here:
16. So go back to the Quick Start page and hit Try It Now under Azure AD Premium
17. Now click “Try Azure Active Directory Premium Now”
18. Then I get the message about the 30 day trial. My MSDN account comes with AD Basic which is part of every Azure subscription. AD Premium is an additional cost. Grrr… Go ahead and hit the check box and wait 2-3 min for it to finish upgrading you to Premium. You might want to manually hit the refresh link it shows you after this page, I’ve seen it hang up sometimes.
19. Now you’ll see your are using Azure Active Directory Premium. Go ahead and click on it:
20. Now you have the options to Assign Users or Enable Directory Features. Go ahead and click Assign Users
21. By default you’re going to see Assigned Users and it will be blank. You have to click on the drop down and select All Users then hit the Check button to actually see the users:
22. Select the user accounts you want to Assign (Ctrl+click) then click Assign at the bottom:
23. It will tell you at the bottom once it has completed
24. Now go back to the Azure AD Premium Quick Start screen and hit Enable directory features
25. Now you’ll see a bunch of Directory Properties. Some really cool stuff in here I’ll have to show you in a different article like the custom branding. Scroll down to “multi-factor authentication” and click Manage service settings.
26. You’ll get redirected to a different portal, don’t worry about this stuff now. Just click Go to the portal near the bottom:
27. Now the 3rd portal opens up. This is the Azure Multi-Factor Authentication Admin portal. Finally! Looks like the old PhoneFactor portal doesn’t it? You can verify on the home page that your Directory has it enabled. Go ahead and click “Server” in the left hand Downloads section.
SETTING UP AZURE MULTI-FACTOR AUTHENTICATION SERVER ON PREMISE
Though Azure MFA is a cloud based service, an on premise component called “Azure MFA Server” is necessary. This is what allows 3rd party systems like NetScaler Gateway to use the solution. If you don’t use the on premise server then you are limited to only being able to use MFA for Microsoft’s cloud and SaaS services like Office 365 only. In a production environment, you’ll want to have at least 2 of these servers and a high availability configuration plus consider your DR sites. These servers will become critical to your organization. Bill Mathers has an excellent guide available here:
NOTE: DO NOT install MFA Server on a Domain Controller. It is meant to be a standalone component. It will take over ports 389, 636, etc. for LDAP/LDAPS traffic so don’t potentially screw up your DCs which will compete for those ports with MFA server. I have not found any documentation from Microsoft on this but this is just from my personal experience watching network traffic on MFA Server. I wouldn’t risk it.
28. Go ahead and click the “Download” link for MFA Server. It’s pretty small file at 122 MB called MultiFactorAuthenticationServerSetup.exe
29. Run it and it will want to install .NET Framework 2.0.50727 from 2007. Yeah, I’m not kidding. I think that’s some kind of oversight by Microsoft from the legacy PhoneFactor days most likely. If you hit Yes it will open IE and you have to download and install it but the website won’t have it since it’s so old. Easier thing is just to install the .NET Framework 3.5 Features on the server which will include 2.0. Wish the installer would just do it for you.
30. Now we can restart the install. Hit Next. The install will take a bit of time.
31. Hit Finish and the wizard will start
32. Hit Next on the Wizard
33. You’ll get this screen asking for credentials
34. Go back to the Azure MFA Admin portal and click Generate Activation Credentials. Take those credentials and put it into the wizard.
35. You can leave the default MFA group
36. Go head and enable replication between servers for later
37. Hit next under Secure Communication
38. Next it talks about creating a PhoneFactor Admins group
40. Select what we’re going to be using MFA for. In our case since we’re using NetScaler Gateway which isn’t on the list. They don’t list LDAP/LDAPS as one of the options. Just select RADIUS. Even if you don’t use it, you have to configure something from the list to move on. Don’t worry, you can still use LDAP or LDAPS later.
41. Type in your NetScaler NSIP for the RADIUS client IP, make up a shared secret, and you can use the default 1645 and 1812 RADIUS ports.
42. The MFA Server can proxy the authentication request to another RADIUS server or against your Windows domain. In most cases you would use Windows domain.
43. Hit Next on the completion page
44. Hit Finish and let the server reboot
CONFIGURING AZURE MFA SERVER FOR NETSCALER GATEWAY
Now we can finally begin setting up the MFA Server for your NetScaler Gateway. In this example I’m going to use plain old unencrypted LDAP on port 389 for example purposes. Please use LDAPS or RADIUS in a production environment.
45. Open the MFA Server Console after the server comes back up and you should see the Status as Online. This means it’s talking to the PhoneFactor servers in the cloud.
46. Go to the LDAP Authentication section and click the check box to enable it
47. Click Add, enter your NetScaler NSIP, application name as “NetScaler” or whatever you like, and check the box that says “Require Multi-Factor Authentication user match”. Remember LDAP traffic on a NetScaler is over the NSIP, not the SNIP. If you have an HA pair you’ll want to add both NSIPs as clients in here. But if you load balance LDAP vservers on the NetScaler, then you will want to use the SNIP. This in effect forces LDAP authentication traffic to go over the SNIP instead of the NSIP. I have used both in production environments depending on the environment so this will be up to you to decide.
48. Now click Directory Integration, you’ll notice a black down arrow on the icon. You can use Active Directory but I like to do LDAP binds so I know exactly how lookups are happening and I’m so used to doing it with NetScalers. Click the “Use specific LDAP configuration” and hit Edit
49. Type in a Server name, Base DN, Bind username, and Bind password. This is the exact same info you would normally use when creating an LDAP policy on your NetScaler except you type in the Bind username, not the Bind DN. Make sure the Queries and Authentications are both set to “Windows”.
50. When you hit Test hopefully you’ll get a message like this
51. Go to Company Settings > Username Resolution and click “Use LDAP unique identifiers attribute for matching usernames”
52. Now it’s time to add a User. Go to the Users section and hit Import from LDAP:
53. Look how versatile it is. You can select an OU where you keep all your User Accounts or you can run LDAP filters to find your users. By default it’s going to import all users with a Mobile specified in AD so make sure all your users have their numbers specified. If you import users without numbers, the MFA server will import them but their account will be set to disabled by default. You can see below my AD account has a mobile number in it.
54. Woohoo! I’m imported and Enabled. Hit Test at the bottom.
55. You can use Windows Domain or the LDAP Bind you did in the drop-down for testing purposes. Enter your credentials and hit OK.
56. You will see this message saying Azure MFA is calling you. One thing to note, make sure you can communicate outbound on port 443 to the following PhoneFactor servers in the cloud. If they are blocked by your organization by a web filter, authentication will not work:
57. You should start getting a phone call from Azure MFA like this:
58. Answer it and hit the # sign, it will immediately end the call
59. And your login test will show it’s successful
60. Now you need to import your service account that will talk to the MFA server and do LDAP lookups. It won’t have a mobile number associated with it so it will import as Disabled. That is fine. Don’t give this service account a mobile number or it will call that number everytime any user attempts to authenticate on your NetScaler Gateway. Should look like this:
61. Remember the black arrow under Directory Integration? See what happens when you enable LDAP sync under the synchronization tab? It goes green. You don’t need to enable it for this example but I just wanted to make you aware that’s why the black arrow shows down. You should go ahead and enable it and add your domain to sync every 5 minutes if you wish to use this feature.
CONFIGURING NETSCALER GATEWAY TO USE MFA
Now finally we can get to the NetScaler config. It’s pretty simple. Let’s create an LDAP authentication policy for MFA. Again I’m using LDAP on port 389 for this example but in a production environment you need to use LDAPS on port 636 or RADIUS. I’m going to assume you already have a working NetScaler Gateway vserver. We’re going to swap out your regular LDAP authentication policy with a new MFA authentication policy.
62. Navigate to NetScaler Gateway > Policies > Authentication > LDAP then clicks the Servers tab and hit Add
63. Now give it a name and configure it just like you would any other LDAP policy. The only difference is the server IP is going to be the IP of the MFA server. You also need to increase the default timeout value of 3 seconds to something like 120 seconds. This is because the NetScaler Gateway has to wait while the PhoneFactor/MFA call is made and you hit the # sign on your phone. In this example my policy name is ldap_vsrv_js_MFA_389.
64. Create a policy with ns_true, in this example mine is called ldap_pol_js_MFA_389
65. Now bind your new LDAP policy to your NetScaler Gateway vserver.
66. Now open up Putty and SSH into your NetScaler. Let’s do:
67. Now go to your NetScaler Gateway login page and attempt a login. You should see it do the LDAP bind in your AAA log and then wait. You will get a phone call and once you hit the # sign, you will see an accept like this for your user ID:
Congrats! You just setup your NetScaler Gateway with Azure MFA!
68. Ok so what about accounts you want to allow login to your NetScaler Gateawy but they don’t have mobile numbers specified in LDAP yet so can’t use PhoneFactor? Just import them all into MFA but leave them disabled like your NetScaler service account. They’ll be able to login just fine like they always have been able to. This effectively bypasses the phone call and authenticates straight against your AD only. When you’re ready to give them MFA/PhoneFactor, just populate their mobile number in LDAP and verify their account gets updated and is Enabled on the MFA server during the next sync. Bam, now they’re an MFA user!
MODIFYING NETSCALER GATEWAY LOGON PAGE FOR MFA
With NetScaler 11 you have the wonder Portal Themes feature now (Internet High Five to the NetScaler Team!). With NetScaler 11.0 build 62.10 they included the X1 theme we’ve all been waiting for. When you are using MFA, there is no notification to the user that the NetScaler is waiting on them to pick up and hit # before logging in. So you have to let them know somehow to expect that. The Portal Themes does not have the ability to add custom text below the login boxes right now so instead you can modify the top area.
69. Navigate to NetScaler Gateway > Portal Themes > click the X1 theme and hit Add
70. Give it a name and follow the prompts. The thing you want to change is the “Form Title” in the “Login Page” section. Change it from “Please log on” to “Please logon and wait for the phone call” or something along those lines.
71. Bind this new Portal Theme to your NetScaler Gateway vserver
72. Open a browser in private/incognito mode so you aren’t pulling from your local web cache and navigate to your NetScaler Gateway logon page. It should look something like this now.
TROUBLESHOOTING MFA AUTHENTICATION ISSUES
I’ve covered how to troubleshoot NetScaler authentication issues in previous articles:
and now you’ve got this man in the middle called MFA. I showed you above how AAA debug still works but you can’t really see what’s going on once the MFA server and Azure cloud/PhoneFactor servers start talking. Here’s how you can troubleshoot this.
73. On your MFA Server navigate to
C:\Program Files\Multi-Factor Authentication Server\Logs and check out the logs. The one that’s going to tell you exactly what’s happening with each user is the “MultiFactorAuthSvc.log”:
74. Open it up and scroll straight to the bottom for the latest entries. You’ll see everything going on between the MFA Server and the PhoneFactor servers for that user. Hopefully you’ll see a sucess like this:
pfsvc|Pfauth succeeded for user 'CN=xxxxxxxxxxxxxx' (distinguishedName format) from xxx.xxx.xxx.xxx. Call status: SUCCESS_NO_PIN - "Only # Entered".
75. If the user hangs up the phone or doesn’t answer, you’ll see messages like this:
pfsvc|Pfauth failed for user 'xxxxxxxxxxxxxxxxxx'. (distinguishedName format) from xxx.xxx.xxx.xxx. Call status: FAILED_USER_HUNGUP_ON_US - "User Hung Up the Phone".
pfsvc|Pfauth failed for user 'xxxxxxxxxxxxxxxxxx' (distinguishedName format) from xxx.xxx.xxx.xxx. Call status: SUCCESS_NO_PIN_BUT_TIMEOUT - "No Phone Input - Timed Out".
pfsvc|Pfauth failed for user 'xxxxxxxxxxxxxxxxxx' (distinguishedName format) from xxx.xxx.xxx.xxx. Call status: SUCCESS_WITH_PIN_BUT_TIMEOUT - "# Not Pressed After Entry".
AUDITING & REPORTING MFA LOGINS
I’ve talked about the importance of syslog and NetScaler in the past in my previous article:
MFA logs in the cloud and on the MFA server but you also want to consider setting up a dedicated syslog server like Splunk, Kiwi, etc. where all this data goes. This is how you do it:
76. Go to the MFA Server > Logging > Syslog tab and check all the boxes and specify your syslog server (separate multiple syslogs servers them with semi-colons). Leave the facility as 16.
Note, the severity is as follows and cannot be changed:
3 for errors
4 for warnings
6 for all other information messages
77. Additionally on the MFA Admin portal you can run all sorts of reports just like back when it was PhoneFactor.
AZURE MFA HACKING ATTEMPT NOTIFICATIONS
Let’s say you’re at dinner and you get an MFA phone call asking you to authenticate. That means someone has stolen your domain credentials and is attempting to login to the NetScaler Gateway. Instead of hitting the # sign, you can hit 0# (or other keys you define on the MFA server) and submit a FRAUD ALERT to MFA. This immediately blocks your account in Azure AD from any further authentication attempts. It does not lock out the user’s AD credentials themselves so lockout attacks are rendered pretty useless. You can even specify your security team get an email when a FRAUD ALERT is submitted by a user. This is how you do it.
78. Login to the MFA Admin Portal and click Fraud Alert. Add your email distribution groups for Fraud Alert Notifications, Account Lockouts, and One-Time Bypasses.
CONSIDER DEPLOYING THE AZURE MFA USER PORTAL FOR SELF SERVICE
If you want to take your environment a step further, you can deploy the Azure User Portal which is a separate set of IIS servers you will need to deploy. Doesn’t have to be but I would highly recommend keeping those web servers separate and in a DMZ. The UI does look a little dated unfortunately. The User Portal will allow your users a level of self service so they’re not always calling your Help Desk for issues, bypasses, changing security questions, changing their phone number, etc. You can pick what options you want to allow. In this example I’ve taken away a few menu items like One-Time Bypass, Change Language, & OATH Tokens as a test but allowed other features to remain:
You can also allow the users to change their MFA preferences, for example switching from using a phone call to using the Azure MFA Authenticator mobile app. I actually find the mobile app to be even a quicker process to login so users will appreciate the ability to choose:
I hope this article has helped you setup Azure AD Connect, AD Premium, and Azure MFA with NetScaler Gateway. 2 factor or multi-factor authentication is an important part of your business no matter what size company you have. It is absolutely necessary for everyone in my opinion. There’s a lot of steps involved in setting up MFA (78 by my count 🙂 ) but once it is in place you have a really powerful system to authenticate your employees.
As always please leave a comment below if this guide has helped you or if you have any questions.