Local Admin Lockouts in vIDM Caused by Failed Login Attempts to NSX

I recently had an issue in VMware Identity Manager (vIDM) that was interesting enough to merit a KB being released. We first noticed the issue when Aria Suite Lifecycle Manager was failing to upgrade Aria Operations due to authentication issues with local admin for vIDM. We reset the password following KB 322720 and verified successful logins from the web UI. However, the account would get locked out very shortly afterwards.

We could see the failed login attempts in the vIDM reports dashboard. But frustratingly enough, the events did not list a source IP address.

And for some reason, these periodic failed attempts were the only ones that did not list a source IP. When manually logging into vIDM directly, the source IP is listed for both successful and unsuccessful attempts.

Eventually, we were able to identify the source of these attempts as NSX Manager through a combination of monitoring the session table for the load balancer and applying a firewall rule to block traffic from NSX to vIDM. Temporarily disabling the vIDM integration in NSX also would have likely worked as a way to confirm. But despite identifying the source of the failed attempts, the actual root cause still eluded us. There was no place in NSX to configure the local admin password for vIDM. And reconfiguring the vIDM integration between NSX and vIDM didn’t resolve the issue. So what exactly was causing NSX Manager to try authenticating to vIDM as admin?

After a bit of time of monitoring and troubleshooting, we were eventually able to identify the root cause. It was from Aria Operations attempting to log into NSX Manager with incorrect credentials for NSX’s local admin. The login attempts began failing after the password was rotated in SDDC Manager. This was causing vIDM’s local admin to get locked out.

You might be asking, “How does an authentication attempt to NSX local admin cause vIDM local admin to get locked out?” Well, NSX has a quality-of-life feature that allows you to log into a vIDM-based account directly from the NSX login page. So what was happening here was:

1. Aria Operations attempts to log into NSX Manager with incorrect credentials for admin. This is supposed to be for NSX’s local admin.
2. NSX attempts to authenticate this attempt to its own local admin account, which fails.
3. Because it failed, NSX passes the login attempt for admin to vIDM.
4. vIDM receives the login attempt and interprets it as one for its own local admin. This attempt also fails.
5. Repeat this process until vIDM’s admin is locked out.

The issue disappeared after we fixed the integration between Aria Operations and NSX. We couldn’t find the root cause for a while because we were specifically looking for login attempts to vIDM local admin. But it actually ended up being failed login attempts to NSX Manager, which it was passing along to vIDM.

As an experiment, try logging into NSX using the password for vIDM’s local admin. You’ll see that you successfully authenticate, albeit with an access denied error because vIDM admin hasn’t been provided a role in NSX.

For funsies, let’s add an NSX role to vIDM local admin and try logging in again. NSX interprets vIDM local admin as admin@System Domain, which is the default local domain in vIDM. You can see that NSX fully allows this login attempt, even though it’s the same username as its own local admin.

This was a good lesson in troubleshooting. We were so focused on vIDM that we didn’t consider the entire environment. Sometimes, taking a step back and confirming everything else in your environment is correct can be a good step forward. A seemingly unrelated issue could be the solution you need.