Emacsen's Blog

OIDC Didn't Work For Me (and lessons learned along the way)

I tried to solve a common web problem using a commodity technology. What happened was nearly two years of headache. Let me help you possibly avoid the same mistake. This is a somewhat technical article, though I've done my best to write it in a casual style.

Introduction

In 2022, I launched a Mastodon instance with the vision of creating a unified experience across several applications under the same banner. I wanted all the sites to look and act the same, and I wanted users to have a unified login experience too. Essentially, I needed a Single Sign-On (SSO) solution to unify logins across these applications and I wanted to do it without needing to reinvent the wheel or heavily modify Mastodon. After exploring my options, I chose OIDC (OpenID Connect). Sadly, my experience with OIDC was far from smooth. Here's why OIDC wasn't the answer for me, and what I learned along the way.

The Need for Single Sign-On

As a brand, I wanted to provide a unified experience across all applications. This meant having a common set of rules, a single visual theme, and a single set of credentials. I think most of us have had the experience of when a company requires we create a new login for some aspect of their product. For example, it's frustrating when you are on a product's website and then if you want to join the forums to get product support, you have to register again. When you register, you have to remember what email address you used the first time, see if the same username is available, and make a decision around credentials. It's a lot of friction and it feels unprofessional.

I wanted to avoid that, and OIDC seemed like a practical solution. What I learned was that OIDC does solve the login problem, but that a unified user experience is about a lot more than signing in.

Using Off the Shelf Software

My experience has been with Mastodon, but my gripe around OIDC is not Mastodon specific. The Mastodon team has been overall positive and encouraging around the issue of OIDC client support, and my team and Mastodon have maintained a positive relationship throughout this process. In my mind, Mastodon is simply a stand-in for almost any web software one might find. If anything, Mastodon is exemplary in that they're FLOSS, so the problem can be solved, and that they're open to fixing problems as they come along.

The problems that we encountered with OIDC are likely to be the same with virtually any off-the-shelf software one might find.

Exploring SSO

In my decision to adopt OIDC, I looked at multiple technologies as possible solutions, and OIDC was the most promising. Specifically, I examined Kerberos/SAML, LDAP, and OIDC.

Kerberos/SAML

Kerberos and SAML are well-understood technologies that have been in use for decades by large organizations who want a unified login experience. Kerberos is commonly associated with Unix systems, but Kerberos is used in Active Directory as well.

While Kerberos does an excellent job, it's meant for a single organization such as a university, large corporation, or government entity. It's not really meant for inter-organizational authentication.

SAML is a protocol that is used to describe information about a user and login information. It can be used with Kerberos, or it can be used separately. In many ways, SAML can be considered a precursor to OIDC. Unfortunately, SAML is also famously complex and difficult to get right.

While SAML is still supported by many organizations, many/most of them would strongly prefer application developers use OIDC instead.

LDAP

By far the most common recommendation people gave me around SSO was "Use LDAP". Not surprisingly, there was a negative correlation between people's use of LDAP and their recommendation of LDAP.

LDAP is fundamentally a database technology and is designed to store information about objects. While the most common use of LDAP is to store information about users, it can be used to store information about almost anything through what are effectively published database schemas that can be re-used.

LDAP is not especially interested in the idea of "users", or "logins". When applications use LDAP, it just means that they store information inside LDAP as a data store, just as they might store them in Postgres, or MariaDB.

For my purposes, this would mean that while it would be possible to have unified credentials across applications, users would still need to log in to each application separately.

To use an analogy, it would be as if you had one set of Google credentials, but you had to log into GMail and Google Calendar separately.

LDAP also suffers because its most popular FLOSS implementation is OpenLDAP, which is non-trivial to configure. While that experience of LDAP is being improved with newer software such as LLDAP, LDAP overall did not fit my needs.

OIDC

As an aside, before I go into OIDC, I'm sure that some OIDC folks will be reading this post, so I want to say up front that I've made a deliberate choice to use some jargon in their common usage, rather than in the way they're used in the identify server space. Specifically, here's a quick rundown of the terms I'm using and their formal OIDC equivalents:

Common Term OIDC Vocabulary
Users Resource Owners
Applications Clients
OIDC Server Identity Provider
User Information Claims

OIDC, formally known as OpenID Connect, checked the boxes on all my criteria. OIDC is a Single-Sign On technology that largely does the same thing as SAML, but is far simpler and more widely adopted on the web.

OIDC allows an application to register with an OIDC server and then a user can log into the application using the OIDC server. If you haven't done this directly, you've likely seen buttons on the web saying "Log in with Google/Facebook/Twitter/Github", or similar. These are all using OIDC.

Experiences with OIDC

The biggest reason for me to choose OIDC was its relatively universal support in various applications, including Mastodon. OIDC feels like a technology one can simply slot in, and it's true that if the application simply delegates authorization to OIDC, that integration is fairly easy.

Unfortunately this ease of integration can be a red herring, making a developer think that OIDC integration is complete when it's not, especially when it comes to wanting to be your own OIDC provider.

Registration Challenges

The way applications view OIDC is typically that the user already has an existing account that they're using to log in. It's a generally safe bet that a user has an existing Google account. But if you're the OIDC server and a user is coming to your application to create an account, there's a bit of a challenge.

Instead of a simple "Log in with" button, we had to remove the existing OIDC provider login options, ie no logging in with Google or Twitter. Then we had to create a button for users to register, something that isn't typical for most OIDC clients.

Now we were now modifying the application. We hoped that this would be the last thing we needed to modify, but we were wrong.

OIDC and Existing OAuth Endpoints

OIDC is based on another protocol called OAuth2 that is used to allow third-party applications to connect to a service on behalf of a user. This is the mechanism by which a Mastodon client can log into the service for you and read/write toots without you needing to supply the client with your actual password.

Sadly the authorization flow of being both an OIDC client and an OAuth2 provider had hiccups, and we had to do some engineering to fix this.

User Information Management Challenges

Perhaps the biggest challenge we faced with OIDC integration was handling of user profiles/user information. To understand this, we need to understand a bit more of how OIDC works, so we'll have to take a brief detour.

As mentioned in the previous section, OIDC is based on another protocol called OAuth2, which is used to allow applications to work on behalf of a user. You've probably seen or used one of these OAuth2 interactions yourself if you've ever had Google, GitHub, or a Mastodon server say "This application would like to take some action on your behalf. Do you allow it to do so?"

This question is part of the OAuth2 consent mechanism.

OIDC takes this existing flow and adds some information from the OIDC server to the application about the user, such as their display name, their email address, profile picture, etc.

The application server can then populate its own internal database with this information. It makes sense that each application has its own internal representation of the user and their state. If we were building a video game leader-board, there's no reason why another application would need or want high scores in its user database. That said, having user information in both the application and the OIDC server can lead to some problems.

Let's imagine that we have a user who decides to update their name, or their profile photo. If they do this inside the application, OIDC does not have a built-in mechanism to allow applications to update information about a user, so that information may not be propagated back to the OIDC server, which also means that other applications that are connected to the same account may not get those changes either.

An obvious solution would seem to be that instead of asking a user to update their information inside the application, send them back to the OIDC server to update their profile there. That solution falls down in a few ways. Firstly, it's a clumsy interface that demands that some user profile information/preferences are inside the application and others are accessed by a button or hyperlink. More than that though, if the application wasn't designed with OIDC in mind, it may not be possible to separate out the user preferences which OIDC knows about from those it doesn't, meaning that someone has to manually separate out that part of the application interface.

Worse yet, even if a user does update their information on the OIDC server, there's no guarantee that this information will flow from the OIDC servers to the applications. While it's best practice for applications to update their user information on login, this is not a requirement, and even if it were a requirement, the act of going to a user's profile would not in itself necessarily represent an update of user information to the application.

Login, Logout, and Revocation Challenges

Normally an application handles its own login and logout process, but one of the benefits of OIDC is that these are supposed to be handled by the OIDC server. Moreover, with OIDC you're supposed to get an added benefit of being able to revoke authentication across the board. In other words, when a user is frozen or deleted, their access to all the applications is revoked too. Sadly, this didn't work as expected for us, and I suspect it would be the same for other applications that aren't built with OIDC in mind.

Login worked as expected. Users could log in through the OIDC server and not have a problem. Registration also worked as expected: When Mastodon saw a new user, it created a user entry for them in its database. Unfortunately, other aspects of OIDC did not work out of the box.

Logout seems like a no-brainer. When you're writing your own application, logout just means that you delete the session token from the web browser, and that forces the user to log in again. The problem is that if you use OIDC and you aren't careful, logout doesn't work. The application will delete its session token, but the OIDC token is still there, so if the user refreshes the website, they'll be logged back in. This is a potential serious security issue, other than the fact that almost no one logs out of websites. OIDC provides a logout endpoint that applications are supposed to use, but this is an easy detail to miss.

In the previous section, I discussed some challenges around updating user information.

In addition to login and logout, the issue of revocation is potentially serious. Ideally, if a user needs to be removed from the system, they can simply be deleted from the OIDC server and their access to the various applications would be cut off.

The OIDC way to handle this is that users are given an access token and a second token called a refresh token. The access token isn't meant to last very long, and then the web browser is meant to request a new access token based on the refresh token. If the user is deleted, then they won't get a new access token when it comes time to refresh.

The problem is that many applications have their own application session tokens, and those can last much longer or not have an expiration at all. If that's the case, the application will never contact the OIDC server and a user may have access indefinitely even if the OIDC server has deleted their account or revoked their access.

We ultimately had to add on OIDC logout support to Mastodon to address the security issue of logout not actually logging users out. Fortunately the only seemed to impact one user directly, but it was a security issue.

In the previous section, I mentioned that user updates might not happen. According to the specification, an application could get updates about a user during the login session, or even possibly at token renewal. Sadly, this best practice isn't universal. Even if it were, we'd still have the other problems with user updates.

Poor OIDC Implementation and Overlapping Functionality

The previously mentioned issues around OIDC implementation involved missing functionality, but we've encountered worse. Mastodon provides authentication and login preferences inside the application that are ignored using OIDC. For example, to prevent spammers, Mastodon has a mode that optionally requires a manual approval process for new accounts. This is to prevent spammers. This functionality is entirely bypassed when using OIDC, but that isn't mentioned anywhere in the interface.

In addition, Mastodon allows users to optionally set up 2FA from within the application, but the 2FA is ignored when using OIDC. This makes sense, as the OIDC server should be providing the 2FA, but this can easily lead to confusion as users aren't sure which 2FA, if any, are in use.

Lessons Learned and Looking Towards the Future

While all the examples presented in this post are about Mastodon, I don't believe this is Mastodon specific problem. It would plague any sufficiently complex off-the-shelf software.

The biggest lesson for me was that OIDC support is not as simple and straightforward as it seems. We had to spend significant engineering time to address. Along the way, we learned that OIDC wasn't really the turnkey SSO solution we'd been looking for.

I think that with a few changes, OIDC could be a great deal more useful for situations like ours. Since this section is targeted at OIDC folks, I'm going to switch to OIDC jargon for a moment. There's no reason why there couldn't be a standard or de-facto standard for applications to update claims on behalf of the resource owner. While not every client would need this functionality, it clearly has uses. Another suggestion is to allow the client to register web hooks to the Identity Provider where updates like revocation could be made without needing to effectively poll for them. Making these two changes would make OIDC far better for use cases like mine.

A common motivation for self-hosting is to get away from control by a large third party, and OIDC's most common use is delegating authentication to a large third party, so it should not be a surprise that OIDC adoption is poor in this space. That said, we've shown that poor OIDC implementation can have numerous implications, including security implications that should be considered.

For us, the decision has been made to migrate our authentication off OIDC. This will cause some user pain in the short term, but we feel it's the right decision. Had we not used OIDC in the first place, we'd have saved dozens of engineering hours and saved ourselves and our users an enormous amount of hassle.

I believe that technologies that provide identity services will be important if we want a world with less centralized control by huge corporations, but that OIDC isn't quite there yet. I look forward to seeing it evolve, or another technology take its place in the future.

Special Thanks

Special thanks for Joris (CSDUMMI) for his incredibly hard work in tracking down and resolving most of the OIDC issues on our server, as well as his tireless work getting his patches accepted upstream.

misc