One of the most important considerations when moving to a new identity management and authentication system like AWS Cognito or Auth0 or Okta for an existing application is how to manage the migration of the user base into the new system.
Just in case you are asking yourself why do that, please do check out this prior article of mine on creating a Cognito User Pool where I dwell into some of the benefits of using a cloud based identity provider for authentication as opposed to managing this in house.
But assuming at this point you understand the benefits and are on board with this, you do have to consider how you are going to migrate your users from your old user store into the new one. There are some straightforward and simpler approaches that might be perfectly valid depending on the nature of your application and user base – simply asking users to re-register into your new identity platform to continue accessing your services. Straightforward but probably more inconvenient for your users and as I said, this is not necessarily a “bad” approach at all and really…depends. A slightly less inconvenient tack on that is to register your existing user base directly in bulk, and “require” them to set a new password on their first attempt at logging in. This is often required because many secure authentication systems do not give a way for you to access the old passwords of your users, and that is a good thing actually.
The most seamless experience undoubtedly though would be to allow your users to simply login to the new system with their existing credentials – password included. Conceptually, the flow would be like the diagram above, user logs in credentials into the new platform, is authenticated against the existing platform, registered with the new platform and al subsequent logins and credential changes like password resets are now managed entirely by the new platform – all this is what we essentially term “lazy user migration”.
AWS Cognito has a feature to support this exact workflow if you are considering or have already considered Cognito as a solution for your applications identity and authentication management needs – something I have personally done so a few times and am happy to recommend, especially if you use AWS for the other components of your stack such as AppSync. Cognito authentication removes much of the hassle with securing your other AWS based APIs.
In this article I will walk through the process of setting up a Migrate User Lambda Trigger to work with a Cognito Pool and how to lazy migrate users from an existing user base to a brand new Cognito User Pool.
Starting with a sample Cognito Pool without a Migrate User Trigger
Prerequisite – a Cognito User Pool
If you would prefer to do this from the AWS Management Console follow the article that I had linked already above to see the steps to create a brand new Cognito User Pool. I will also provide the way to create it using Cloudformation templates which will have the details right below here, but I will not spend as much time on the context as I do in that linked article. The focus will of course be on implementing the lazy user migration Lambda to seamlessly create your users into the new pool as they login.
This is the cloudformation yaml for a simple Cognito User Pool. Without too much context as warned, this allows users to sign in only, not sign up and requires an email to login but also all allows a preferred user name to be set by the user as an alternative.
Cognito will handle the sending of emails, but for a live Cognito Pool, you will need to have the pool integrated with AWS SES.
SampleUserPool:
Type: AWS::Cognito::UserPool
Properties:
UserPoolName: sample-authentication-pool
AutoVerifiedAttributes:
- email
AdminCreateUserConfig:
AllowAdminCreateUserOnly: true
Policies:
PasswordPolicy:
MinimumLength: 8
RequireLowercase: true
RequireNumbers: true
RequireSymbols: true
RequireUppercase: true
TemporaryPasswordValidityDays: 7
UsernameConfiguration:
CaseSensitive: true
AliasAttributes:
- email
- preferred_username
EmailConfiguration:
EmailSendingAccount: COGNITO_DEFAULT
MfaConfiguration: OFF
Schema:
- AttributeDataType: String
Mutable: true
Name: email
Required: true
- AttributeDataType: String
Mutable: true
Name: preferred_username
Required: falseGoing hand in hand with the User Pool, we will also need at least one App Integration to actually login – this will be for the existing old app users to start using instead.
SampleUserPoolClient:
Type: AWS::Cognito::UserPoolClient
Properties:
UserPoolId: !Ref SampleUserPool
ClientName: existing-app
ExplicitAuthFlows:
- ALLOW_USER_SRP_AUTH
- ALLOW_REFRESH_TOKEN_AUTH
GenerateSecret: true
AllowedOAuthFlowsUserPoolClient: true
AllowedOAuthFlows:
- code
AllowedOAuthScopes:
- email
- openid
EnableTokenRevocation: true
PreventUserExistenceErrors: ENABLED
AccessTokenValidity: 1
IdTokenValidity: 1
RefreshTokenValidity: 30
SupportedIdentityProviders:
- COGNITO
CallbackURLs:
- http://localhostImplementing a Migrate User Lambda Trigger
First lets review the basic flows, contract and expectations that Cognito requires for this Lambda to work.
Cognito Triggers are basically invocation points in the user authentication lifecycle that allow you to implement custom logic to handle cases that may not be possible within Cognito. The Migrate User Lambda Trigger is a clear cut example where Cognito does not know if the user attempting to login is a valid user or not. It is triggered at the point the user enters their login name and password. Cognito supports other triggers at various points in the lifecycle which you can make use of should you need like a post sign up trigger to shoot welcome emails and such.
Specific to the Migrate User Lambda Trigger, there are two “TriggerSources” that would result in Cognito invoking the Lambda handler:
1. UserMigration_Authentication – this is the default use case. One of your users logs into your Cognito Pool for the very first time, Cognito passes the credential details to the Migrate User Lambda for you to authenticate – or reject! More the specifics coming right up in this section.
2. UserMigration_ForgotPassword – this is the necessary case where you have an existing user who has forgotten their password and requesting a reset from Cognito before they have been migrated. Might seem complicated, but we only need to let Cognito know if they are an actual user or not and Cognito will take case of the process of sending them verification codes and such to reset their password directly in Cognito.
The Cognito Migrate User Trigger Event Request
For each of the above trigger source conditions, Cognito sends us an event JSON with both the incoming request details within the “request” key which has all the details like the critical username and password data. Significantly – it also sends us a partially prefilled “response” object in the event which we need to add details to. We simply return the modified event object to the user.
This is what the initial event object looks like when the Lambda is invoked, the important fields that we will need to work with. There will be other data that you can ignore or chose to use depending on additional auditing or validations your system or organization may need, but for the core requirement, we really are going to need only these fields listed below.
{
"triggerSource": "UserMigration_Authentication",
"userName": "logged_in_username",
"request": {
"password": "DO_NOT_LOG_THE_CLEAR_TEXT_PASSWORD!",
...
},
"response": {
...
}
}It is important to NOT log the event in its entirety as you create security risks by logging the password. Simply process it in the Lambda and let Cognito take that off your shoulders going forward.
The Response Cognito is expecting back (for valid authenticated users)
As mentioned, the response object simply needs to be updated by the Migrate User Lambda after you authenticate the user using the credentials sent by Cognito and the entire event returned back to Cognito.
As to what you are updating – firstly the “userAttributes” object needs to be set. This will largely depend on how your User Pool is configured. In our sample pool, we only have email and preferred_username in the schema, so in addition to these two attributes, we need to set a unique username for the user. If the pool is configured to allow only emails as login, then only that would be required. And if there are other attributes required like first and last names, then all those need to be set as well. One attribute to remember – email_verified needs to be set to true since we know this is an authenticated user and we don’t want Cognito re-verifying the email.
In addition to the Cognito Pool attributes, the following are also configurable depending on your use case:
1. finalUserStatus – CONFIRMED or RESET_REQUIRED. If your goal is the same as this article which is to do a seamless lazy user migration – I would go with CONFIRMED to avoid forcing the user to change the password.
2. messageAction – this must be set to suppress to avoid the Cognito Welcome Email from going out. Again, since we are attempting a seamless migration, it makes no sense to send a welcome email to the user as we all get enough email notifications already most likely. I would set this to SUPRESS.
3. desiredDeliveryMediums – can be ignored if suppressed. The default is EMAIL.
4. forceAliasCreation – depends on how your user pool and existing user base is created. If you don’t have user aliases and such with the same email, this also you can ignore.
So for our hypothetically successful login, we would update the request object in the Migrate User Event with the following attributes like so:
"response": {
"userAttributes": {
"username": "A_UNIQUE_USERNAME_LIKE_A_UUID",
"email": "the_email",
"preferred_username": "i_am_optional",
"email_verified": true
},
"finalUserStatus": "CONFIRMED",
"messageAction": "SUPPRESS",
...
}The forgot password condition
When you get a forgot password case, obviously, do not attempt to authenticate the user as the password will not be there in the request, just do a user existence check.
As far as how you respond to Cognito assuming the user does exist, almost the same as above simply do not set the finalUserStatus attribute.
"response": {
"userAttributes": {
"username": "FORGOT_PASSWORD_USER_CASE",
"email": "the_email",
"preferred_username": "i_am_still_optional",
"email_verified": true
},
"messageAction": "SUPPRESS",
...
}Cognito will handle the email verification and stuff, I plan to cover some more details on that in future posts.
Handling invalid authentication cases
Return anything other than the event will fail the authentication in the Cognito prompt. Do the same even for user existence errors. The Cognito login prompt allows for a reset attempt.
Coding the Migrate User Lambda – In Python
Here is a sample Lambda – I am using Python, but all you really need to do is modify the JSON event as described in the previous section in the coding language poison of your choice.
import os
import uuid
import requests
old_authentication_api = os.environ["OLD_AUTHENTICATION_API"]
def lazy_user_migration_lambda_event(event, context):
try:
trigger_source = event["triggerSource"]
username = event["userName"]
if trigger_source == "UserMigration_Authentication":
password = event["request"]["password"]
json_request = {
"username": username,
"password": password
}
response = requests.post(old_authentication_api, json=json_request)
resp = response.json()
is_authenticated = resp["is_authenticated"]
if is_authenticated:
event["response"]["userAttributes"] = {
"username": str(uuid.uuid4()),
"preferred_username": str(uuid.uuid4()),
"email": username,
"email_verified": True
}
event["response"]["finalUserStatus"] = "CONFIRMED"
event["response"]["messageAction"] = "SUPPRESS"
return event
else:
return f"incorrect password or user"
elif trigger_source == "UserMigration_ForgotPassword":
json_request = {
"username": username
}
response = requests.post(old_authentication_api, json=json_request)
resp = response.json()
is_existing_user = resp["is_existing_user"]
if is_existing_user:
event["response"]["userAttributes"] = {
"username": str(uuid.uuid4()),
"preferred_username": str(uuid.uuid4()),
"email": username,
"email_verified": True
}
event["response"]["messageAction"] = "SUPPRESS"
return event
else:
return f"no such user"
except Exception as e:
return f"Uh Oh...Cognito will fail to create the user"
Integrating the Lazy User Migration Lambda with the Cognito Pool
AWS Management Console Approach Steps
See below for the Cloudformation template updates.
First upload the above function as a Lambda into your AWS account. Then via the Console simple navigate to the User Pool Properties tab in Cognito and select Add Lambda Trigger …
And in the second prompt choose the type as Sign-up and the sub type as Migrate User and finally the Lambda you created and uploaded.
Cloudformation YAML Updates
With Cloudformation, we can simply make all of this part of the same stack with all the benefits of replicating the template in other regions or accounts.
First, lets add our Lambda definition.
LazyUserMigration:
Type: AWS::Serverless::Function
Properties:
Architectures:
- arm64
Runtime: python3.9
CodeUri: ./lazy-user-migration/
Handler: lazy-user-migration.lazy_user_migration_lambda_event
FunctionName: lazy-user-migration
Environment:
Variables:
OLD_AUTHENTICATION_API: "https://old-api-location/authenticate"Next we need to add an IAM permission to allow Cognito to invoke the Lambda
LazyUserMigrationPermission:
Type: AWS::Lambda::Permission
DependsOn:
- LazyUserMigration
- SampleUserPool
Properties:
FunctionName: !GetAtt LazyUserMigration.Arn
Action: lambda:InvokeFunction
Principal: cognito-idp.amazonaws.com
SourceArn: !GetAtt SampleUserPool.ArnFinally, configure the User Pool with the trigger definition and associated Lambda function. I am just showing the extra property “LambdaConfig” added to the Pool defined above here.
LambdaConfig:
UserMigration: !GetAtt LazyUserMigration.ArnAnd that should be all set for migrating your users to the pool. Try this out, with the caveat that the most common problems people encounter are because of the missing IAM permission for Cognito to invoke your Lambda. First ensure Cognito is able to actually invoke your Lambda before testing the details with your old authentication system.