In this blog post series, we want to give you a sneak peek into how we develop and maintain our applications. We have recently developed an internal service called 4DIdentity. This service acts as our central identity provider for internally used tools like ArgoCD and HashiCorp Vault. We also plan to use it in our future managed service offerings, so stay tuned!

When developing the 4DIdentity service, we leveraged a set of best practices. As a result, the service is entirely managed through Infrastructure as Code (IaC), deployed to all stages using GitLab pipelines, and completely built on serverless infrastructure. Making it straightforward to maintain, low-cost, and highly resilient.

Today, we start the blog post series by discussing our motivation and the use case for our new internal service. Next, we provide a high-level overview of the serverless architecture, including our thoughts on the disaster recovery strategy. Finally, we will look at our monitoring setup for the services our solution is based on.

Use Case

The motivation to build our own internal IdP is mainly driven by our efforts to build managed services for our clients. These managed services will include components that either we or our clients will have to log into. Assuming that most of our customers have their own identity solutions, we need to be independent to access the components we are responsible for.

As a result, we decided to create our own identity management service that can easily integrate with the applications being used for our managed services and leverage existing identity providers.

Architecture

The goal of 4DIdentity is to provide a centralized identity provider (IdP) that is reliable, secure, and easy to maintain. We decided to use Amazon Cognito due to its serverless nature and its capability to be fully managed using IaC. An additional advantage provided by AWS is the per-user pricing, which lets us run the application at a low cost. At the same time, it meets our high availability and service recovery strategy.

Additionally, we take advantage of the user pool feature of the Cognito service. User pools provide various identity management capabilities such as sign-in, sign-up, and identity federation with external IdPs.

Amazon Cognito integrates with several other AWS services, such as CloudFront, Route 53, Simple Email Service (SES), Certificate Manager (ACM), and Web Application Firewall (WAF). The image below shows a simplified architecture overview of the 4DIdentity components.

It looks like you need to configure many different services, but in fact most of these are configured automatically by the Cognito service. Depending on the use case, you may only need a subset of the services. For example, when using a domain provided by AWS, you don’t need the ACM service, and all Route 53 configurations are done by the Cognito service.

Ensuring high availability

At this point, let’s focus on some important availability aspects, that we chose for the 4DIdentity service, as they affect the architecture. In case of a disaster, we rely on the Pilot Light strategy proposed by AWS: all required components are deployed and configured in a primary and a dormant secondary region.

This strategy has the following advantages:

• The failover site is ready to be activated immediately, minimizing the number of activities significantly and thereby reducing the risk of handling mistakes, while being under pressure to recover the service.
• Leveraging IaC reduces the efforts to deploy a copy of the required components in the failover region.
• As the entire solution is serverless and we only pay-per-use, the costs for the failover deployment are minimal.

However, the strategy has its challenges:

• As user pools rely on shared/global services, e.g., Route 53 or SES, the deployment process and region compatibility (especially with regards to SES) needs to be analyzed and adapted carefully.
• Deploying the services in multiple regions also requires proper monitoring in all of those.

The image below displays the architecture of the 4DIdentity service using the Pilot Light DR strategy.

A few things to note:

• If you are using a custom domain for your Cognito user pool, the primary and secondary user pools need separate domains configured, due to Route 53 being a global service.
• SES and Cognito regions can only be combined as predefined by AWS. Hence, we decided to deploy SES in a region that can be used by user pools in the primary and secondary regions. Furthermore, this allows us to also failover the SES service in case of an issue in its primary region.

Monitoring

Although we solely rely on serverless services provided by AWS, we still want to monitor these services and get alerted if any of the services are down or impaired. Amazon EventBridge does the job and runs as a serverless service too.

Amazon EventBridge allows you to configure rules, that report any AWS Health events for all of the services being used by 4DIdentity. Since these rules are regional resources, they need to be deployed into the specific regions that we want to monitor. The rules are configured to send the events to the central event bus in our cloud management account. From the central event bus, the health events are forwarded to SNS topics where an AWS Chatbot picks them up and sends a message to our Slack channel.

The image below displays the above-described architecture.

Once more: all of these services are running serverless, are highly available, and fully maintainable with IaC. In addition, it can be expanded by listening to additional events from CloudWatch, if required, e.g., getting notified whenever someone logs in using Cognito or when a new user signs up.

Conclusion

In this post, we addressed the design goals and its architecture. Furthermore, we outlined how the service is recovered in case of a disaster and how it’s being monitored.

In the second part of this blog post series, we will explain how to deploy all of the infrastructure components using IaC and GitLab pipelines, to fully automate the setup of all environments, including a neat trick to manage ephemeral environments.

Please feel free to reach out to us for any feedback or questions.