Using Azure AD as SSO for Self-hosted Apps?

So I was thinking to use Authentik or Authelia (OAuth 2.0) as self-hosted service to be the main SSO for various other apps... but then I thought since I already have MS Azure AD (newly known As MS Entra ID), would I able to use that instead... any proper guides on how to use that with other open source projects such as Guacamole, Home Assistant, etc...?

Worth mentioning that I have MS 365 Basic/Standard subscription from GoDaddy (not premium MS 365 subscription), so no MS Intune service and the Azure AD is not on-premise (all online via entra.microsoft.com).