401 Unauthorized Error on Solution UI After Org Provisioning

Problem

After a CloudSense org has been provisioned or reprovisioned, users encounter a 401 Unauthorized error when attempting to access the Solution UI or perform operations like submitting baskets or calculating totals.

Symptoms:
- 401 Unauthorized error when accessing Solution Console or Solution Manager
- Error occurs immediately after org provisioning or certificate renewal
- Basket submission fails with authentication error
- Calculate Totals operation returns 401 error
- Error message in browser console: POST https://cs-messaging-dispatcher-eu-app.cloudsense.com/sms/baskets/components?action=get 401 (Unauthorized)
- Operations that worked before provisioning now fail with authentication errors

Additional symptoms if endpoints are outdated:
- Errors referencing old Heroku URLs (e.g., cs-cfg-service-sandbox-sp20.herokuapp.com)
- Timeout errors or connection failures
- Batch engine not running or stuck

Root Cause

This issue typically has two related causes that often occur together after org provisioning:

Cause 1: Org Not Active in Security Service

During the provisioning process, the org's active flag in the CloudSense security service may not be set correctly, or it may be set to false. This prevents the org from authenticating with CloudSense backend services, resulting in 401 Unauthorized errors.

The security service maintains an allowlist of active orgs that are authorized to make API calls to CloudSense services (Configuration Service, Messaging Dispatcher, etc.). If the org is not marked as active, all authentication attempts will fail.

Cause 2: Outdated Heroku Endpoints Not Updated to AWS

CloudSense migrated from Heroku to AWS infrastructure. After provisioning, some orgs may still have old Heroku endpoint URLs configured in their Salesforce settings. These outdated endpoints either no longer exist or are not properly routed, causing connection failures and authentication errors.

Old Heroku endpoint example:
- cs-cfg-service-sandbox-sp20.herokuapp.com

New AWS endpoint example:
- cs-cfg-service-sp20-sandbox-eu-app.cloudsense.com

Resolution Steps

Step 1: Verify the Error

1. Open the browser developer console (F12 or right-click > Inspect)
2. Navigate to the Console tab
3. Attempt the operation that is failing (e.g., open Solution Console, click Calculate Totals)
4. Look for 401 Unauthorized errors in the console output
5. Note the endpoint URL in the error message (e.g., cs-messaging-dispatcher-eu-app.cloudsense.com)

Example error:

POST https://cs-messaging-dispatcher-eu-app.cloudsense.com/sms/baskets/components?action=get 401 (Unauthorized)
Error: Request failed with status code 401

Step 2: Contact CloudSense Support to Activate the Org

The org's active flag in the security service can only be updated by CloudSense Support or Infrastructure team.

1. Create a support ticket with the following information:
2. Subject: "401 Unauthorized Error After Org Provisioning"
3. Org ID: [Your 15-character Salesforce Org ID]
4. Environment: [Production/Sandbox]
5. Error message: "401 Unauthorized when accessing Solution UI"

6. Provisioning ticket reference: [Reference the original provisioning ticket number]

7. Request the support team to verify and update the org's active flag in the security service

8. Wait for confirmation that the flag has been updated

Step 3: Update Endpoints from Heroku to AWS

While waiting for the security service update, check and update your endpoint configurations:

1. Navigate to Custom Settings in Salesforce:
2. Go to Setup > Custom Settings
3. Click Manage next to SM Options (Solution Management Options)

4. Click Edit for your user or org-level setting

5. Update the Configurator Service endpoint:

6. Find the cssmgnt__endpoint__c field
7. Old value (Heroku): https://cs-cfg-service-sandbox-sp20.herokuapp.com
8. New value (AWS): https://cs-cfg-service-sp20-sandbox-eu-app.cloudsense.com
9. Update the URL to the AWS endpoint
10. Click Save

Note: The exact AWS endpoint depends on your region and environment:
- EU Sandbox: cs-cfg-service-sp20-sandbox-eu-app.cloudsense.com
- EU Production: cs-cfg-service-sp20-prod-eu-app.cloudsense.com
- APAC Sandbox: cs-cfg-service-sp20-sandbox-apac-app.cloudsense.com
- APAC Production: cs-cfg-service-sp20-prod-apac-app.cloudsense.com

1. Update Remote Site Settings:
2. Go to Setup > Remote Site Settings
3. Find the Configurator Service remote site
4. Click Edit
5. Update the Remote Site URL to the new AWS endpoint

6. Click Save

7. Update Messaging Dispatcher endpoint (if applicable):

8. In SM Options custom setting, find the messaging dispatcher endpoint field
9. Old value: https://cs-messaging-dispatcher-sandbox-eu.herokuapp.com
10. New value: https://cs-messaging-dispatcher-eu-app.cloudsense.com
11. Update and save

Step 4: Enable the Batch Engine

After updating endpoints, ensure the batch engine is running:

1. Navigate to the Admin Console:
2. Add /apex/csam__AdminConsole to your Salesforce URL

3. Example: https://yourinstance.salesforce.com/apex/csam__AdminConsole

4. Check the batch engine status:

5. If the engine is stopped, click Start Engine

6. Verify the status changes to "Running"

7. Verify no stuck messages:

8. Check for any alerts about stuck messages in RabbitMQ queues
9. If messages are stuck, they should start processing once the engine is running and endpoints are updated

Step 5: Clear Cache and Test

1. Clear the Configurator cache:
2. Navigate to SM Options custom setting
3. Use the cache clearing functionality (or run the clear cache script provided by your implementation team)

4. Or navigate to /apex/cssmgnt__ClearCache in your Salesforce org

5. Clear your browser cache:

6. Press Ctrl+Shift+Delete (Windows) or Cmd+Shift+Delete (Mac)
7. Select "Cached images and files"

8. Click "Clear data"

9. Test the operation:

10. Open the Solution Console
11. Try accessing a basket
12. Click Calculate Totals

13. Verify the operation completes without 401 errors

14. Check the browser console:

15. Open developer tools (F12)
16. Verify no 401 errors appear
17. Verify API calls return 200 OK status

Step 6: Verify Certificate Configuration

If the issue persists after Steps 2-5, verify the certificate configuration:

1. Check the certificate name in SM Options custom setting:
2. Field: cssmgnt__certificate_name__c

3. Verify it matches the certificate name used in the provisioning request

4. Verify the certificate exists in Salesforce:

5. Go to Setup > Certificate and Key Management
6. Find the certificate with the name from Step 1
7. Verify it is not expired

8. Verify it is the correct certificate (check the Common Name and expiration date)

9. If the certificate is missing or incorrect:

10. Contact CloudSense Support to obtain the correct certificate
11. Import the certificate into Salesforce
12. Update the certificate_name__c field in SM Options to match

Verification

After completing the resolution steps:

1. Test Solution Console access:
2. Navigate to a basket
3. Verify the Solution Console loads without errors

4. Verify all solutions and configurations are visible

5. Test Calculate Totals:

6. Click Calculate Totals on a basket
7. Verify the operation completes successfully

8. Check browser console for any 401 errors (should be none)

9. Test Basket Submission:

10. Submit a test basket
11. Verify the submission completes without authentication errors

12. Check that the basket stage updates correctly

13. Verify batch engine:

14. Navigate to /apex/csam__AdminConsole
15. Confirm the batch engine is running
16. Check for any error messages or stuck jobs

Prevention

To prevent this issue in future provisioning activities:

• Provisioning Checklist: Create a post-provisioning checklist that includes verifying org active status and endpoint configurations
• Endpoint Documentation: Maintain a document with current AWS endpoint URLs for all regions and environments
• Certificate Management: Track certificate expiration dates and renewal schedules
• Automated Testing: After provisioning, run automated tests to verify Solution Console access and basic operations
• Communication: Ensure provisioning team communicates with support team to verify org activation
• Endpoint Validation: Before provisioning, verify that all endpoint configurations in the org are up-to-date
• Post-Provisioning Verification: Always test Solution Console access immediately after provisioning is complete

Important Notes

• The org active flag can only be updated by CloudSense Support or Infrastructure team - customers cannot change this themselves
• Endpoint updates from Heroku to AWS are required for all orgs as part of the infrastructure migration
• The 401 error will persist until BOTH the security service flag is updated AND endpoints are correct
• Certificate issues can also cause 401 errors, but they typically present with different error messages (e.g., "certificate validation failed")
• If you recently renewed your certificate, ensure the new certificate name is updated in SM Options
• The security service tracks orgs by their 15-character Org ID (not the 18-character ID)
• After updating endpoints, always clear cache and restart the batch engine to ensure changes take effect
• If the issue persists after all steps, check CloudWatch logs or contact support for deeper investigation
• Some orgs may have multiple endpoint configurations (EU, APAC, sandbox, production) - ensure all are updated