Best practices

To ensure system stability it's important that the team handling operations creates routines that handles day to day task such as monitoring, backups etc.

The purpose of this document is to provide a tool to the technician (which can be PhenixID personal, a partner or the customer) for initial best practices when installing and configuring PhenixID products in production environments, to make sure products are properly protected and can be operated smoothly.

Hardware / OS

Server requirements can be found here:
https://document.phenixid.net/searches?utf8=%E2%9C%93&text=title%3A%22server+requirements%22&commit=Search

Installation/configuration

Make sure to only expose the ports and uri’s absolutely needed.
The rest of the services should only be accessible on the internal network.

Checklist:

1. Use port 8443+SSL for Configuration Manager.
    Only allow 8443 from local host and/or dedicated machines. Machine/ip allowance is configured outside PhenixID. Use a network tool such as a firewall. 

2. Use port 9443 for config API, when used
    Only allow 9443 from local host and/or dedicated machines. Machine/ip allowance is configured outside PhenixID. Use a network tool such as a firewall. 

3. Use port 10443 for API authentication, when used
    Only allow 10443 from local host and/or dedicated machines. Machine/ip allowance is configured outside PhenixID. Use a network tool such as a firewall. 

4. Use port 443+SSL for other web interfaces

5. Set /config authentication to LDAP using uid/pwd/otp.

6. Always place a web front / proxy (facing the internet) in front of PhenixID Authentication Services.

7. To allow / disallow certain URI patterns, add rules to the web front / proxy.

8. Complex, repeated config update, many stages environment -> Use orchestration.

9. If orchestration is used, /config authenticator can be removed from HTTP_Authenticators

10. Separate disk partition for PhenixID installation and logs

11. Set complex encryption key

Functionality

1. Always enable MFA Admin application (for lockout- and token management) and assign it to the local http interface (for protection).

2. When signing is used (for tickets etc), upload a specific certificate. Do not use the certificate shipped with the product.

3. Configure SLO from the beginning for federation/oidc flows.

4. When using multiple methods for one idp/op, use one pipe for all federation:
https://support.phenixid.se/sbs/step-by-step-advanced-skolfederation-configuration/

5. Change the redirect for / (defaults to /config). Change from /config to something relevant at the customer. 

Monitoring

1. Monitor system logs for errors and/or deviant  behaviour on a daily basis.

2. Also monitor disk size usage. Disks/partitions running out of disk space will cause system failure.

3. Ensure time synchronization is set up properly. This is crucial.

4. Monitor the connectivity to and from the server. Watch for changes in network setup. Typically closed communication paths between server and user stores (LDAP etc.).

5. Minimize network latency. This is especially important when running in cluster mode.

Logging

Logging is primarily done to event.log and server.log.
Event.log contains server events like startup, deployment, user authentication and more.
Server.log contains system information used when troubleshooting, more information below.
What information is written to the log will depend on the information available in the authenticators and the communication. See example from event.log below.

Recommendations:

1. Log-level should be set to INFO. Change temporarily if needed during troubleshooting. (Set to DEBUG only for packages needed to be debugged)
More information:
https://document.phenixid.net/m/90910/l/1138916-edit-log-settings#debug-for-troubleshooting 
https://document.phenixid.net/m/90910/l/1138916-edit-log-settings#debug-for-specific-packages

2. Set logging retention according to company policy and recommendations.
More information:
https://document.phenixid.net/m/90910/l/1138916-edit-log-settings#file-retention

3. Send event logs to SIEM (if siem exist at customer).
More information:
https://document.phenixid.net/m/90910/l/1138916-edit-log-settings#send-events-to-syslog

Examples from event.log (version 4.0.4 of PAS):

2021-05-20 10:09:27,628 [EVENT] #nbM7pfPmuYxK77gs INFO: 2021-05-20T10:09:27+02:00 PAS1 CEF:0|PhenixID|PAS|4.0.4|EVT_001020|OTP delivery success|2|dst=+461234567890 duser=jdoe phenixIDIdentifier=smsaccount proto=PhenixID<space>SMS
2021-05-20 10:09:36,130 [EVENT] #uzcA4yfqir81cJjp.nbM7pfPmuYxK77gs INFO: 2021-05-20T10:09:36+02:00 PAS1 CEF:0|PhenixID|PAS|4.0.4|EVT_001022|User authentication success with username, password & OTP|2|duser=jdoe phenixIDTraceId=#uzcA4yfqir81cJjp.nbM7pfPmuYxK77gs proto=RADIUS src=192.168.10.234
Click to copy 
2021-05-20 14:20:59,299 [EVENT]  INFO: 2021-05-20T14:20:59+02:00 PAS1 CEF:0|PhenixID|PAS|4.0.4|EVT_001020|OTP delivery success|2|dst=+461234567890 duser=jdoe phenixIDIdentifier=smsaccount proto=PhenixID<space>SMS
2021-05-20 14:21:07,221 [EVENT]  INFO: 2021-05-20T14:21:07+02:00 PAS1 CEF:0|PhenixID|PAS|4.0.4|EVT_001018|Provided OTP was correct|2|duser=jdoe
2021-05-20 14:21:07,377 [EVENT]  INFO: 2021-05-20T14:21:07+02:00 PAS1 CEF:0|PhenixID|PAS|4.0.4|EVT_001022|User authentication success with username, password & OTP|2|destinationServiceName=myapps duser=jdoe phenixIDTraceId=#Z1c4OKhuN8DOHGa3 src=0:0:0:0:0:0:0:1
2021-05-20 14:21:09,498 [EVENT]  INFO: 2021-05-20T14:21:09+02:00 PAS1 CEF:0|PhenixID|PAS|4.0.4|EVT_003105|User authentication success|2|destinationServiceName=myapps duser=jdoe phenixIDTraceId=#URj39fm2zKMSBfNV src=0:0:0:0:0:0:0:1
Click to copy

Backup

Make sure to have a good backup/restore plan for disaster-recovery scenarios.
This plan should be tested and verified continuously.

1. Follow the backup procedures for PAS here:
https://document.phenixid.net/searches?utf8=%E2%9C%93&text=post+AND+installation+AND+backup&commit=Search
Daily backup is also performed for the configuration (phenix-store.json), to the directory /config/backups.

2. Backup of database:
- Standalone installations will have the database in the PAS file structure, so as long as the complete file structure is backed up, database should be as well.
Scheduled backup of the database is also done to the directory /data/backup.

- Clustered installations up until version 3.0 also has the database in the PAS file structure.

- Clustered installations with version 3.2 has external OrientDB database. Backup should be enabled in the file <dbinstallationdir>/config/orientdb-server-config.xml and configured in the file <dbinstallationdir>/config/automatic-backup.json.

- Clustered installations with version 4.x and later use external MySQL/MSSQL database and backup is done by the people responsible for the database.

Cluster

Clustering changed in version 4.x and later, where an external database is used.
Versions before 4.x use the OrientDB database.
General information about clustering can be found here:
https://document.phenixid.net/m/96408/l/1358440-about
Information about operating the cluster is available here:
https://document.phenixid.net/m/96408/l/1358454-operating-the-cluster
https://document.phenixid.net/m/90910/l/1293150-verifying-the-cluster-version-3-2-and-later