Deployment Outcome


  1. Home    /
  2. iForm Installation Guide    /
  3. Deployment Outcome
Back To TOC

Deployment Outcome

In this section of the Guide, we will cover the following topics:

  1. Key-Security considerations and Encryption
    1. Reviewing Applicatio and System Logs in CloudWatch
  2. How to Operate the Solution
    1. Monitoring
      1. RDS Monitoring
      2. EC2 Monitoring
      3. Network Monitoring
    2. Basic Administration Tasks
      1. Establish SSH connection
      2. Software patches and Logs rotation
      3. SSL/TLS Renewals
      4. Secrets Rotation
    3. Recommendations for Max. Uptime and Reliability
      1. Health Checks and CloudWatch Alarms for EC2/RDS/Network
  3. Backup and Recovery
  4. Operational Maintenance
    1. Checking for AWS Service Limits
    2. Horizontal Scaling (Up and Down)
    3. Expenditure and Usage Awareness
    4. AWS Budgets and Alerts
  5. Emergency Maintenance
    1. Handling Fault Conditions

1. Key-Security considerations and Encryption

The iForm Deployment Solution follows one data protection approach by encrypting all user data and information by default.

Encrypted RDS Database

img

TLS/SSL Transport between Application and RDS endpoint

img

Encrypted Amazon EFS Datastore

img

Encrypted Amazon EBS Volumes

img

1.1. Reviewing Application and System Logs in CloudWatch

iForm App posts application and system logs to Amazon CloudWatch. That allows reviewing logs from a single place. Also, it keeps logs in one durable place without the ability for logs tampering. Furthermore, it provides the ability to create CloudWatch Alarms and notify staff about application usage and cybersecurity incidents.

iForm Logs list

img

Let’s review the “iForm-Application-Logs” log stream.

img

iForm App has built-in security mechanics to detect anomalies. One of the anomalies you can see on the screen above. Mark [1] reports with an exception that a new domain creation is detected. That should happen only for new installation (obviously, that is what we have done - we have just deployed our new version of the iForm).

Mark [2] reports that the system has sent an email about that incident to the admin user. Mark [3] shows generic log output for typical application usage. As you can see, someone tried to reset their password. Mark [4] displays individual log output, which you can use to trace specific user application use and what pages the user opened during their session. It tracks the date and time of the request, user IP address, traceable request UUID, HTTP method and path, and some other essential information, such as URL params.

2. How to Operate the Solution

2.1. Monitoring

iForm Solution intensively logs all application and system-related information to Amazon CloudWatch. That allows building complex monitoring solutions within Amazon CloudWatch. Please review the logs regularly.

2.1.1. RDS Monitoring

From the RDS Monitoring tab, you can inspect various metrics of your RDS database usage and performance. Based on readings, you should consider database upgrades when necessary.

img

2.1.2. EC2 Monitoring

From the EC2 Monitoring tab, you can inspect various metrics of your EC2 instance usage and performance. Based on readings, you should consider a server upgrade when necessary.

img

Please note that Health Checks for EC2 Target Group are enabled and configured automatically for your Elastic Application Load Balancer. Health Checks of your EC2 instances are also enabled automatically.

img

2.1.3. Network Monitoring

Using Amazon CloudWatch, various ways to monitor the usage of your services are available to you. For example, you can review Nginx Proxy Server logs for activity with your Application server before it hits Puma Application Server.

img

2.2. Basic Administration Tasks

2.2.1. Establishing SSH connection

To SSH into your Application EC2 Server, you should use your Bastion Server as a proxy. To do that, first, make sure you have access to the SSH keys you generated during deployment. Then in your ~/.ssh/config file, add something like the following:

Host <BASTION_HOST_IP>
    User ec2-user
    IdentityFile <PATH_TO_YOUR_IFORM_TF>/ssh-keys/ssh-public-key-region-key-pair

Host 10.0.*
    User ec2-user
    IdentityFile <PATH_TO_YOUR_IFORM_TF>/ssh-keys/ssh-key-region-key-pair
    ProxyCommand ssh ec2-user@<BASTION_HOST_IP> -W %h:%p

Please replace <PATH_TO_YOUR_IFORM_TF> and <BASTION_HOST_IP> with real values. Please note that the key ssh-public-key-region-key-pair is used for the Bastion Server and ssh-key-region-key-pair key for the Application Server.

Once you have ssh configuration ready, you should be able to SSH into your Application Server by running the following command:

ssh ec2-user@<YOUR_IFORM_APP_SERVER_PRIVATE_IP>

You may need to type “yes” for each server on ssh prompt. Please note, your <YOUR_IFORM_APP_SERVER_PRIVATE_IP> is an IP that starts with 10.0..

2.2.2. Software patches

To install system updates, please ssh into your Application or Bastion Server and run:

sudo yum -y update

2.2.3. SSL/TLS Renewals

This Solution assumes you use Amazon Certificate Manager, which automatically renews SSL/TLS certificate for you.

2.2.4. Secrets Rotation

The iForm App keeps sensitive credentials in the .env file on a physically encrypted drive. This file is located in the root directory of the application. To rotate your secrets, such as SES or RDS credentials, please update your .env file and restart your Puma Application Server by running the following command:

sudo systemctl restart puma

2.3. Recommendations for Max. Uptime and Availability

2.3.1. Health Checks and CloudWatch Alarms for EC2/RDS/Network

We strongly recommend enabling multi-az to provide fault-tolerance for your Amazon RDS database instance. Please review the appropriate section under Deployment Section.

You can use Amazon EC2 Health Checks to notify your staff about the application’s faulty state. We recommend adding CloudWatch Alarms for your EC2 and RDS instances by choosing appropriate metrics to inform your team. For example, consider adding an Alarm for EC2 instance when CPU or RAM usage goes above 80%. Once your server reaches that usage level - you can vertically scale it by migrating to a larger instance type. iForm does not have auto-scaling capabilities yet.

To scale iForm Application Server, please review Amazon Elastic Compute Cloud User Guide.

Please follow Amazon Elastic Compute Cloud User Guide to create CloudWatch Alarms.

You can also add CloudWatch metrics to detect anomalies such as extra network load by exploring the traffic volume to and from the EC2 instance. That way, you can proactively react to security incidents.

Please follow Amazon Elastic Compute Cloud User Guide to create CloudWatch Alarms.

Please review this article from AWS Knowledge Ceter on how can you create CloudWatch alarms to monitor the Amazon RDS free storage space and prevent storage full issues.

We recommend reviewing Amazon CloudWatch User Guide on how to create an Alarm on anomaly detection as well as Create a CloudWatch Alarm Based on a Static Threshold

3. Backup and Recovery

3.1 Backup

iForm Deployment Solution allows you to choose a backup strategy for your RDS instance. We recommend enabling multi-az to provide fault-tolerance for your database server. Please review the appropriate section under Deployment Section.

3.1.1 Amazon RDS Backups

Amazon RDS is a fully-managed database service, which has automated backups enabled by default. Amazon RDS also uploads transaction logs for DB instances to Amazon S3 every 5 minutes, which provides an ability to recover your database to any point in time. You can restore your iForm RDS database to any point in time within your backup retention period. To see the earliest restorable time for each DB instance, choose Automated backups in the Amazon RDS console. Please keep in mind that the Amazon RDS service creates automated backups every 24 hours with transaction logs in the Amazon S3. That means you may not see your backup right after the Solution deployment. Please follow the section Restoring a DB instance to a specified time in the Amazon RDS User Guide for more information.

3.1.2 Amazon EFS Backups

iForm keeps user uploads in Amazon Elastic File System. By default, automatic backups are turned off. Please consider enabling automated backups in Amazon EFS. By allowing automated backups for EFS, you will incur additional charges as defined in the Amazon EFS pricing model.

To enable Amazon EFS automatic backups, you must edit the existing iForm EFS Filesystem and check the “Enable automatic backups” checkbox. Also, please consider setting “Lifecycle management” by establishing a retention policy for your backups.

img

3.1.3 Amazon EC2/EBS Backups

To backup your EC2 Application Server (this includes application source code and system libraries), please consider enabling “EBS Lifecycle Manager” to create automated EBS Snapshots of your instance. You will need to create a “Lifecycle Policy” in Data Lifecycle Manager and define your retention policy, schedule, and other options.

You can copy EBS snapshots across regions using Data Lifecycle Manager. You can enable policies which, along with create, can copy snapshots to one or more AWS regions. Copies can be scheduled for up to three regions from a single policy and retention periods are set for each region separately. This will allow you to recover your EC2 Application server from Region failure. Please review Amazon Data Lifecycle Manager User Guide for more information.

3.2 Recovery

iForm Application EC2 server does not store any mission-critical user data. That is why it can be replaced at any time without data loss. That assumes you have automated backups configured for Amazon RDS and Amazon EFS. In case of failure in which your EC2 Application server becomes unavailable, unhealthy, or compromised, you can redeploy it from AMI or your EBS automated snapshot. Please review Terraform template for the exact AMI ID.

To restore your EC2 instance from an Amazon EBS snapshot or an AMI, please review Amazon Prescriptive Guide.

To recover your Amazon RDS instance, please follow AWS RDS User Guide.

To restore your Amazon EFS File System, please follow AWS Backup User Guide.

This Deployment Solution provides a Recovery Time Objective of less than one hour and a Recovery Point Objective of less than 24 hours. Please ensure that RDS and EFS automated backups are enabled and create a policy for EBS snapshots every twelve hours or less.

After you have deployed a new EC2 server from the EBS snapshot, please make sure to add the new Instance to Elastic Load Balancer Target Group. See the picture below for more information.

Edit your Load Balancer Target Group.

img

Attach new Instance to your Load Balancer Target Group.

img

4. Operational Maintenance

4.1. Checking for AWS Service Limits

We recommend regularly review CloudWatch Logs and enable log expiration to remove stale data. Also, please keep in mind that iForm Solution uses AWS SES to send emails. That means you must regularly review AWS SES service to avoid going beyond your limits. If you need to expand your sending limits - please contact your AWS Support within the AWS SES service.

Please review Service Quotas Integration and Usage Metrics in Amazon CloudWatch User Guide.

4.2. Horizontal Scaling (Up and Down)

iForm Solution does not support autoscalling. This means you must reguraly review your system usage and scale EC2 and RDS instances manually if needed. Please review corresponding section on Monitoring in this Guide.

4.3. Expenditure and Usage Awareness

CloudWatch can be used to trigger alarms on various metrics, including AWS Service usage. Please review Amazon CloudWatch User Guide for more information.

4.4. AWS Budgets and Alerts

We encourage you to use AWS Budgets to trigger Alarms when your AWS expenses go beyond specified limits. Please follow AWS Documentation to manage your costs with AWS Budgets.

5. Emergency Maintenance

5.1. Handling Fault Conditions

iForm is configured to publish application and system logs to CloudWatch. In case of faulty conditions, please review logs with the following strategy in mind.

  1. iForm-Application-Logs - this is main application log stream. If you can see errors in this file - that is most likely a bug in our software.
  2. iForm-Nginx-Access-Logs - this is the main proxy server log stream. If you can see new logs in this file but getting en error on your browser - that means Puma application server is down.
  3. iForm-Puma-Access-Logs - this is the main application server log stream. Here you will find Puma application server issues if any.
  4. iForm-Puma-Error-Logs - this is the main application server log stream. Here you will find Puma application server issues if any.
  5. iForm-System-Messages-Logs - this is the main system log stream. Here you will find all OS related issues if any.
  6. iForm-User-Data-Logs - this is populated with the output when the EC2 instance is first provisioned. If you try to deploy and get some issues - please review this file.
  7. iForm-Yum-Logs - this is the main yum log stream. Here you will find OS updates and installed software output.
  8. iForm-Nginx-Error-Logs - this is the main proxy server error log stream. If you can see new errors pupulate in this file and getting en error on your browser - that means Puma application server is down.

Please review the picture below to locate each log stream in Amazon CloudWatch. Please make sure to check application and system logs regularly.

img

Previous Section Next Section