# Rule Engine
## Basic Concept
Since the rule logic of the insurance business can be highly complex, it is essential that the rules be configured to be maintained quickly and flexibly for constant updates based on the changing business needs.
The Rule Engine is used to configure not only insurance underwriting or validation logic, but also other common logic, such as auto-clause attachments or workflow assignments.
## Composition of Rule Engine
### What is Rule?
During insurance operations, various client-specific business rules control different system behaviors.
Unlike ordinary program code which requires code release for configuration changes, and presents modification challenges in many scenarios, a rule configuration capability is needed. This capability should be comparatively simple, enabling direct operations on the configuration page with the following attributes:
* Easy to maintain and configure.
* Able to modify and test results in real time.
* Able to publish to other runtime environments for use in real time.
The team is committed to continuously enhancing rule configuration tools, and optimizing the configuration experience as much as possible.
### Rule Usage Scenario
The Rule Engine can be used in a variety of business scenarios, and the most common ones in our insurance business are as follows:
* Underwriting
Underwriting is a common rule in insurance business. When triggered, it generates a feedback message. The operator then decides whether the case can continue or needs to be escalated to an underwriter for approval.
* Validation
Validation rules are sometimes similar to underwriting rules in business checks and message prompts. However, the difference is that underwriting rules allow manual decisions regarding pass or fail, whereas validation rules are strict and do not permit a pass under any circumstances.
In other words, underwriting rules mostly have soft controls based on human judgment, while validation rules tend to be hard stopped.
In addition to the business usage, some validation rules can also be related to the system to ensure the integrity of the system data rather than being used for business information checks.
* Auto Additional Clause/Form
In some cases, different clauses need to be added to the policy based on specific field selections. For example, the selection of terrorism coverage may result in additional terrorism extensions to be sent out.
* Endorsement Content Generation
If policy information changes during the endorsement, the endorsement needs to be automatically generated to demonstrate the difference in information before and after the endorsement. Different content may contain different business conditions to determine the type of wording or conditions that are triggered.
* Other Complex Check Rules
In summary, our Rule Engine supports all of the scenarios mentioned above and allows system analysts to configure systems dynamically. It has the advantage of being flexible and adjustable.
The document will primarily provide examples related to policy modules and domains. However, the Rule Engine can also be utilized for other purposes, such as claims.
### Rule Engine Component
The Rule Engine consists of three major elements, including rule, rule set and rule domain.
* **Rule**
The rule is the basic unit to implement business logic.
In the case of homeowner's insurance, a typical rule validates the building type. Typically, buildings can be made of wood or reinforced concrete. Wood buildings have a higher risk of fire and may require some special underwriting considerations or a maximum sum insured. Sometimes there is a maximum amount of insurance that can be underwritten. If the end user wants to change the sum insured at the time of approval, a cap verification can be mandatory and the limit is not allowed to be exceeded.
Another example is the vehicle type, which also requires special treatment for high-end car brands or sports cars.
Since most rules are currently written in the Groovy script, these rules can be managed through a relevant rule management mechanism.
* **Rule Set**
The rule set is a grouping of rules.
During configuration, some analysts may be responsible for the auto insurance configuration rules, while some may be responsible for the configuration of home-related rules.
To prevent task interference, our rule engine supports grouping these rules separately, putting auto insurance validation rules into one rule set and home validation rules into another. Likewise, their endorsement-related rules can be grouped differently.
Given the numerous rules to configure, we can further categorize them by several dimensions, including multiple types such as:
- The motor or health insurance
- Issuance or endorsement transaction
- The underwriting or validation rule
Also, another key consideration for using rule set is to define execution sequence among different rules.
Groovy rules and rule sets are associated 1-to-many, and one rule can only belong to one set.
* **Rule Domain**
The domain is the trigger point for rule execution.
To execute any rule, it must be linked to a rule domain, which serves as the trigger point for system API calls. Thus, rules need to connect to the rule set, and the rule set must in turn link to the rule domain.
A rule set can be seen as a management unit, while a rule domain serves as a system trigger unit. The rule set is used to facilitate the search or retrieval of specific rules during the configuration process, while the rule domain is used for system triggering.
The rule set and rule domain are associated in a 1-to-many manner. There are multiple rule sets under one rule domain, and one rule set only belongs to one rule domain. When a domain is called, all rule sets under that domain will be run through and the rules under the rule set will be executed sequentially.
### Other Important Configuration Modules
This document mainly deals with Rule Engine operation itself. In addition, the following modules are also important during rule configuration:
* Groovy & DSL
To be able to use Rule Engine, it is also vitally important to grasp how to write rule scripts. See [DSL Guide](https://docs.insuremo.com/common/doc_00000019) for more information.
* Message Code
Sometimes, we need to throw out rule message in the rule body. See [Message Guide](https://docs.insuremo.com/common/doc_00000042) to learn how to maintain the message.
* Configuration Table
During the configuration, we often need to look up a dynamic table to judge whether the condition matches the expectation or not. See [Configuration Table Guide](https://docs.insuremo.com/common/doc_00000045) to learn how to maintain a dynamic table.
### Rule V1 and V2
After the introduction of Rule V1, the insureMO team received significant feedback, particularly from business users who found script writing too technical and desired a more user-friendly solution. Additionally, our internal review revealed the underlying room to improve the current rule execution mechanism for better performance.
Based on the above, Rule V2 is released, especially with considerable improvements in:
1. There is a new script UI which allows business users to select all the parameters rather than write them plainly.
2. There is a new runtime engine with considerable performance improvement (10+ times).
3. There is dynamic rule label search and filter configuration.
4. There is a simplified rule management structure.
Below is an illustration table to compare V1 and V2 for the rule engine.
| Characteristics | V1 | V2 |
|-----------------|--------------|----------------------------------------------|
| Edit Mode | Script Writing | Function Selection + Script Writing |
| Calling Structure | Event -> Rule Group -> Rule (Many to Many) | Domain -> Rule Set -> Rule (One to Many) |
| Performance | Medium | Excellent |
| Tracking Size | Big | Medium |
| DD Context (Common/Tech PD) | No Usage | For model choice and runtime filter |
| Pub Context (Common/Market PD) | Runtime filter and deployment | Only deployment |
| Deployment Scope | Context & Group | Context & Rule Domain |
Also, there's some DSL difference between V1 and V2：
| Characteristics | V1 | V2 |
|-----------------|--------------|----------------------------------------------|
| End Line | No need ; | Need ; |
| Fetch Data | Need .val | No need .val |
| DSL Scope | Free to use most of groovy operator | Only limited pre-defined operator |
| Declaration/Action | Together | Split |
| Fetch Current Level | Both Policy().aaa.val & aaa.val | Only aaa |
| Rule Action | No Restriction | Restricted by Rule Domain |
Since the DSL scripts in V1 and V2 are different, there will be certain efforts for users to make the switch between them. In future releases, insureMO team will further review it and provide a migration program.
## Rule UI Operation Guide
caution
* All operations in this text need to be performed in the portal master configuration environment (MC env). They are not allowed in other environments unless for troubleshooting purposes.
* Please make sure you have read and write authority to perform the operation beforehand.
### Search Rule
Upon entering the Rule Maintenance, the system will display all available rules in the system for analysts to work with. Analysts can then add or edit a new rule or search for rules based on various categories.
If users choose to search rule, there are two options:
1. Search by specific rule name
2. Search by module and other conditions
After users select the module, the system will list all available rule labels for users to choose. These rule labels can be further filtered if users further select the rule domain.
### Rule Maintenance - V2
#### Create and Edit Rule
Once users enter the maintenance of the individual rule V2, the UI displays three sections:
1. Rule Detail - This is to maintain basic rule definition information.
* **Rule Name**: self-defined, the name will be unique for all rules.
* **Rule Level**: the applicable level of the business model defined in the data dictionary which will drive how to fetch data in an object for the rule. Once the rule is saved, this field can't be changed.
* **Context Type/Context**: the data dictionary context which will be automatically populated once the rule level is defined and always remain read-only.
* **Status**: to enable or disable a rule.
* **Start Date**: to define when the rule should take effect.
* **End Date**: to define when the rule should expire.
* **Release Date**: to define the time when the rule is released.
2. Rule Context - This is to define two things: which context condition will trigger the rule and how to deploy the rule.
3. Rule Set - This is to link the rule to the rule set (and the rule domain) and define when the rule will be triggered.
#### Rule Editor
If it's rule V2, then there will be an entire tab for users to maintain the rule script.
##### Grammar Structure
Different from V1 which allows users to freely construct the rule script, in V2, there are two key segments that users need to understand and write the script:
1. Declaration - This is to declare the variable which can fetch data from the policy object or a fixed value.
2. Business Action - This is to determine the condition when triggering the rule and defining the detailed rule behavior.
For the script function selection, from the selection usability perspective, we have further split declaration into two parts:
1. Declaration (Direct) - This is to directly declare a new variable.
2. Declaration (With If) - This is to declare a new variable based on some if-else condition.
##### Function Select
This is to select **Rule Script** in a visualized and user friendly UI.
##### Plain Script
This is to write the **Rule Script** in a plain script manner. Now for all scripts generated from function selection can be converted into plain scripts. However, not all plain scripts can be converted into function selection because they might contain more complex language structure and DSL.
All scripts generated from function selection can be converted into plain scripts. However, not all plain scripts can revert to function selection due to potentially more complex language structures and DSLs.
The grammar is similar to rule V1. See [DSL Guide](https://docs.insuremo.com/common/doc_00000019) for more information.
### Rule Domain Maintenance
Rule domain are maintained in **Rule & Rating > Rule Domain**.
Users can manage all the rule definitions in this page.
#### Rule Business Attribute
Here users can define several attributes which are important for the rule to execute:
1. **Effective Period Field** - This is to define the field to control the effective period of the rule for execution.
2. **Release Date Field** - This is to define the field to control the release period of the rule for execution.
3. **Rule Action Names** - This is to control the available rule DSL for the rule to take actions.
If it's a platform rule, tenant users will be able to override it.
#### Rule Label
Users can find the rule label in this section. Generally speaking, there are two usages for the rule label:
1. Search criteria in the rule management UI - Any defined rule label will be displayed in the rule search page for users to query.
2. Runtime execution filter criteria - Any defined rule label with a runtime filter tag might be executed or not during actual rule execution depending on the situation.
Currently, rule labels must be a selection and users need to define a code table for them.
If it's a platform rule, tenant users will be able to override it.
#### Rule Set
This is to maintain different groupings in one rule domain. Also, users can define the execution sequence between different rule sets to for better management.
### Rule Tracking
After each call to rule API, the user can view the details of the trigger in the rule tracking page.
Click **View** to check the tracking details.
### Rule Library Copy
The system supports copying rules from other tenants.
You can choose some rules to copy or **Copy All** the rules of other tenants to the current tenant.
Choose some rules to copy, and click "**Copy Selected**".
After the copying is done, you can check the copying result to confirm the rules copied and whether the system has handled it successfully.
### Rule Configuration Deployment
For rule configuration data release, see [Guide](https://docs.insuremo.com/common/doc_00000016#rule-common).
## Rule Versioning
Currently we don't support multiple physical versions of the same rule (the same rule names). Instead, users can create multiple rules (different rule names) with multiple effective/expiry dates to achieve versioning scenarios. Each rule will have its own version and effective/expiry date. Also, all the ratetable invoked in the rule will follow the same versioning logic as the rule script.
The following are the steps about how to achieve such versioning:
1. Set "**Effective Period Field**" or "**Release Date Field**" for rule domain.
2. Set "**Start Date**" "**End Date**" and "**Release Date**" for the rule under a rule set and domain.
As shown in the image below, the system will take policy's EffectiveDate field as rule Active duration check.
* If the policy's EffectiveDate field is between 2023-11-03 and 2023-11-31, the rule will be fired.
* If the policy's EffectiveDate field is before 2023-11-03, the rule will not be fired.
* If we remove the "**Release Date Field**" from rule set and the "**Release Date**" from rule, the Policy can be fired between 2023-11-01 and 2023-11-31 comparing with the system time.
As shown in the image below, the system will use the policy's EffectiveDate field to check the rule's active duration.
* If the policy's EffectiveDate field is between 2023 - 11 - 03 and 2023 - 11 - 30, the rule will be fired.
* If the policy's EffectiveDate field is before 2023 - 11 - 03, the rule will not be fired.
* If we remove the "**Release Date Field**" from the rule set and the "**Release Date**" from the rule, the policy can be fired between 2023 - 11 - 01 and 2023 - 11 - 30 compared with the system time.
## Rule Debugging and Troubleshooting
### Method
After the rule is configured, we need to run the rule to view whether it meets the business requirements. The process, also called debugging, virtually contains two modes to debug the rule.
* Front-end Testing
This method assumes that the front-end application has been well-developed in advance. Then the system will return the result of the rule to the front-end by clicking certain trigger points like buttons during the front-end process (clarify the specific process as needed). It is regarded as the most user-friendly approach for end business users and business analysts to conduct functional testing of the system.
* API Testing
Sometimes, front-and-middle-office based models, where the front-end office may not be available during configuration, increasingly rely on API validation for rule debugging.
Adopting this approach requires a technical underwriting of data model calls by system users in the first place, which can be used for developer unit testing or massive automated testing.
### API Testing
Different APIs may be required to trigger different types of rules. There are specific APIs for underwriting and others for validation, so they have slightly different API parameters. For example, validation rules may require different domains to trigger different rules during insurance, underwriting, or policy submission and issuance. We can control the various stages with API parameters to validate different data.
After the API is called, if any rule fails, a failed validation message will be returned. An example is displayed in the figure. There may be common message information (e.g. validation date, current system time) and specific business information (e.g. business message code or business message name).
To facilitate rule debugging, the rule tracking is provided to view the history of all rule executions. Suppose there is a rule that has a function to obtain the sum insured, compare the sum insured with the limit, and then return a message. The traces of all these key steps can be recorded and viewed. This is an exceptionally useful feature that allows analysts to debug the rule.
For the APIs that can fire the rules, see [API Description](#sample-reference-rule-domain-and-api-for-gi-business).
### Precondition for Execution
* The rule is active.
* The rule is in the effective period.
* The rule is mapped to common or an expected market product.
* The rule is mapped to a rule set.
* The rule set is mapped to a rule domain.
### Trace ID
Each time a Rule API is fired, a trace ID will be returned to indicate the trigger.
Users can then obtain the trace ID from the response's header and search for it in the rule Tracking.
### Tracking Log Performance Housekeeping
Nowadays, our tracking log is switched on in all environments by default including production to facilitate the development team in conducting the troubleshooting process. In order to balance the production system performance, there will not be line number display and tracking is captured in an asynchronous way.
Since the tracking log is highly comprehensive and includes every aspect of rule execution. Sometimes, if too many rules are triggered, the log table (`t_rm_tracking`) can become so large that it requires a regular clean-up.
As part of the platform default setting, we offer a "ruleTrackingDeleteExpiredDataJob" batch which keeps running every night to clean up the log which was created more than 90 days ago. If the records increase too fast, users can also go to the batch module to run it with a shorter time interval.
Please pay special attention to the fact that if your client is very performance demanding, you can request the insureMO support team to disable the tracking entirely.
### Rating Tracking Configcenter Parameter (Private Cloud Only)
By default, in all environments, when debugging rule logic, we always need to check the rule logic details. Then we can use the rule tracking function.
Sometimes we might want to disable the tracking in other environments, especially in the case of private clouds, for the sake of best performance.
Users can go to config-server on the platform and check the following configuration items (if the configuration items do not exist, they will take the default value).
1. rule.tracking.flag
* true-enable (default)
* false-disable
2. dsl.tracking.flag
* true-enable (default)
* false-disable
3. rule.tracking.async
* true-async (default for production)
* false-sync (default for non-production)
4. dsl.tracking.lineNumber.flag
* true-display (default for non-production)
* false-not display (default for production)
## Rule Development Guide
Once the rules are configured, we can test all rule-related APIs and embed them into application usage.
By default, the platform has a set of pre-defined domain APIs to facilitate rule usage. Also, our developers can extend these APIs with back-end calls.
The following is an example of general proposal related to rule validation API and rule sets. Users can also go to the insureMO portal to get the latest list of APIs.
* {{server}}/pa/v1/invokeRuleEvent?eventCode={{eventCode}}&langId=en_US
tip
Set the parameter **langId** in the URL parameters, then the returned message will be in the target language. Also, if you only want to set the target language in the request header, then set the header **x-mo-lang-id**.
Please note that for the above-mentioned policy validation APIs, in addition to the rule validation, we have other data dictionary definitions to validate as well:
* The value of the field does not match the defined data type.
* The value is not in code table (ID column).
If a field's value is not input, the empty "" will be taken as an input value. So please do not use "", just set value to be **null**, or **remove** the field from JSON request.
* The length of the value does not match the defined one.
* The structure is not consistent with the market product. For example, the benefit is set to the incorrect coverage, or is not defined in the market product.
* The rule is required by the system. For example, PolicyLob is a must and Risk should be under the PolicyLob.
## FAQ
### 1. What is the best way to initiate rule configuration?
Practice is the most crucial part in rule configuration.
For beginning rules, they may be difficult for users to get started due to inevitable errors and notably human errors. However, after a few rule configurations, users will become familiar with common error patterns and then gradually develop a faster way to correct mistakes, especially by using the rule tracking provided by the system.
Our experience has witnessed that things will be much easier later on, and users will comprehend the beauty of its simplicity and plainness.
### 2. Is the Rule Engine configured in the same way as the system was developed?
Rule configuration is indeed performed via the UI, not development. Development usually involves a complex process of compiling, deploying and debugging the console.
However, script configuration does require more demanding technical talents. Sometimes since business users or analysts are reluctant to learn a little more about the programming language, they may not find rule configuration easy to use.
In other words, we cannot force business users to learn programming; instead, there is another recommended approach called parametric promotion.
We can arrange developers or programmers to configure rules. But during configuration, we try to turn those parameters in the rule script into a configuration and then expose the configuration table to the business unit for configuration. Take a simple rule for maximum sum insured validation as an example. We can put the maximum sum insured in the rule script. Alternatively, we can also put the value in a decision table and then arrange for the business unit to maintain that table directly.
To implement a product factory, we recommend that our technical consultants collaborate with business users to gather configuration requirements as a team. Our technical colleagues can then configure the body of rules through scripted configuration, while extracting all parameter values (e.g. coverage limits and house types) into the configuration table. Afterwards, during system maintenance and after the system goes live, the business team only needs to pay attention to all the extracted configuration tables, and then make adjustments and issue releases. If new rules need to be added and it involves scripting adjustments, again the technical department needs to be involved to accomplish this task.
### 3. I do not quite comprehend the concept of parameterization. Could you please enlighten me more in this area?
Suppose we want to implement a rule that checks the maximum sum insured limit.
There are two implementation options:
Option 1: Direct Implementation
```dsl
if(SumInsured.val > 35000000){
// Direct Judgment
return uwResultWithUwLevel("TWBNP-ZERF-INSUREMO-W0007","0")
}
```
Option 2: Rule to Call Configuration Table
```dsl
def ActionResult = 0
ActionResult = lookup("ZERF_LIMIT",["CoverageCode": ProductElementCode.val,"Amount":SumInsured]).Result
//Call Configuration Table to Get the Result for the Judgment
if(ActionResult == "2"){
return uwResultWithUwLevel("TWBNP-ZERF-INSUREMO-W0007","0")
}
```
We can observe that for Option 1, the maximum limit is configured in the script itself. Therefore, if the business needs modification, system maintainers need to tackle the script modification. But the script for option 2 is relatively stable. All the changeable parameters have been maintained in a tabular structure that is easy to manage for business users.
All tools are available, and not many configuration efforts are taken to extract parameters.
A positive mindset matters.
### 4. If I want a certain rule to be triggered during the quote phase but not during the bind phase, how can I do it?
Basically, there are two ways:
1. Add proposal status as part of the if condition in the rule body.
- This is okay if the number of such cases is small.
2. Create a new rule label for proposal status in the rule domain with runtime filter conditions.
- This is better as it supports both search management and runtime filters.
## Sample Reference Rule Domain and API for GI Business
The default system rule domain is provided with our general insurance services as shown below. If users need to use the default rule domain, please find default platform rule domain and create rule set under it. If you use any customized domain, just use it as param in URL such as "/quotation/v1/validate?eventCode=xxx" (xxx is the customized rule domain).
caution
Please do not mix rules of different usages into one domain. Please separate them by usage to separate domains, and call the proper usage API.
| Service | API URL | Default Rule Domain | Description |
| ------ | ------ | ------ | ------ |
| quotation | /quotation/v1/underwriting | UnderwritingRuleCheck | Business Underwriting |
| proposal | /proposal/v1/underwriting | UnderwritigRuleCheck | Business Underwriting |
| endorsement | /endo/v1/underwriting | EndoUnderwritingRuleCheck | Business Underwriting |
| quotation | /quotation/v1/validate | PolicyValidationVheck | System Validation |
| proposal | /proposal/v1/validate | PolicyValidationVheck | System Validation |
| endorsement | /endo/v1/validate | EndoValidationCheck | System Validation |
| quotation | /quotation/v1/attachElement | AttachElement4Quotation |Attach Element |
| proposal | /proposal/v1/attachElement | AttachElement4Proposal | Attach Element |
| endorsement | /endo/v1/attachElement | AttachElement4Endorsement | Attach Element |
| proposal | /proposal/v1/generateClauseContent | EVENT_GEN_CLAUSE_CNT | Generate clause content |
| endorsement | /endo/v1/generateClauseContent | EVENT_GEN_CLAUSE_CNT_ENDO | Generate clause content |
| endorsement | /endo/v1/generateContent | EndoContentEvent | Generate Endorsement Content |