# Code Table and Data Table

## Basic Concept

Insurance requirements can be highly flexible and customized on a per-client basis. This necessitates the storage of various configuration data in table formats to generate different dropdown lists, perform data validation, and manage system behaviors through alerts.

Our dynamic and extensible design for code tables and data tables is well-suited to address these needs.

## User Scenario

Users who need to maintain definition data in a table format.

## Concept

There can be many pre-defined dropdown code values in the insurance business. The functionality of code table and data table is designed to support such scenarios. 

* **Code Table** is defined as a list of pre-defined items which can usually be selected from. A source data pool needs to be specified to supply it.  

* **Data Table** is defined as a set of table data sources. This type of code table is commonly used for the fields such as limit, deductible and description.

Normally, it is a one-to-one relationship, but there are scenarios, such as vehicle master tables where one vehicle master is configured as a data table to provide data source for multiple code tables, like vehicle make, model or type.

- Data table defines the specific data sources.  

![datatablelist](./image/dd/datatablelist.png)

![datatable-s_info](./image/dd/datatable_s_info.png)

- Code table defines the display rules.

![codetablelist](./image/dd/codetablelist.png)

![codetable_s_info](./image/dd/codetable_s_info.png)

- Data table and code table support multilingual capabilities.
- Data table and code table can import and export environmental data and automatically release them.


## UI Operation Guide

<div class="docs-caution"><span class="docs-admonitions-text">caution</span>

*  All operations mentioned in this text need to be performed in the master configuration environment (MC env) on insureMO Portal. They are not allowed in other environments, except for troubleshooting purposes.
*  Ensure that you possess both read and write authority before performing any operations.

</div> 



### Data Table Maintenance

#### Search Data Table

When you enter the data table maintenance UI, you can first identify the level of the data table based on **Maintained By** column. There will be some platform-level data tables for platform requirements. You can extend and maintain your own tenant-level data tables according to the specific business.  

- Fuzzy queries for data table

   Input keywords for fuzzy queries.

  ![datatable_search_query_one](./image/dd/datatablelist.png)

* Detailed information about data table

    Click ![edit](./image/dd/edit.png) at the end of each line to obtain detailed information.

  ![datatable_search_one_detail](./image/dd/datatable_search_one_detail.png)

#### Add a New Data Table

1. Select **Module** and **Group** based on the requirements.

   ![Select Module & Group](./image/dd/datatable_select_group.png)

2. Click **Add** to display the basic information dialog, and input the basic information.

   * **Name**: Self-defined.

   * **Is Small**: Set to **NO** if the number of records exceeds **1000**.

   * **For Configuration**: Flag to indicate whether the data table is used to record configuration items. If flagged, the primary key of the data table will be generated automatically if necessary.

   * **Update Records Remotely**: Flag to indicate whether the data table record is allowed to be updated from the client.

   * **Code Table**: Flag to indicate whether the data table is mapped to a code table. If it is mapped, the system will automatically mark it..

   * **Bind ES Index**: Flag to indicate whether to synchronize data to a specific index. There's a condition that index must be pre-defined via index management before putting here. Now auto synchronization will only happen during data table biz-data deployment. If user want to test in MC, user can click the refresh icon to synchronize manually.


     ![datatable_add_basicinfo](./image/dd/datatable_add_basicinfo.png)

3.  Click **Add** to add fields or columns to the data table.

    * **Name**: Field or column name of the data table.    

    * **Field Type**: Data type of the field. The double number should be limited to 8 digitals. 

    * **Primary Key**: Flag the field as the primary key of the data table. If it is packed, the value of the field must be unique.

    * **Is Indexed**: Flag the field as an indexed field of the data table. Indexing improves search efficiency especially for large tables(where **Is Small** is set to **NO**). Properly set indexes as needed.

    * **Not Null**: Flag whether the field cannot be null or empty.	

    * **Description**: Provide information on the table usage.

    * **Reference Code Table**: The code tables that use this data table as data source.

    * **Allow Value Override**: Flag whether the field can be overridden by the tenant. For platform-maintained data tables, an existing record is allowed to override the field value, which means that the value can be reset by the tenant. 	

    * **Support Line Break**: Flag whether the field can support line break. It's useful when there's a need to maintain a large paragraph of wording such as a piece of code example or documents.	

    * **Is Scope**, **Scope Type**: Flag whether the field can support scope type. It's available only if the data type is **Date**, **DateTime**, **Long** or **Decimal**. When the Scope is set, value must be confined into the restricted format: (Number: 0..100 or MIN..MAX; Date: 2020-01-01..2020-05-01; Date Time: 2020-01-01T00:00:00..2020-01-01T23:59:59)

   
4.  Click ![save](./image/dd/save.png) at the end of each line to save the field settings.

5.  After adding all fields, click **Save** to save the data table structure.

    ![datatable_add_fields](./image/dd/datatable_add_fields.png)

6.  In the Record section, click **Add** to add the table records.

    ![datatable_add_add_data_button](./image/dd/datatable_records.png)
  
#### Update Data Table

##### Update the Basic Information

  Update the basic information and click **Save**.

 ![datatable_update_basicinfo](./image/dd/datatable_s_info.png)
 

##### Update Fields

  1. Click ![edit](./image/dd/edit.png) at the end of each line to edit the field.

  2. Click ![delete](./image/dd/delete.png) at the end of each line to delete the field.

  3. Click **Add** to add a new field.

  4. Click **Save** to save the modification.

Generally, modifications to fields necessitate corresponding adjustments to the table data.

![datatable_update_fields](./image/dd/datatable_records.png)

##### Update Table Data

  1. Click ![edit](./image/dd/edit.png) at the end of each line to edit the table data.

  2. Click ![delete](./image/dd/delete.png) at the end of each line to delete the table data.

  3. Click **Add** to add a new table data.

  4. Click **Save** to save the modification.

#### Delete Data Table

1. Click ![delete](./image/dd/delete.png) at the end of each line to display the deletion confirmation dialog.

2. Click **Yes** to confirm.

#### Download and Upload Data Table

You can follow below steps to download data table and then upload it after changing some of values:

1. Select the existing data tables you want to download.

2. Click **Download Excel**.

3. Edit it locally to add the required data tables.

4. Select the desired module and group.

5. Click **Upload Excel**.

##### Tips for Download

In the download excel, any column defined as a code table will be automatically converted into a dropdown (also named as data validation in excel) as following:

![data_table_download_dropdown](./image/dd/data_table_download_dropdown.png)

The source data for these dropdowns is pulled from another sheet named  **CodeTable_Reference**:

![data_table_download_dropdown_source](./image/dd/data_table_download_dropdown_source.png)


##### Tips for Upload

After uploading an Excel file, if the data table already exists, it will be updated. If it does not exist, the system will report an error, and users need to create the data table structure via UI first.

Currently, the upload process of all data tables is asynchronous. Despite the success in the upload dialog, the actual upload result will be posted in the message center. You need to visit the message center to see details of the upload.

![message details of the uploading](./image/dd/datatable_upload_message.png)

If the message is as follows, it means that your data table exceeds 1,000 records and you need to flag **Is Small** to complete the upload.

`
Upload dataTable excel Failed ! File named BR_City.xlsx ,log traceId : 054b7bbb824fe0e7, error log : com.ebao.unicorn.platform.foundation.exception.BusinessException: Small DataTable records more than 1000.
`       


#### Override Data Table

You can extend some platform maintained data tables with more values or columns, and override the old data table.

1. Query data table.

2. Override the data table maintained by the platform.

3. Automatically create a data table with the same name (In the overwritten data table, the records in the platform maintenance cannot be deleted or modified).

  ![datatable_override_update](./image/dd/datatable_override_update.png)


4. Allow new columns or new records to be added to the data table.

  ![datatable_override-addcolumn_update](./image/dd/datatable_override-addcolumn_update.png)
 
<div class="docs-info"><span class="docs-admonitions-text">info</span>

To ensure that the platform data can be properly enforced, there are some basic rules for tenant overriding data tables:


* All columns of the platform table must be presented in the tenant table. Tenant can add more columns to the table itself.

* All values of the platform table rows must be presented in the tenant table. Tenant can only change values when "Allow Value Override" is flagged as **Yes** for the specific column. 

Sometimes, users might find it unable to update values in the tenant override table. It is probably because there are some platform upgrade which change the basic platform table structure or values. Once that happens, please follow the following steps to update the tenant override table:
  
1. Download the tenant maintained data table for backup.

2. Delete the tenant maintained data table.

3. Override the data table again.

4. Move the tenant maintained records from the backup to the newly overridden data table.

5. Commit the changes to Git, and perform CI/CD.

</div>



#### Data Table - One Table One Page Display

For scenarios where business users need to maintain specific tables without access to all tables, you can configure these tables into separate menu items.

* Configuration Steps:

   1. Go to URP module and add a new menu permission.

   2. Input **Entrance URL** in the following format：  
    `$originalUrlPlaceholder$/ui/datadictionary/#/dataTablePage?tableName=XXX&showRecordsFlag=true&showInfoFlag=true&contextFields={"ProductCode":"productcode","ProductVersion":"1.0"}&editAble=true&apcforca=M-SJZD-DMB`.

   3. Parameters:
       * **showRecordsFlag**: To show the records list tab or not, default is true.

       * **showInfoFlag**: To show the table definition tab or not, default is true.

       * **editAble**: To show the table   editable or not, default is true.

       * **contextFields**: To show the context related records only.

    ![One Page Config](./image/dd/data_table_one_page_config.png)

* Final Display:

    ![One Page Result](./image/dd/data_table_one_page.png)

<div class="docs-note"><span class="docs-admonitions-text">note</span>

For tenants intending for business users to maintain data in the production environment instead of MC, remember to flag "Update Records Remotely" to "Yes" so that the deployment package will contain only the data table structure.

</div>

#### Data Table - Update Records Remotely

Sometimes, there might be some tables that users want to directly maintain in the production environment instead of in the MC environment. This requirement implies two things:

*  MC definition is still required but will be limited to structural definition only. We still need the configuration deployment process to deploy the table structure to all environments so that program can recognize and function it.

*  The data defined in MC need to be overridden in each runtime environment. For safety, even if test data is configured in MC, it should not be exported to override the configuration setting in the target environment.

For such cases, we have supported **Update Records Remotely**. This is a flag upon any of the table creation. 

* By default, it will be **N** which means that table structure and data need to be both maintained in MC environment.

* If users change it to **Y**, it means that only table structure will be maintained in MC and deployed without any data, and users will find no data actually deployed in the runtime. 

In both cases, users only need to click **Export Release Package**, and the system will automatically split them into different files.

There might be a difficulty for initial production setup. If there's only structure to be deployed, then users need to manually input values in all environments. One possible solution is to initialize all tables to **N**. Then only after production launch, change it to **Y**, which will prevent any new data override by mistake. 


<div class="docs-note"><span class="docs-admonitions-text">note</span>

Please proceed carefully when changing the Update Records Remotely from N to Y. Once saved as Y, this field becomes read-only and can no longer be modified, the module and group cannot be changed either.

</div>



#### Data Table - Search Context Type 

##### Table Context

Table context refers to tables that have their context at the table level. 

This is useful when each product has a distinct set of tables. For example, let's assume there are two products, A and B. By using table context, each product has a whole set of rate table. It is commonly used in general insurance business scenarios where the entire configuration structure can be totally different between products.


| Table A - Product Context A |      |
|-----------------------------|------|
| Sum Insured                 | Rate |


| Table B - Product Context B|      |
|-----------------------------|------|
| Sum Insured                 | Rate |


##### Column Context

Column Context refers to tables which belong to multiple contexts defined at each table record level. 

This is useful when each product has a distinct set of tables. For example, with two products (A and B). If it's column context, there can be just one single table which has a product code column to indicate whether it's product A or B. It is often used in life business scenarios where most structures are the same and product variances can be illustrated as value differences.

| Table - Common Context|           |      |
|-----------------------| ------    |------|
| Product              | Sum Insured| Rate |
| Product A            |            |       |
| Product B            |            |       |


##### Comparison

When searching, you can search by table context or column context. Combining both contexts allows users to manage the whole attribute impact of each product.

There are two major differences between table context and column context:

* **Table Setup Consideration**: Whether invoke program is written at a common level or product level.

  - For premium calculations with distinct logic for different products (e.g., vehicle rate vs. home rate), separate tables with different columns are appropriate.

  - For common system logic like campaigns or discounts that apply across product lines, a common table with a product column for context is easier to maintain.


* **Table Deployment Consideration**: Whether deployment is conducted at the table level or record level.

  - For table context, as long as the table is tagged to specific product, it is deployed as a whole within the product package .

  - For column context, two deployment steps need to be performed: deploy the table structure at a common level and deploy the table row values by each product. 



##### Deployment Consideration

Now the system supports to tag multiple contexts to one data table, which is common in table context where one table is used for multiple products.

Although it provides some facility for maintenance, it introduces significant risks because the mentioned table will be exported and check-in all these product file. Thus, if there’s update on one table itself, users might only remember to check in specific products while not checking others. This oversight might lead to different versions of the data being present in the git packages, which, if not detected, might be mistakenly deployed to production or other environments later on. 

In order to reduce such deployment risks, our suggestion will be:

* Change the table to common context and remove all product context bindings. 

* Change the table to column context to flag the product at the item level.

* If continuing to use multiple table context, ensure to re-check in all products whenever the table is changed. The system has a mandatory duplicate check-in validation to prevent discrepancies.


#### Data Table - Change History

To manage the audit logs efficiently, especially for data tables with a large number of detailed records, we have implemented a streamlined approach for batch operations such as Excel uploads and batch APIs. This approach focuses on recording only the essential information about the event once.

For batch operations like uploading records of a table via Excel, only one audit log will be recorded to indicate that an Excel upload operation occurred, and the details of the changes will not be recorded.

For operations recorded individually, such as create, delete, and update operations, audit logs will be maintained. For create and delete operations, all attributes of the information will be recorded. For update operations, only the attributes that have changed will be recorded, including both the before and after states of the changes.

#### Data Table - ES Data Table

There are chances that you need to perform some complex search method, such as fuzzy search, on specific tables. You can configure data table as the data source while also an ES index attached to the data table so that you can use ES search API to achieve complex search method.

The following steps show how to create and use ES datatable:

1. Create an index in Index Management, with the name as you wish (example: datatableABC_index) containing the exactly same fields (name and type) from your datatable in your MC.
2. Edit your datatable in Data Table, set `Bind ES Index` to your index name in step 1. You may need to create a new datatable or override it if the datatable is provided by Platform in your MC.
3. Export the index and its fields into your own bizasset. Let's call it bizasset_search in this instruction.
4. Export the datatable to your own bizasset, for data records and `Bind ES Index` field. Let's call it bizasset_table in this instruction.
5. Deploy both bizasset to other environment, like ptdev or internal. Ensure that `bizasset_search` is deployed first and is successful before deploying `bizasset_table`. (If it's in the same bizasset, platform is already making sure the deployment sequence. If it's not in the same bizasset, user need to manually make sure the sequence or double deploy data table after index deployment is successful).
 
If everything is right, the data records in the datatable will be synced to es automatically during step 5. then you can query the index `datatableABC_index` as you wish, fuzzy and paging and everything else.

<div class="docs-note"><span class="docs-admonitions-text">note</span>

If you decide to do it all in MC just to test, you can skip step 3,4,5 and call an API manually to sync the data. If you have much data in data table, you might find timout error when exeucting below API. This timeout will not block execution and you can wait for several minutes and try the data later.

</div>

{dd.app.name}/codetable/deploy/support/v1/action?actionFlag=Y

POST method, body template:

{

	"DataTableNameList":["datatable1","datatable2"]

}
 

In some scenarios, if users want the data table data is maintained in runtime instead of MC, it can be coped with **update records remotely** flag which will turn the whole process into below:

1.	Create data table both flagged as update records remotely and ES data table and create index in MC.

2.	Deploy index and table without data to runtime.

3.	Call data table MANAGEMENT SDK to insert data into data table - /dd/public/code/mgmt/v1/dataTable/data/saveRecordsInBatch in runtime.

4.	Call ES data table synchonization API to synchronize data table to index - {dd.app.name}/codetable/deploy/support/v1/action?actionFlag=Y.

5.	Call ES API to search the data.

There might a problem for point 4 as it’s a synchronous API which might get error for timeout (though it will still be running in the backend). Also, there will be performance pressure for ES server. It's in the backlog that platform can come up with a new asynchronous API to meet the scenario in future release.



### Code Table Maintenance

#### Code Table Type

There are three types of code tables supported in the system:

* Data Table

    After data table is created, users can associate code table with the data table, and load all values defined in data table. This is the most common way of using code table, allowing users to add or modify values at the data table level.


* Service Table

    Unlike the data table which is created dynamically via our tools, service table sources their data from the established physical structures. A typical example can be a list of product names/codes that take data from the Product Factory service, using product name as the display value and product code as the data value. 

    This table is rather complex to add because it requires relative services to implement the code table interface first. For detailed instructions on how to add it, see [development guidance](#development-guidance).

    Some of the platform pre-embedded examples are as follows. Users can search and find these code table names in the Code Table Maintenance UI:

    * UserInfo 
    * PubBranch
    * Product 
    * Products
    * Plan 
    * SalesChannelAPI
    * SalesAgreementAPI


* DB Table (Deprecated)

    This type involves extracting  data directly from data table. Now it is deprecated and no new tables of this type are allowed to be added. Due to historical reasons, only a few platform tables still keep using this type. 


#### Search Code Table

* Fuzzy query for code table

   Input the keywords for the fuzzy query and click **Search**.

  ![codetable_search_keyword](./image/dd/codetablelist.png)

#### Add a New Code Table

1. Select **Module** and **Group** based on requirements.

  ![Select Module & Group](./image/dd/codetable_select_group.png)

2. Click **Add** to display the code table dialog.

  ![codetable_add_button](./image/dd/codetable_s_info.png)

3. Input the code table information and click **Save**.

|Field Name|Use|Values|
|------------------|-----------------------------------------------------------|-----------------------------------------------------------|
|Code Table Name|Name of the code table| user-defined|
|Code Table Type|Identify the type of data sources, generally 'DataTable'|Data Table, DB Table, Rule Name, Service, DomainModel, ES Search|
|Module & Group|Identify the type of module and group for code table|- |
|Data Source|The name of the data source, such as the data table name, DB table name.|- |
|Data Table|Select the appropriate data table via the drop-down list of data tables| a name of data table|
|Id Field|Identify the ID column for the code table| a field of the data table|
|Display Column|Identify the display column for the code table| a field of the data table|
|Code Column|Identify the code column for the code table| the default setting is the same as Id Field |
|Condition Field |Filter the conditions of the code table| a field of the data table |
|Sort Column |Sort by condition of the code table| a field of the data table |
|Sort Type |Sort type of the code table| ASC, DESC |
|Need Multiple Language |Flag whether I18n is required |Yes, No |
|Invalid Field |Field name used to mask the invalid records  | invalid column name  |
|Invalid Flag |Flag used to mask the invalid records |  the records' value in column. After being set, the records with column's value equal to "Invalid Flag" will not show up. |

#### Update Code Table

1. Click ![edit](./image/dd/edit.png) at the end of each line to display the modification dialog of code table.

2. Update the basic information of code table and click **Save**.

  ![codetable_update_basicinfo_save](./image/dd/codetable_s_info.png)

#### Delete Code Table

1. Click ![delete](./image/dd/delete.png) at the end of each line to display the deletion confirmation dialog.

2. Click **Yes** to confirm.

#### Download and Upload Code Table

1. Select the existing code tables you want to download.

2. Click **DownLoad Excel**.

3. Edit it locally to add required new code table.

4. Select the desired module and group.

5. Click **Upload Excel**.

<div class="docs-info"><span class="docs-admonitions-text">info</span>

 * After uploading, if the code table exists, it will be updated. If it does not exist, it will be added.
 * Currently, despite the success in the upload dialog, the upload result will be posted by message in the upper-right corner on UI. You need to check the message details of the upload.
   
   ![message  details of the uploading](./image/dd/datatable_upload_message.png)
 
</div>
  
#### Import and Export Code Table

- Export the selected code table

  Select the expected code table and click **Export Deployment Package** to get a zip file.

  ![codetable_export_selected](./image/dd/codetable_export_selected.png)

- Export code tables

  1. Select **Module** and **Group**, and click **Search**.

  2. Click **Export Deployment Package For All** to get a zip file.

  ![codetable_export_all](./image/dd/codetable_export_all.png)  

<div class="docs-tip"><span class="docs-admonitions-text">tip</span>

If in the related data table, **Update Records Remotely** is set as **Yes**, the records will not be exported and only the related information defined in the data table will be exported. Clients can maintain the records via the API.

</div>


- Import code tables

  1. Click **Import Deployment Package** to display the upload file dialog.

  ![codetable_import_button](./image/dd/codetable_import_button.png)

  2. Import the zip file successfully by placing it into the specific area.

  ![codetable_import_success](./image/dd/codetable_import_success.png)



## Scenario Study

### Code Table or Data Table with Large Number of Records

There are various methods to handle different business scenarios with large number of records:

* Method 1 - Standard Data Table and Code Table

If data table records is less than 1,000, the system defaults **Is Small** to **Y** and runs the standard logic. Then the Code table API will retrieve all data for you.

* Method 2 - Big Data Table and Code Table

If data table records is more than 1,000, it's compulsory that you mark **Is Small** as **N**. The system will run the large data table logic，and you also need to define 1-3 logical index fields in order to use DB's index to improve query performance. Through this approach, you can still adopt code table API but it will only provide a maximum of 1,000 records at a time, even if the total number of records is larger than 1,000. User cannot do any search for records beyond 1,000. It is recommended to include indexed cascading columns in requests to filter results to under 1,000 records.


* Method 3 - ES Code Table via Platform Standard Index

If data table records is more than 1,000 and there's a need to search beyond 1,000, Elasticsearch(ES) support is required.

In this case, not only **Is Small** needs to be marked as **N**, you also need to mark the code table as ES code table. Then users can  adopt both the code table API and the separate ES code table API for data retrieval. The advantage of this method over method 2 is that code table API allows keyword searches, including fuzzy searches.


* Method 4 - ES Data Table via Customized Index

If users want to search multiple columns in data table but only a code dropdown list with over 1,000 records, user need to search via ES data table.

Not only **Is Small** needs to be marked as **N**, you also need to maintain search index in **Index Management** menu, and bind it to the data table via the BindIndexName field. The index should match all fields in the data table by name and type. When data table is deployed to a new environment, the platform service will automatically synchronize every record in the data table to the bound index (An index should be bound to only one data table). 

You need to write your own code using the ES query API (queryBaseResult or advancedQuery in SearchIndexServiceResource if using ‘vela-search-sdk’) to search on input in the UI while filtering by specific fields. 


|           Senario   | Data Records Less than 1000 |  Data Records More than 1000        |
|-----------------------------------------|-----------------------------|----------|
| Don't need filter                       | Method 1                    | Method 2 |
| Need to filter by specific fields       | Method 1                    | Method 2 |
| Need to search by keyword               | Method 1                    | Method 3 |
| Need to search by all data table column | Method 1                    | Method 4 |



#### Filter DataTable Records with Conditions (management time)

{{server}}/{{apigw-platform-url}}dd/public/code/mgmt/v1/recordList/filterWithDataTableId?id={{datatableid}}

* Sample Request Data

```
{
"IsActive": "Y",
"ProductCode": "TBTI"
}
```

* Sample Response Data

In the response, if **HasMoreRecords** is **Y**, it indicates that there are more than 1000 matching data records, and some records may not be returned. It is advised to set more condition fields to retrieve more accurate data records.


```
{
    "HasMoreRecords": "false",
    "Records": [
        {
            "@pk": 631872466,
            "DataTableId": 628104532,
            "RecordId": 631872466,
            "RecordUsage": 5,
            "BizSource": "1",
            "CommissionRate": 10,
            "Id": "1",
            "IsActive": "Y",
            "ProductCode": "PPA",
            "SalesChannelCode": "D000001",
            "SalesChannelName": "CUSTOMER-1",
            "ValidityPeriod": "2023-07-01..2099-12-31T23:59:59"
        }
    ],
    "TotalCount": 1311
}
```


#### Lookup DataTable Records with Conditions (runtime)

{{server}}/{{apigw-platform-url}}/dd/public/codetable/v1/dataTable/lookup?dataTableName={{dataTableName}}

* Sample Request Data

```
{
"IsActive": "Y",
"ProductCode": "TBTI"
}
```

* Sample Response Data

```
[
    {
        "BizSource": "1",
        "CommissionRate": 10,
        "Id": "1",
        "IsActive": "Y",
        "ProductCode": "TBTI",
    }
]
```


### Code Table and Data Table Override

By default, the platform provides a list of data tables and code tables. If you need to extend these tables with additional values, you can use the override functions.

After overriding a table, the system supports three operations:

* Add a new row into the platform table, which may be necessary if the platform table is empty or lacks specific business-required values.

*  Add a new column into the platform table, which can filter the platform-level value.

*  Adjust a column value from the platform. It is allowed only if the column attribute **Allow Override** is set to **Yes**. If it is not allowed, please contact the platform team for help.
     
Take the EndorsementType as an example. Some tenants only need to keep the BIE and the Cancellation in the UI drop-down list.

1.	Add a filter column **IsActive** to the data table.
    For details on data table override, see [Override Data Table](#override-data-table).

2.	Add a code table with the same name as the overridden one and set the newly added column **IsActive** as conditional.

    Because the code table is NOT recommended to be overridden, ensure that changes will NOT affect the platform function. 

3.	Query the code table or data table with the specified filter conditions.


#### Code Table VoList

{{server}}/dd/public/codetable/v1/codeTableVoList/byNameList

```
[
    {
    "CodeTableName": "EndorsementType",
    "ConditionMap": {
	"IsActive": "Y"

    }
  }
]
```

**Response:**

```
[
    {
        "BusinessCodeTable": {
            "@pk": 505608712,
            "CodeField": "DataValue",
            "CodeTableId": 505608712,
            "CodeTableType": -1,
            "ConditionFields": [
                "IsActive"
            ],
            "DescField": "DisplayValue",
            "GroupId": 126,
            "IdField": "DataValue",
            "IsSmall": "Y",
            "Name": "EndorsementType",
            "NeedCache": "Y",
            "NeedMultipleLanguage": "Y",
            "RecordUsage": 5,
            "ServiceName": "dd"
        },
        "BusinessCodeTableValueList": [
            {
                "Code": 1,
                "ConditionFields": [
                    {
                        "IsActive": "Y"
                    }
                ],
                "Description": "Basic Information Endorsement",
                "Id": 1
            },
            {
                "Code": 3,
                "ConditionFields": [
                    {
                        "IsActive": "Y"
                    }
                ],
                "Description": "Cancellation Endorsement",
                "Id": 3
            }
        ],
        "ConditionMap": {
            "IsActive": "Y"
        }
    }
]
```

#### Query Large Data Table

{{server}}/dd/datatable/large/records/v1/query?dataTableName=EndorsementType

```
{
	"IsActive": "Y"
}
```

**Response:**

```
[
    {
        "@pk": 505608689,
        "DataTableId": 505608688,
        "RecordId": 505608689,
        "RecordUsage": 5,
        "BusinessObjectCode": "BasicInfoEndorsement",
        "DataValue": 1,
        "DisplayValue": "Basic Information Endorsement",
        "IsActive": "Y"
    },
    {
        "@pk": 505608690,
        "DataTableId": 505608688,
        "RecordId": 505608690,
        "RecordUsage": 5,
        "BusinessObjectCode": "CancellationEndorsement",
        "DataValue": 3,
        "DisplayValue": "Cancellation Endorsement",
        "IsActive": "Y"
    }
]
```





## Development Guidance

### Load Code Table in BFF Code for Consuming

When developing BFF services and needing to load Data Tables or Code Tables for front-end application consumption, it's crucial to use runtime service methods with caching instead of management restful API. For example, when you need to get a Data Table by name, you should use com.ebao.unicorn.platform.data.code.CodeRuntimeService#findDateTableByNameByCache, instead of com.ebao.unicorn.platform.data.restful.DataTableManagementRestfulService#findDataTableByDataTableName.

The underlying reason is that runtime service will automatically create cache. After cache is setup, the system can retrieve data from the cache without additional API calls. However, since management restful API is designed for configuration management with frequent data changes and does not include caching, the system will always extract the latest data. This is suitable for infrequent code table management tasks.

Even you might find APIs in both methods are similar, misusing management restful API for runtime applications will cause severe performance issues, rapid growth of log, network expense, and service memory peaking.

So if there is no management purpose, our recommended interface to consume Data Table/Code Table is **CodeRuntimeService in vela-codetable-runtime** which you can find in vela-sdk. You can find every possible methods to consume Data Table/Code Table, including definitions and records. The method names are clear, developers can choose appropriate method according to their need.
 
 
### Using Code Table in Rainbow UI

Rainbow UI's Select components can be directly bound to code tables, allowing the options to display data sources provided by the associated data table. There are some usage examples as follows.

#### With Configuration

1.  Click **Dictionary** > **Data Table** and add data source.

  ![datatable_demo1](./image/dd/datatable_demo1.png)

2.  Click **Dictionary** > **Code Table** to build relationships between code table and data table.

  ![codetable_demo1](./image/dd/codetable_demo1.png)

3.  Pass the code table name and the option value to the Select component.
```xml
<Select label='Status' codeTableName='Batch_v2_JobStatus' value={2}/>
```
4.  Run the web app and open the page.

  ![codetable_webui_pic1](./image/dd/codetable_webui_pic1.png)

#### Without Configuration

Sometimes the data source of the drop-down list is provided by back-end API, or generated by front-end hard coding. In this case, no configuration is required.

```xml
const PolicyStatusCt = [
  { "Code": "1", "text": "Not Effective", "id": "1" }, 
  { "Code": "2", "text": "Effective", "id": "2" },
  { "Code": "3", "text": "Cancelled", "id": "3" }, 
  { "Code": "4", "text": "To Be Cancelled", "id": "4" }
];
```
```xml
<Select label='Product' codeTable={PolicyStatusCt} value={searchInfo.Product} onChange={(value) => changeValue(value, 'Product')} />
```



### Add a New Service Code Table in BFF

#### Creation

1. Add code table definition via code table UI.

   *  Mandatory Fields: Code Table Name, Id Field, Code Field, Microservice Name.

   * The Display Field and Id Field are used to flag the display value and data value.

2. Use BFF service to implement the CodeTableDataProvider interface (Key).

   Example Code:

```
@Override
    public boolean support(BusinessCodeTable codeTable) {
        if (CodeTableName.equals(codeTable.getName())
                && BusinessCodeTableType.SERVICE == codeTable.getCodeTableType()
                && CommonParamReader.getApplicationName().equals(codeTable.getServiceName())){
            return true;
        }
        return false;
    }
```

3. Take the following method to construct ID, code, and description.

```
@Override
    public List<BusinessCodeTableValue> loadCodeDataList(CodeTableConditionRuntimeVo codeTableConditionRuntimeVo) {
        List<BusinessCodeTableValue> valueList = new ArrayList<>();
        List<UserInfo> userInfoList = userResource.loadAllUsersInfo();
        for (UserInfo userInfo : userInfoList) {
            BusinessCodeTableValue value = new BusinessCodeTableValue();
            value.setId(userInfo.getUserId());
            value.setCode(userInfo.getUserId());
            value.setDescription(userInfo.getUserName());
            valueList.add(value);
        }
        return valueList;
    }
```

#### Usage

* In HTTP call, all APIs follow CodeTableRuntimeRestfulService with variations only in the service path (notice the service path mentioned below is for products rather than platform-busi-config):

     https://tenantcode***.insuremo.com/api/platform/product/public/codetable/v1/codeTableVoList/byNameList

**Request Body:**
```
[
    {
        "CodeTableName": "Products",
        "ConditionMap": {
            "Id": "TBTI"
        }
    }
]
```
The ConditionMap supports ID and Code as filter conditions.

* In service internal call, methods provided by CodeRuntimeService is adopted.

```
CodeTableVo getCodeTableByConditionWithNameVo(CodeTableConditionRuntimeVo codeTableConditionWithNameVo);
CodeTableConditionRuntimeVo:
 {
        "CodeTableName": "Products",
        "ConditionMap": {
            "Id": "TBTI"
        }
    }
```

### Use Data Table API to Develop Your Own Dynamic Table

For complex table configuration user experience which exceed the capabilities of existing data table UI, you can choose to develop you own UI by utilizing our dataTable management API.  

<div class="docs-caution"><span class="docs-admonitions-text">caution</span>

Please make sure it's only applicable to data table for configuration data only. You should not use this feature to maintain the busienss transaction data.

</div>

#### Define the Data Table Structure

The custom data table is implemented by the **Data Table** solution. The table structure is defined in the data table UI and can be uploaded through the online editor or Excel.

Usually, such extension happens on production which means business user want to directly change the table value in production instead of changing in MC and go through deployment. If this is the case, please remember to flag **Update Records Remotely** to Yes and deploy table structure to respective environments first.


#### Maintain Configuration Data

Development team have two options for user to maintain configuration data:

* Upload via Data Table Online Editor or Excel

  This approach can satisfy most requirements.

* Develop Custom UI

  If you have special requirements for maintaining configuration data, you can develop a custom UI to maintain the configuration data and call the RESTful API of the data sheet to read or save the data.

For both approach, users can refer to [Table Management API Documentation](https://docs.insuremo.com/tech/pro/sdk/api/tablesmgmt) for choose the API to call.

For saving the record, user can call: /dd/public/code/mgmt/v1/dataTable/data/saveRecordsinBatch

To use the lookup, you must clear the cache first; otherwise, you’ll still read the old values.
After saveRecordsinBatch, simply call:
`GET {{server}}/api/platform/common/public/cache/v1/clear/bucket?proxyTo=platform-busi-config&bucketCode=CODETABLE`

Suppose users want to save values as below:

| Id | Make | MakeName | YearRange  | CC   |
|----|------|----------|------------|------|
| 1  | 45   | Toyota   | 2005..2015 | 1500 |
| 2  | 50   | Hyundai   | 2020..2023 | 1600 |


Users can organize the request body as follows:

[
{
"DataTableId": 532288534,
"ID": "1",
"Make": "45",
"MakeName": "Toyota",
"YearRange": "2015..2015",
"CC": "1500"
},
{
"DataTableId": 532288534,
"ID": "2",
"Make": "50",
"MakeName": "Hyundai",
"YearRange": "2020..2023",
"CC": "1600"
}
]



## FAQ


### 1. Code Table vs. Data Table

**Question:**
What is the difference between code table and data table?

**Answer:**
Code table is for front-end use. When using the system, there is always a series of drop-down boxes that need to be displayed. Such drop-down boxes will have many definitions. For example, we can define the sorting method to display the value first when it is retrieved, or define the code table to translate from Traditional Chinese to Simplified Chinese or English to make the information multilingual. When we want to display the code, we can usually store the data value 1/2/3 in the system, representing 1 million/2 million/3 million for UI selection, or to display building type, such as Type I/Type II/Type III. 

Unlike code table, data table simply provides values for the code table. It works like a pure table setup tool to maintain the source of the data. After the code table is configured, it can pick the appropriate columns from the data table for detailed use.

### 2. New Product Onboarding Steps

**Question:**
To create a new product, do I need to prepare all the data tables, code tables, rules, add the product and start to maintain the new policy risk in the component library of the product factory?

**Answer:**
We can perform all these steps simultaneously. 

Of course, technical products are related to the setup of the policy model. But in reality, we can start with listing the basic technical product framework so as to generate DD. This approach allows a smooth parallel work between business configuration and system configuration, which can be managed by different teams.

### 3. Code Table Versioning

**Question:**
Is there any version control for code table configuration?

**Answer:**
No. Code tables prioritize data integrity as a first principle due to their relation to meta-data maintenance. Sometimes flags can be used to indicate whether any volume is obsolete, especially for those with a migration strategy. But this is different from version control.

We have a configuration module for the ratio tables or decision tables in need of versioning.

### 4. Data Table API Integration Checklist

**Question:**
If I have a data table and clients want to use our API to develop their own master table maintenance UI, what do I need to consider?

**Answer:**
Firstly, they are all data table behavior not related to code table.

For data table maintenance, there are several reminds users need to know:

*	Update records remotely:

     * Flag **update-records-remotely** to **Yes** if the table is not defined in portal MC environment.
     
*	Differentiate between small data table and big data table:

     * Small table: No constraints; the search API returns all records.
     
     * Big table: Several behaviors to note.

       * Specify at least one index field.
       
       * Index fields are mandatory search criteria, and additional non-index fields can be used for joint search criteria.
       
       * All searches must be exact; no fuzzy searches are allowed.
       
       * If the search result with index fields as criteria exceeds 1,000, the system will return a maximum of 1,000 results. Provide more search criteria to view records beyond 1,000.

### 5. Can multi-language data be “structure-only, without synchronization”?

**Question:** 
Does the CodeTable’s multi-language data have an “Update Records Remotely” attribute or feature that allows configuration of only the structure in MC without handling the data?  

**Answer:** 
Currently not supported. You can group the CodeTables that need to be maintained at runtime separately and prevent MC from exporting the i18n data for that group.

### 6. How to synchronously clear local cache across multiple nodes?

**Question:** 
At runtime, external APIs can modify the Data Table, Code Table, and Configuration Table in real time. After data synchronization, we can call a method to clear the local cache. However, in a production environment with multiple nodes, once node A clears its cache, how should the local cache on node B be handled?  

**Answer:** 
The cache layer already incorporates an MQ-based broadcast mechanism. When any node calls the cache eviction method, a message is broadcast; other nodes receive it and automatically clear their local caches. See LocalCacheWrapper for details.


