Introduction

Vtiger’s Data Source Designer and the Virtual module address the data integration issue. 

Data Source Designer is a part of VTAP. The Data Source Designer allows you to connect with other applications and integrate data as a part of Vtiger. This feature enables you to build connectors for various actions in Vtiger using standard REST APIs. The module provides a rich user interface to configure URLs, parameters, and authentication details.

Connector Functions

Each function in the connector performs specific actions in the CRM, such as:

• Get Records: Lists application data in a record list.
• Get Record: Displays a specific record.
• Create Record: Allows the creation of information in your application from Vtiger.
• Update Record: Enables users to update information within Vtiger.
• Delete Record: Deletes data from your application.
• Related Records: Connects Vtiger information with application data, e.g., linking contact phone numbers with call logs.
• Search: Searches application data.
• Navigation: Facilitates moving through data lists in Vtiger.

Vtiger’s Data Source Designer is built on the concept of Zero-Copy Integration. Let us learn about Zero-Copy Integration.

Concept of Zero-Copy Integration

According to a survey, large enterprise companies end up using, on average, 367 applications. Their employees spend one-third of the week finding the data and information required to do their jobs effectively. Each of these multiple applications creates and stores data in its database leading to data silos and hundreds of integrations, thereby increasing the complexity and costs.

Apart from this, we are still copying sensitive data to multiple places. This duplication of data results in regulatory risks, security threats, and privacy liabilities.

This is where Zero-Copy integration is gaining traction. It is increasingly being used to mitigate risks and enable the easy use of data across applications. This gives you the power to work with the data itself, not the applications which carry it. By eliminating data duplication and ensuring real-time access, zero-copy integration fosters better data quality, improves operational efficiency, reduces storage overhead, and empowers sales teams with a more holistic view of the customer. 

Use cases

• Real-Time Inventory Visibility
• Financial Data Integration
• Enriched Customer Profiles
• Fraud Detection and Risk Management
• Social Media Listening

Configuration in the Data Source Designer

Each action requires configuring URLs, headers, query, and body parameters, typical in any HTTP API call. Once connected, the data appears to be residing within Vtiger.

Merge Tags

The interface supports dynamic values using merge tags, such as:

• $filter.pagelimit$: Accesses the number of records displayed on the list page.
• $filter.currentpage$: Accesses the current page number.
• $filter.nextpage$: Accesses the next page number.

Configure Fields

Configuring fields is crucial. Each field has a display label and an internal name mapping to API data. Field names can be used in the UI to create or update information. For instance, an email field can be accessed using $input.email$ in API calls.

Code Editor

For complex data handling, a code editor is available to write scripts in JavaScript. This allows for data transformation before being displayed in Vtiger. The editor supports HTTP calls to external applications and fetching Vtiger data.  For more details, read our document on Server-side Scripting, an approach we follow at Vtiger.  

Once the configuration is done, you can test the connection using the Test button next to the Code Editor. You can:

• Use the Response tab to view the response of the API call
• Use the Log tab for debugging. Use the log function for debugging purposes.

Each function defined inside a main function comes with a contextual parameter. This parameter has details relative to the action performed. For example, when you are on the list page, you will get which page you are on, if any filter/search is applied, etc. 

Common parameters available for all functions include:

• URL : Application API URL configured in the UI..
• headers : Key-value map of header information.
• queryparameters :  Key-value map of URL query parameters.
• body :  Key-value map of parameters passed to the form or request body.
• username, password, bearer: Available if you are using authentication.
• id : Available when viewing detailed information with Record Identifier.
• filter : This is available when you are searching in the list view.
• currentpage/page : This will give the current page number
• pagelimit : It gives a number of records to be displayed in the list view.
• nextpage : It gives the next page number in the list view.   
• sortcolumn : It gives a field name that is to be sorted.
• sortorder : It gives the order in which sort is to be applied.
• pagenavdir : This gives next for next page navigation and previous when you go back to list views. This is useful when you do not depend on page number for navigation.
• parentid and parentmodule : these are available when you are relating data to existing modules and defining the Related records function.

Additional parameters can be sent as part of the response and used for state management, which are available in extra parameters.

Building Virtual Modules

Creation

Once the connector is built and published, define a Virtual module in Vtiger to map the data exposed by the connector. Virtual modules have no data but possess a schematic structure of the data.

Configuration

When creating a Virtual module, provide:

• Module Name - Name of the new module.
• App Name - App name where your module will reside.
• Data Source - Select one of the published data sources.

Test the connection after supplying authentication details. If successful, fields are automatically mapped, with options to create a Record Identifier and enable editable properties for data points.

Connection and Field Mapping

After selecting the Data Connector, you will be prompted to test the connection by providing the relevant authentication details. If the connection is successful, the fields where the data needs to reside are automatically mapped. 

During this mapping process, the system will auto-select the appropriate field types and offer the following options:

• Record Identifier: Create a non-editable identifier for the application data.
• Editable Property: Enable the editable property for data points that you want users to be able to edit.

The data, when used in silos, is not very useful, but connecting them to Vtiger will make them very powerful. You can relate data to existing modules. This will make use of the Related Records function to extract the information. Once the mapping is done, you can create the module and see it in action.

Use with Prompt Builder

Suppose the Calculus module is installed, and the connector defines the Related Records function. In that case, AI can leverage data to enhance application synergy by including the data in the prompt builder.

Control Access

You can control the visibility of these Virtual modules just like other modules and restrict the access to create/update certain roles by using Roles and Profiles on the Settings page.

Module Designer

All Virtual modules are available in the module designer to enable you to customize and add more functionality to your use cases.

Limitation

This module is currently not available in many of the internal features like:

• Reports
• Workflows
• Module layout and fields
• Approvals
• Business Process Designer
• Email and Print Templates

Examples

Example 1: Show Hotel Bookings in Vtiger, and give the option to make changes in the reservations.

First, we will bring all the bookings into Vtiger and show them in a Virtual module named Bookings.
Let us begin by building a Data Connector first. To access all the bookings, we must first configure endpoints and map fields. We are mapping the Get Records function to hotelbooking/bookings endpoint with a Token used for authentication.




 

Field mapping is done based on the response given by the endpoint. Here is the sample of the response given by the endpoint, for which the field mapping is done by extracting keys that hold the data.





 

Once this mapping is done, we can save and publish the connector and create a new Virtual module for the Data Connector Booking system.

Give the appropriate name for the module and click on the Test button to check for connection. Once successful, we will automatically map the fields by their types. You can select one of them as a Record Identifier, which will be used to update/delete/view the booking details. Editable property enables you to control the booking updates, so mark only those that are needed. In our case, we will enable only the status, date, and number of members to be updated in Vtiger.



 

Save the module and head over to the menu to select the Hotel Bookings module under the Inventory app to see the booking details.




Now let’s connect this to Contacts in Vtiger. We must bind the Hotel Bookings Email address to the Contacts Email address. Let's first add the relationship between Hotel Booking and Contact. Go to Platform > Data Source Designer > Settings > Virtual Module. Select Hotel Bookings and add Contacts under Add Relationship. 

We must define a Related Records function in the Data Connector to enable this relationship.
We need to set the URL and the endpoints. We expect the email parameter, Contact’s email address, to be available in $input merge tag, so we use $input.email$, which will be replaced with the Contact's email address at the run time. 




Save the Data Connector and go to any Contacts module detail view to see all the customer bookings under Hotel Bookings.



Likewise, we can enable user agents to make updates to these bookings. We need to define an Update Record function to enable other actions from within Vtiger.
 

Example 2: Show customers payments from Stripe account inside Vtiger.

Customer payments are stored inside the Charges object in Stripe. Each Charge object is internally linked to the Stripe Customer object. Customer objects have few properties like name, email, phone number, etc. 

You can map this to the Contacts module’s Email field in Vtiger. So inside the Contacts module, add a relationship with the new Stripe Charges Virtual module and show all the payments made by the customer in real time. You also need to connect to Stripe using bearer authorization and procure an API key.

Let's first build a Data Connector to Stripe and access the Charges object. Here is the curl request to connect to Charges:

 

Create a new Data connector by selecting the Type as Rest and Auth Type as Bearer Token. Go to the Define Functions section. Under the Get Records function, add the URL as https://api.stripe.com/v1/charges and click on Test. You can see the data that comes from Stripe in the Response tab. 





This information is not directly consumable. So you need to transform it by writing code. You also need to use details such as amount, currency, customer, and a few more details. So we will extract only those details.
 

nextPageExists is used to enable the navigation button in the List View. ending_before and starting_after are used to retain the state information which will be used in subsequent calls.

Let's save and publish the connector. Before saving, you need to save the Fields. Remember to map Field Names properly with response records keys since they are case-sensitive.





Now let's go to the Settings cog icon in the Data Connector list page and create a Virtual Module. Once the correct token is given the Field Mappings are automatically filled and extracted from Stripe Charge Data Connector.



 

Set Stripe Charges ID as the Record Identifier as this will be used to access each individual Charge when defining the Get Record function. This function will be explained later. You can also see another property - Editable, which will be used when creating/updating Charges.

We want these charges to be shown for the Contacts module. So add a relationship with the Contacts module at the bottom of field mapping. Save and proceed. You should be able to see the Stripe Charge module under the Inventory app in Vtiger CRM.


 

Now let us display these inside the Contacts module. The charges are linked to Customers using the Customer ID. You can identify these charges either with the phone number or email address of the Customer object in Stripe.

For this, you need to define the Related Records functions in the Data Connector. You need to search the Charges objects with customers. In this case, the customer is identified by their Stripe ID. In Vtiger you can use the Contact’s Email Address to search in Stripe. You can extract its customer ID reference and then search in Charges Object. 

Steps to get Charges for an email from the Contact’s email address in Vtiger:

1. Fetch the Contacts email address using the parentid which points to Vtiger Contact ID.
2. Fetch Stripe’s Contact details using the email address.
3. Fetch Charges using the Stripe Contact ID.

Now let's enable creating, updating, and deleting charges from Stripe within Vtiger. For creating Stripe Charge, you can take input from the Vtiger form using the $input merge tag. Before this ensure the fields are marked as editable when you create the Virtual Module.

Here we need to send a post request to the Charge API with the input values in the body. Since the API is straightforward you need not write a code for this. Likewise, you will need to do the same for updating the Charge record.




 

When moving from one page to another, Stripe expects you to send parameters like starting_after and ending_after to show the next and previous pages. So you need to send this information when loading the list page in the Get Records functions. These will be available in the context of the Navigation Functions.

Finally, how do you enable filtering in the List View? For this, you have the Search Function. Context has a filter that needs to be converted into a format that Stripe understands.