Enhance Log Management with Utility Service Logging

Introduction

Application logging and error handling are critical parts of log management and help a business run smoothly. If you don’t have any proper logging records, it is difficult to identify the events, making troubleshooting a long and time-consuming process.

Generally, developers use different logging practises to log the events, but in case of a mishap, it is a default to find the required data that is helpful for troubleshooting. This again requires a long process of additional coding or debugging to derive the relevant data.

To overcome the above scenarios, we have developed a framework for utility service error handling. This essentially standardises the message logging and exception handling scenarios in any kind of MuleSoft API. The out-of-box utility service minimises developer efforts by 10–15%.

What is utility service?

Utility Service is a framework that logs the messages with all the required information, which makes troubleshooting easier. It has the capability to log the message in different formats.

Services offered by the utility service:

It supports different log formats.
Maintains one transaction across different layers (system, process, and experience). Using this, one can easily track the transaction from start to finish.
No additional formatting or logging is required, and it is easily publishable to data aggregation platforms like Splunk, ELK, and others.
It calculates the latency from start to end across layers, which is useful to monitor the performance of the application.
Request level details can be tracked using the RequestID and FunctionalID parameters. The RequestID will be unique for different layers.
Additionally, it supports up to 10 context keys to log any other information.
It propagates the actual error details to the consumer, even in the case of an API-led approach.
One can configure it to log the original and errored payloads.
Logs the source and target names, which are helpful for the triage team.

Different formats supported by the utility service

JSON
XML
CSV

Setting up the utility service

Using the Runnable Jar:

1. Download the utility service jar.

2. Include the Utility service library in maven local repository using the below command

mvn install:install-file -Dfile=\<path>\co-utility-service-v1-1.0.0-mule-application.jar -DgroupId=com.co -DartifactId=co-utility-service-v1 -Dversion=1.0.0 -Dpackaging=jar -DgeneratePom=true

3. Add the below dependency to your mule project pom.xml

<dependency>

<groupId>com.co</groupId>

<artifactId>co-utility-service-v1</artifactId>

<version>1.0.0</version>

</dependency>

4. Import “co-utility-service.xml” from the utility service into your project global elements and use the utility service for logging and error handling

Code snippet

Using utility service

The utility service is simply used by calling the flows defined using Flow Reference. From here, it automatically handles the errors and logs the transaction in the specified format.

Prerequisite properties to configure:

Property	Usage	Supported values	Example
app.sysname	Application Name	Any	co-utility-service
app.version	Application Version	Any	V1
app.source	Source Name	Any	SFTP
app.target	Target Name	Any	Salesforce
app.log.originalPayload	Logs the original payload from the source if set to true	true/false Default: false	true
app.log.erroredPayload	Logs the errored-out payload in case of error if set to true	true/false Default: false	true
app.log.format	Logs the message in the specified format	json/xml/csv Default: json	json

The utility service logs the below information

Log Fields	usage
serverName	Name of the server where application hosted
appName	Name of the Application
appVersion	Application Version
environment	Environment of the application
source	Source Name
target	Target Name
transactionId	Transaction Id
transactionState	Transaction state
requestId	Request Id
executionState	Execution State
serviceResource	The end point called
serviceMethod	Method of the resource
payload	Payload
logTimeStamp	Current Time Stamp
logMessage	Logs the Message provided
functionalId	Any unique ID from the Request
contextKey1 .. contextKey10	Any additional details for logging
logLevel	Log Level
latency	Total time taken for the process

List of flows defined in the utility service

loggerStart
loggerInProgress
loggerComplete

LoggerStart

The logger start flow is the initial step of the utility service. The loggerStart initiates the details required for the transaction; it is called after the source.

Steps to configure:

Drag and drop the flow reference after the source.
Select loggerStart as a flow name from the drop-down.
Configure the app.log.originalPayload property to log the original payload if required.
Configure log format and other properties as per requirements.

Snippet of LoggerStart

After the logger starts, the utility service logs the transaction and creates two new variables. These variables can be reused in further processes if required.

Variable Name	Usage
logDate	Logs the initial details of the transaction, which will be used in later processing of the transaction
serviceStartTime	Initial transaction time

Below is a snippet of the variable generated

{
    "serverName": "LAPTOP-KALVA",
    "appName": "demo-sample-papi",
    "appVersion": "v1",
    "environment": "dev",
    "transactionId": "a70610b0-0764-11ed-a20a-d20f0c30c779",
    "transactionState": "Transaction_Started",
    "requestId": "1aa35a63-6b92-4ac2-ac11-2e546160c31e",
    "executionState": "Started",
    "serviceResource": "/papi/orders",
    "serviceMethod": "GET",
    "payload": "{\n  \"orderId\": 987456321,\n  \"customerId\": \"12934\",\n  \"items\": [\n    \"ABB-123\",\n    \"ACC-452\"\n  ]\n}",
    "logTimeStamp": "2022-07-19T18:44:00+0530",
    "logLevel": "INFO",
    "latency": 0
}

Logger In-Progress

The loggerInProgress flow logs the transaction at different transaction points where it is called.

Steps to configure:

Drag and drop the flow reference where you want to log the message in between the processes. The loggerInProgress can be called multiple times if you want to log the message at different trace points.
Select loggerInProgress as a flow name from the drop-down.
Set variables like logger message, context keys, and function keys before logging any additional information.

Utility services use different conditional variables to log different types of request details

Variable Name	Type	Usage
logPayload	Boolean	Logs the payload if set to true
functionalID	String/Number	Any unique ID from the request to track the transaction
logMessage	String	Transaction points like “before dataweave”
contextKey1 .. contextKey10	String/Object	Any additional details for logging

Snippet of loggerInProgress

The sample transaction log for loggerInProgress is

{
    "serverName": "LAPTOP-KALVA",
    "appName": "demo-sample-papi",
    "appVersion": "v1",
    "environment": "dev",
    "transactionId": "a70610b0-0764-11ed-a20a-d20f0c30c779",
    "transactionState": "Transaction_InProgress",
    "requestId": "1aa35a63-6b92-4ac2-ac11-2e546160c31e",
    "executionState": "InProgress",
    "serviceResource": "/papi/orders",
    "serviceMethod": "GET",
    "logTimeStamp": "2022-07-19T18:44:00+0530",
    "functionalId": 123,
    "contextKey1": "test",
    "logLevel": "INFO",
    "latency": 297
}

Logger Complete

The logger-complete flow should be called at the end of the process.

Steps to configure

Drag and drop the flow reference at the end of the flow.
Select loggerComplete as a flow name from the drop-down.
Set variables like logger Message, context keys, and function keys before logging any additional information

loggerComplete will calculate the latency of the transaction from start to end.

Snippet of logger complete

The sample transaction log for logger complete

{
    "serverName": "LAPTOP-KALVA",
    "appName": "demo-sample-papi",
    "appVersion": "v1",
    "environment": "dev",
    "transactionId": "76430b30-0765-11ed-a20a-d20f0c30c779",
    "transactionState": "Transaction_Completed",
    "requestId": "fdb4df04-531f-4c4a-b66c-3acf1c5097f7",
    "executionState": "Completed",
    "serviceResource": "/papi/orders",
    "serviceMethod": "GET",
    "logTimeStamp": "2022-07-19T18:49:47+0530",
    "functionalId": 123,
    "contextKey1": "test",
    "logLevel": "INFO",
    "latency": 52
}

If you’re looking to streamline your business operations with better log management, feel free to get in touch with us at Cloud Odyssey. Our team of technical experts with decades of experience in the industry can give your business a full uplift!