Converting Logs into JSON Format

Product: FID

Question: Can we change default logging format of logs into JSON format.

Answer: Yes. This is possible. We use Log4j2 which provides many different configuration options, including JSON formatting for logs.

Steps to Change the format: 

Note: In this article, only an example was provided but options to format the json file will depend on each client requirement and have to be performed at your end.

 Edit Log4j2 configuration file in ZooKeeper

1. In Expert Mode (from Control Panel) -> Navigate to ZooKeeper Tab. Open log4j2-vds.json under           radiantone/v2/<cluster_name>/config/logging.

2. Change the server.log.pattern value to  "%d{ISO8601} %-5p %C:%L - $$${rli:exec.context}%m%n"

{
"name" : "server.log.pattern",
"value" : "%d{ISO8601} %-5p %C:%L - $$${rli:exec.context}%m%n"
},

3. In the same file, navigate to appenders. Change pattern layout to JsonLayout as shown in the below snippet.

"appender" : [ {
"type" : "RollingFile",
"name" : "VDSServerAppender",
"JsonLayout" : {
"KeyValuePair" : [ {
"key" : "timestamp",
"value" : "$${date:yyyy-MM-dd'T'HH:mm:ss.SSSZ}"
}, {
"key" : "myCustomField",
"value" : "Hi this is a custom field example"
} ],
"complete" : true,
"compact" : false,
"eventEol" : true,
"objectMessageAsJsonObject" : true,
"stackTraceAsString" : true,
"includeNullDelimiter" : true
},
"fileName" : "${server.log.file}",
"filePattern" : "${server.log.file.archive}",
"SizeBasedTriggeringPolicy" : {
"size" : "${server.log.file.maxSize}"
},

The output server.log will look like this:

"instant" : {
"epochSecond" : 1681144284,
"nanoOfSecond" : 55000000
},
"thread" : "#Replication# <-- Detect thread for domain 'o=hdap'(7576) -- store",
"level" : "DEBUG",
"loggerName" : "com.rli.nc",
"message" : "[HDAPstoreEnumeration cn_replicationjournal] closed with entry count=0",
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
"threadId" : 209,
"threadPriority" : 5,
"timestamp" : "2023-04-10T09:31:24.055-0700",
"myCustomField" : "Hi this is a custom field example"
}
, {
"instant" : {
"epochSecond" : 1681144284,
"nanoOfSecond" : 55000000
},
"thread" : "#Replication# <-- Detect thread for domain 'o=hdap'(7576) -- store",
"level" : "DEBUG",
"loggerName" : "com.rli.slapd.replication.domain.i",
"message" : "... [Replication] Replication domain {o=hdap} detected changes: 0",
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
"threadId" : 209,
"threadPriority" : 5,
"timestamp" : "2023-04-10T09:31:24.055-0700",
"myCustomField" : "Hi this is a custom field example"
}

Attached node (radiantone/v2/<clustername>/config/logging/log4j2-vds.json) details for reference.