Parsing Logs using Scala

spark

#1

Hi All,

how can we parse below kind of logs using Scala. I want to read this kind of data and put that into Hive table.

log timestamp=“2018-04-06T22:43:19.565Z” eventCategory=“Application” eventType=“Error”

log contents are actually in HTML tag of < />
Please advise.


#2

Please share the exact structure of log message. If you are having HTML Tags, then you can come up with solution using regular expressions.

But in real projects information will not be logged in HTML format, it will be logged in the form of standard log4j message or other formats like JSON.


#3

below is the log message structure:
< log timestamp=“2018-04-06T22:43:19.565Z” eventCategory=“Application” eventType=“Error” message=“Requestor:,AppName:AuthRulesService,MethodName:AuthenticateUser” />