JSON logging

UnrealIRCd 6 introduces optional JSON logging in log files and in server notices sent to snomasks. This makes it easy to parse and process log messages by programs and bots.

The JSON log also contains a lot more detail than the text log because it includes, for each user, things like realname, account, operlogin, geoip information, tls information, channels, etc.

JSON Example output

This is an example of the JSON log output when someone types JOIN #five:

Enabling in disk logging

Usually you will want to keep your ircd.log logging as plaintext (non-JSON) because it is easy to read by humans.

So, just add another NEW log block with JSON logging enabled, to for example ircd.json.log, like this:

log {
        source {
                all;
                !debug;
        }
        destination {
                file "ircd.json.log" { maxsize 100M; type json; }
        }
}

As you can see, you simply set the log::destination::file::type to json.

And then /REHASH.

Colorized JSON

The JSON logged to the file is not nicely formatted and does not contain color. The colorized JSON log as shown in the screenshot on this page is generated with the tool jq. So this is all completely optional, just mentioning it for anyone interested in seeing it colorized:

1. Install jq, eg through: sudo apt-get install jq
2. Then tail (or cat) the log through it, eg: tail -F ~/unrealircd/logs/ircd.json.log|jq

Enabling on IRC

IRCOps receive server notices through snomasks. It is possible to receive the associated JSON logging data on IRC as well.

No IRC client supports this natively out of the box, so here is a technical explanation:

1. Send CAP REQ unrealircd.org/json-log to indicate that you want to receive JSON logging, eg /QUOTE CAP REQ unrealircd.org/json-log
2. Become IRCOp using /OPER (if you are not one already)
3. Open up a debug window in your client so you can see raw IRC traffic. Eg: /debug -pt @debug in mIRC. Other clients typically have something similar.
4. Now when you receive a server notice via snomasks (for example a new user connecting), then the JSON data will be sent in the unrealircd.org/json-log message tag.

In mIRC scripting the JSON data in the server notice will be in $msgtags(unrealircd.org/json-log)

Your bot/program can then parse this as JSON data and process it. The format is much more consistent than regular text messages and typically shows a lot more details about the objects involved (client, channel, etc)

Example on IRC

First, request the capability, oper up, and for this example we will also set the join snomask (+s +j):

CAP REQ :unrealircd.org/json-log
OPER ...........
MODE Yournick +s +j

Then when another client types JOIN #five we will receive the following raw server traffic:

@unrealircd.org/json-log={"timestamp":"2022-05-23T11:02:13.519Z","level":"info","subsystem":"join","event_id":"LOCAL_CLIENT_JOIN","log_source":"maintest.test.net","msg":"User\sSyzop\sjoined\s#five","client":{"name":"Syzop","id":"001F9RV02","hostname":"xyz.example.org","ip":"198.51.100.1","server_port":6697,"client_port":37030,"details":"Syzop!~x@xyz.example.org","connected_since":"2022-05-23T11:02:06.000Z","idle_since":"2022-05-23T11:02:06.000Z","user":{"username":"~x","realname":"IRC\sUser","vhost":"oper/netadmin.test.net","cloakedhost":"Mask-8608861.example.net","servername":"maintest.test.net","reputation":12,"security-groups":["unknown-users","tls-and-known-users","tls-users"],"modes":"iotxz","geoip":{"country_code":"NL"},"snomasks":"bcdfkoqsBOS","operlogin":"Syzop","operclass":"netadmin-with-override","channels":["#five","#four","#three","#two","#one"]}},"tls":{"certfp":"aafe66a7d808e1fca077805c54b1274a92d30c3023e35ec130f358d238218296","cipher":"TLSv1.3-TLS_CHACHA20_POLY1305_SHA256"},"channel":{"name":"#five","creation_time":"2022-05-23T11:02:13.000Z","num_users":1,"modes":"ntf\s[4j#R1,5m#M1,3n#N1,3t#b1]:2"},"modes":"o","source":{"file":"join.c","line":292,"function":"_join_channel"}}
:maintest.test.net NOTICE oper :join.LOCAL_CLIENT_JOIN [info] User Syzop joined #five

With the help of a JSON library the client can then decode it to objects.

UnrealIRCd JSON data

Every JSON log message contains at least the following data:

Variable	Description	Example value
timestamp	Date/time when the log message was generated	2022-05-23T11:02:06.000Z
level	Severity of the log message	info
subsystem	Subsystem which generated the log message	join
event_id	Event ID of the log message	LOCAL_CLIENT_JOIN
log_source	Servername which generated the log message. This is because UnrealIRCd will receive various log messages from remote servers.	irc1.example.net
msg	The human readable message (this is used in plaintext logs and snomasks).	User Syzop joined #five

In addition to that there are a couple of fields that are often included (but not always!):

Variable	Description	Example value
client	The client causing the event. For example, for a KICK this would be the person issuing the /KICK command.	See client object
target	The client being the victim of the event. For example, for a KICK this would be the victim.	See client object
channel	The channel where the event is taking place.	See channel object

There are many possible log messages and the additional fields that are available for each event differ. You can look at the List of all log messages for a particular event to see how it is generated. Or you can capture real log items or IRC data.

client object

This can be a user, a server or a freshly accepted connection that is still in the handshake and we don't know yet what it will be.

Variable	Description	Example value
name	The name of the client (nick name, server name, ..)	Syzop
id	The unique client ID	001F9RV02
hostname	Resolved hostname (or IP)	xyz.example.org
ip	IP address of the user (empty for services)	198.51.100.1
server_port	Server TCP port (unrealircd side)	6697
client_port	Client TCP port (client side)	37030
details	Detailed description of the client. The output depends on the type of client and what information we have.	nick!user@host irc1.example.org@198.51.100.1 [198.51.100.1]
connected_since	Date/time when the client connected (only available for local clients)	2022-05-23T11:02:06.000Z
idle_since	Last time the client said anything in PM or channel (only available for local clients)	2022-05-23T11:02:06.000Z
tls	If the client is using SSL/TLS then this contains a tls object which has two members	(see next)
tls.cipher	The TLS cipher negotiated with the client	TLSv1.3-TLS_CHACHA20_POLY1305_SHA256
tls.certfp	The client Certificate fingerprint (if any)	aafe66a7d808e1fca077805c54b1274a92d30c3023e35ec130f358d238218296
geoip	GeoIP information regarding the IP address of the user (not always available)	(see next)
geoip.country_code	The ISO 3166-1 alpha-2 country code	NL
user	If the client is a user (a person) then additional information is available in this object.	See client.user object
server	If the client is a server then additional information is available in this object.	See client.server object

See also below for the user and server objects.

client.user object

If the client is a user then there is a user object within the client object:

Variable	Description	Example value
user.realname	The real name / GECOS	IRC User
user.vhost	If the user is +x or +t this contains the vhost/cloakedhost	oper/netadmin.test.net
user.cloakedhost	The calculated cloaked host, even if the user is not +x	Mask-8608861.example.net
user.servername	The server to which the user is connected	irc.example.net

client.server object

If the client is a server then there is a server object within the client object:

Variable	Description	Example value
server.info	The server description	Primary IRC server of example.net
server.num_users	Number of currently connected users	123
server.boot_time	Date/time the server was started	2022-05-23T11:02:06.000Z
server.synced	Set to 1 if the server is synced, 0 if the server is currently linking in and syncing users/channels/etc.	1
server.features.software	Server software version in use	UnrealIRCd 6.0.0
server.features.protocol	UnrealIRCd protocol version	6000
server.features.usermodes	User modes supported by this server	abcd
server.features.chanmodes	Channel modes supported by this server. This is an array of 4 strings, the same format as IRC numeric 005 CHANMODES=.	['abc','def','ghi','jkl']
server.features.nick_character_sets	Nick Character Sets supported by this server. Comma separated string.

channel object

Variable	Description	Example value
name	The name of the channel	#five
creation_time	Date/time the channel was first created	2022-05-23T11:02:13.000Z
num_users	Number of users in the channel	1
topic	Topic of the channel (if a topic is set)	Welcome everyone
topic_set_by	Name of the user who set the topic (if a topic is set)	Syzop
topic_set_at	Date/time when the topic was set (if a topic is set)	2022-05-23T1:30:00.000Z
modes	Channel modes that are set (not including list modes)	ntf [4j#R1,5m#M1,3n#N1,3t#b1]:2

When using the channel.get API call, more details are available:

Variable	Description	Example value
members	List of members (nicks) that are in the channel	["One", "Two", "Three"]
bans	List of all bans (+b) in the channel	[{"name": "some!nice@ban","set_by": "some_user","set_at": "2023-01-04T07:52:54.000Z"}]
ban_exemptions	List of all ban exceptions (+e) in the channel	[{"name": "some!nice@exempt","set_by": "some_user","set_at": "2023-01-04T07:52:54.000Z"}]
invite_exceptions	List of all invite exceptions (+I) in the channel	[{"name": "some!nice@invex","set_by": "some_user","set_at": "2023-01-04T07:52:54.000Z"}]

socket_error object

Included in socket errors. Such as a socket being closed due to a network problem or otherwise.

Variable	Description	Example value
error_code	The error code as an integer	5
error_string	The error as a string	Input/output error

NOTE: The error codes and strings come directly from the OS and C library, hence they will differ between OS's (Linux, Windows, FreeBSD, ..).

tls_error object

Included in SSL/TLS errors. Such as failure to read a certificate file or a socket being closed due to a network problem.

Variable	Description	Example value
name	The first encountered error as a printable string, for convenience. This will be identical to error_stack[0].string.	5
error_stack	An array of items on the OpenSSL "error stack"	(see next)
error_stack[].code	The OpenSSL error code as an integer	5
error_stack[].string	OpenSSL error as a string	???

NOTE: The error codes and strings come directly from OpenSSL (outside our control).

link_block object

This is the content of a Link block. It is typically included in server linking errors, but only if it is known which server to link to/from.

Variable	Description	Example value
name	Name of the link block, as in link irc1.example.net { }	irc1.example.net
hostname	link::outgoing::hostname for outgoing connects (may not be present)	irc1.example.net
hostname	Resolved IP for outgoing connects (may not be present)	198.51.100.1
file	link::outgoing::file for outgoing connects (not often used, likely not present)	/tmp/server.sock
port	link::outgoing::port for outgoing connects (may not be present)	6900
bind-ip	link::bind-ip for outgoing connects	198.51.100.2
class	link::class	servers

tkl object

This is a server ban, such as a GLINE or Spamfilter. It is also used for ban exceptions (ELINE).

Variable	Description	Example value
type	Type of the server ban. One of: gline, kline, gzline, zline, spamfilter, qline, except, shun, local-qline, local-exception, local-spamfilter.	gline
type_string	Type of the server ban in a more friendly user printable string. Possibly prefixed with "Soft" eg "Soft G-Line".	G-Line
name	The target of the ban or except. For Spamfilter this is the regex/matcher.	*@*.badisp.example.net
set_by	Name of the person or server who set the ban	Syzop
set_at	Date/Time when the server ban was added	2022-05-23T11:02:06.000Z
set_at_string	Date/Time when the server ban was added in a more human printable string.	Mon May 23 11:02:06 2022
expire_at	Date/Time when the server ban will expire. NULL means: never.	2023-05-23T10:00:00.000Z
expire_at_string	Date/Time when the server ban will expire in a more human printable string. Uses "Never" for never.	Tue May 23 10:00:00 2023
duration_string	How long the ban will last from this point in time (human printable). Uses "permanent" for forever.	permanent
set_at_delta	How many seconds ago the ban was placed.	1800
reason	The reason of the ban	Lots of abuse from this ISP
exception_types	Only for ban exceptions! Specifies the exception types (letters). See HELPOP ELINE for a list of those letters.	kGzZsF
match_type	Only for spamfilters! The matching type. One of: simple, regex	regex
ban_action	Only for spamfilters! The action to take on spamfilter hit.	gline
spamfilter_targets	Only for spamfilters! Which targets the spamfilter must filter on.	cpnN