Tip of the day: Almost every channel mode can be disabled. Don't like halfops? Use blacklist-module chanmodes/halfop;

JSON logging

From UnrealIRCd documentation wiki
Jump to navigation Jump to search

UnrealIRCd 6 introduces optional JSON logging in log files and in server notices sent to snomasks. This makes it easy to parse and process log messages by programs and bots.

The JSON log also contains a lot more detail than the text log because it includes, for each user, things like realname, account, operlogin, geoip information, tls information, channels, etc.

JSON Example output[edit]

This is an example of the JSON log output when someone types JOIN #five:

Unreal json example.png

Enabling in disk logging[edit]

Usually you will want to keep your ircd.log logging as plaintext (non-JSON) because it is easy to read by humans.

So, just add another NEW log block with JSON logging enabled, to for example ircd.json.log, like this:

log {
        source {
                all;
                !debug;
        }
        destination {
                file "ircd.json.log" { maxsize 100M; type json; }
        }
}

As you can see, you simply set the log::destination::file::type to json.

And then /REHASH.

Colorized JSON[edit]

The JSON logged to the file is not nicely formatted and does not contain color. The colorized JSON log as shown in the screenshot on this page is generated with the tool jq. So this is all completely optional, just mentioning it for anyone interested in seeing it colorized:

  1. Install jq, eg through: sudo apt-get install jq
  2. Then tail (or cat) the log through it, eg: tail -F ~/unrealircd/logs/ircd.json.log|jq

Enabling on IRC[edit]

IRCOps receive server notices through snomasks. It is possible to receive the associated JSON logging data on IRC as well.

No IRC client supports this natively out of the box, so here is a technical explanation:

  1. Send CAP REQ unrealircd.org/json-log to indicate that you want to receive JSON logging, eg /QUOTE CAP REQ unrealircd.org/json-log
  2. Become IRCOp using /OPER (if you are not one already)
  3. Open up a debug window in your client so you can see raw IRC traffic. Eg: /debug -pt @debug in mIRC. Other clients typically have something similar.
  4. Now when you receive a server notice via snomasks (for example a new user connecting), then the JSON data will be sent in the unrealircd.org/json-log message tag.

In mIRC scripting the JSON data in the server notice will be in $msgtags(unrealircd.org/json-log)

Your bot/program can then parse this as JSON data and process it. The format is much more consistent than regular text messages and typically shows a lot more details about the objects involved (client, channel, etc)

Example on IRC[edit]

First, request the capability, oper up, and for this example we will also set the join snomask (+s +j):

CAP REQ :unrealircd.org/json-log
OPER ...........
MODE Yournick +s +j

Then when another client types JOIN #five we will receive the following raw server traffic:

@unrealircd.org/json-log={"timestamp":"2022-05-23T11:02:13.519Z","level":"info","subsystem":"join","event_id":"LOCAL_CLIENT_JOIN","log_source":"maintest.test.net","msg":"User\sSyzop\sjoined\s#five","client":{"name":"Syzop","id":"001F9RV02","hostname":"xyz.example.org","ip":"198.51.100.1","server_port":6697,"client_port":37030,"details":"Syzop!~x@xyz.example.org","connected_since":"2022-05-23T11:02:06.000Z","idle_since":"2022-05-23T11:02:06.000Z","user":{"username":"~x","realname":"IRC\sUser","vhost":"oper/netadmin.test.net","cloakedhost":"Mask-8608861.example.net","servername":"maintest.test.net","reputation":12,"security-groups":["unknown-users","tls-and-known-users","tls-users"],"modes":"iotxz","geoip":{"country_code":"NL"},"snomasks":"bcdfkoqsBOS","operlogin":"Syzop","operclass":"netadmin-with-override","channels":["#five","#four","#three","#two","#one"]}},"tls":{"certfp":"aafe66a7d808e1fca077805c54b1274a92d30c3023e35ec130f358d238218296","cipher":"TLSv1.3-TLS_CHACHA20_POLY1305_SHA256"},"channel":{"name":"#five","creation_time":"2022-05-23T11:02:13.000Z","num_users":1,"modes":"ntf\s[4j#R1,5m#M1,3n#N1,3t#b1]:2"},"modes":"o","source":{"file":"join.c","line":292,"function":"_join_channel"}}
:maintest.test.net NOTICE oper :join.LOCAL_CLIENT_JOIN [info] User Syzop joined #five

With the help of a JSON library the client can then decode it to objects.

UnrealIRCd JSON data[edit]

Every JSON log message contains at least the following data:

Variable Description Example value
timestamp Date/time when the log message was generated 2022-05-23T11:02:06.000Z
level Severity of the log message info
subsystem Subsystem which generated the log message join
event_id Event ID of the log message LOCAL_CLIENT_JOIN
log_source Servername which generated the log message.
This is because UnrealIRCd will receive various log messages from remote servers.
irc1.example.net
msg The human readable message (this is used in plaintext logs and snomasks). User Syzop joined #five

In addition to that there are a couple of fields that are often included (but not always!):

Variable Description Example value
client The client causing the event. For example, for a KICK this would be the person issuing the /KICK command. See client object
target The client being the victim of the event. For example, for a KICK this would be the victim. See client object
channel The channel where the event is taking place. See channel object

There are many possible log messages and the additional fields that are available for each event differ. You can look at the List of all log messages for a particular event to see how it is generated. Or you can capture real log items or IRC data.

client object[edit]

This can be a user, a server or a freshly accepted connection that is still in the handshake and we don't know yet what it will be.

Variable Description Example value
name The name of the client (nick name, server name, ..) Syzop
id The unique client ID 001F9RV02
hostname Resolved hostname (or IP) xyz.example.org
ip IP address of the user (empty for services) 198.51.100.1
server_port Server TCP port (unrealircd side) 6697
client_port Client TCP port (client side) 37030
details Detailed description of the client.
The output depends on the type of client and what information we have.
nick!user@host

irc1.example.org@198.51.100.1
[198.51.100.1]

connected_since Date/time when the client connected (only available for local clients) 2022-05-23T11:02:06.000Z
idle_since Last time the client said anything in PM or channel (only available for local clients) 2022-05-23T11:02:06.000Z
tls If the client is using SSL/TLS then this contains a tls object which has two members (see next)
tls.cipher The TLS cipher negotiated with the client TLSv1.3-TLS_CHACHA20_POLY1305_SHA256
tls.certfp The client Certificate fingerprint (if any) aafe66a7d808e1fca077805c54b1274a92d30c3023e35ec130f358d238218296
geoip GeoIP information regarding the IP address of the user (not always available) (see next)
geoip.country_code The ISO 3166-1 alpha-2 country code NL
user If the client is a user (a person) then additional information is available in this object. See client.user object
server If the client is a server then additional information is available in this object. See client.server object

See also below for the user and server objects.

client.user object[edit]

If the client is a user then there is a user object within the client object:

Variable Description Example value
user.realname The real name / GECOS IRC User
user.vhost If the user is +x or +t this contains the vhost/cloakedhost oper/netadmin.test.net
user.cloakedhost The calculated cloaked host, even if the user is not +x Mask-8608861.example.net
user.servername The server to which the user is connected irc.example.net

client.server object[edit]

If the client is a server then there is a server object within the client object:

Variable Description Example value
server.info The server description Primary IRC server of example.net
server.num_users Number of currently connected users 123
server.boot_time Date/time the server was started 2022-05-23T11:02:06.000Z
server.synced Set to 1 if the server is synced, 0 if the server is currently linking in and syncing users/channels/etc. 1
server.features.software Server software version in use UnrealIRCd 6.0.0
server.features.protocol UnrealIRCd protocol version 6000
server.features.usermodes User modes supported by this server abcd
server.features.chanmodes Channel modes supported by this server.
This is an array of 4 strings, the same format as IRC numeric 005 CHANMODES=.
['abc','def','ghi','jkl']
server.features.nick_character_sets Nick Character Sets supported by this server. Comma separated string.

channel object[edit]

Variable Description Example value
name The name of the channel #five
creation_time Date/time the channel was first created 2022-05-23T11:02:13.000Z
num_users Number of users in the channel 1
topic Topic of the channel (if a topic is set) Welcome everyone
topic_set_by Name of the user who set the topic (if a topic is set) Syzop
topic_set_at Date/time when the topic was set (if a topic is set) 2022-05-23T1:30:00.000Z
modes Channel modes that are set (not including list modes) ntf [4j#R1,5m#M1,3n#N1,3t#b1]:2

When using the channel.get API call, more details are available:

Variable Description Example value
members List of members (nicks) that are in the channel ["One", "Two", "Three"]
bans List of all bans (+b) in the channel [{"name": "some!nice@ban","set_by": "some_user","set_at": "2023-01-04T07:52:54.000Z"}]
ban_exemptions List of all ban exceptions (+e) in the channel [{"name": "some!nice@exempt","set_by": "some_user","set_at": "2023-01-04T07:52:54.000Z"}]
invite_exceptions List of all invite exceptions (+I) in the channel [{"name": "some!nice@invex","set_by": "some_user","set_at": "2023-01-04T07:52:54.000Z"}]

socket_error object[edit]

Included in socket errors. Such as a socket being closed due to a network problem or otherwise.

Variable Description Example value
error_code The error code as an integer 5
error_string The error as a string Input/output error

NOTE: The error codes and strings come directly from the OS and C library, hence they will differ between OS's (Linux, Windows, FreeBSD, ..).

tls_error object[edit]

Included in SSL/TLS errors. Such as failure to read a certificate file or a socket being closed due to a network problem.

Variable Description Example value
name The first encountered error as a printable string, for convenience.
This will be identical to error_stack[0].string.
5
error_stack An array of items on the OpenSSL "error stack" (see next)
error_stack[].code The OpenSSL error code as an integer 5
error_stack[].string OpenSSL error as a string ???

NOTE: The error codes and strings come directly from OpenSSL (outside our control).

link_block object[edit]

This is the content of a Link block. It is typically included in server linking errors, but only if it is known which server to link to/from.

Variable Description Example value
name Name of the link block, as in link irc1.example.net { } irc1.example.net
hostname link::outgoing::hostname for outgoing connects (may not be present) irc1.example.net
hostname Resolved IP for outgoing connects (may not be present) 198.51.100.1
file link::outgoing::file for outgoing connects (not often used, likely not present) /tmp/server.sock
port link::outgoing::port for outgoing connects (may not be present) 6900
bind-ip link::bind-ip for outgoing connects 198.51.100.2
class link::class servers

tkl object[edit]

This is a server ban, such as a GLINE or Spamfilter. It is also used for ban exceptions (ELINE).

Variable Description Example value
type Type of the server ban. One of: gline, kline, gzline, zline, spamfilter,
qline, except, shun, local-qline, local-exception, local-spamfilter.
gline
type_string Type of the server ban in a more friendly user printable string.
Possibly prefixed with "Soft" eg "Soft G-Line".
G-Line
name The target of the ban or except. For Spamfilter this is the regex/matcher. *@*.badisp.example.net
set_by Name of the person or server who set the ban Syzop
set_at Date/Time when the server ban was added 2022-05-23T11:02:06.000Z
set_at_string Date/Time when the server ban was added in a more human printable string. Mon May 23 11:02:06 2022
expire_at Date/Time when the server ban will expire. NULL means: never. 2023-05-23T10:00:00.000Z
expire_at_string Date/Time when the server ban will expire in a more human printable string.
Uses "Never" for never.
Tue May 23 10:00:00 2023
duration_string How long the ban will last from this point in time (human printable).
Uses "permanent" for forever.
permanent
set_at_delta How many seconds ago the ban was placed. 1800
reason The reason of the ban Lots of abuse from this ISP
exception_types Only for ban exceptions! Specifies the exception types (letters).
See HELPOP ELINE for a list of those letters.
kGzZsF
match_type Only for spamfilters! The matching type. One of: simple, regex regex
ban_action Only for spamfilters! The action to take on spamfilter hit. gline
spamfilter_targets Only for spamfilters! Which targets the spamfilter must filter on. cpnN