Skip to content

About

High-performance hierarchical logging for Julia modules, with fine-grained runtime control and near-zero filtered-call overhead.

Resources

Stars

4 stars

Watchers

0 watching

Forks

Latest commit

 

History

99 Commits

Folders and files

Repository files navigation

ComponentLogging

Dev Build Status Benchmarks

ComponentLogging.jl is a lightweight, high-performance logging layer for Julia built around explicit component loggers and hierarchical log-level control.

Installation

julia>] add ComponentLogging

Quick start

This package lets you manage a hierarchy of log levels. Log messages are filtered or printed by group. Groups are represented as NTuple{N,Symbol}. The minimum enabled log level is usually specified as an integer. To construct a logger, pass groups=>level pairs to ComponentLogger().

using ComponentLogging

logger = ComponentLogger(
    :core         => 0,      # Info+
    :io           => 1000,   # Warn+
    (:net, :http) => 2000;   # Error+
    sink=PlainLogger()
)
@forward_logger logger

clog(:core, 0, "starting job"; jobid=42)   # 0 = Info
clog(:io, 1000, "retrying I/O"; attempt=3) # 1000 = Warn

What the rules mean

  • Key: a group (Symbol or NTuple{N,Symbol}) such as :core or (:net,:http).
  • Value: the minimum level enabled for that group. Messages below this level are filtered out.
  • Define a catch-all like :__default__ => 0 to control unmatched groups.

Enter logger in the REPL or call display(logger) to print the logger's hierarchy.

ComponentLogger
 sink:  PlainLogger
 min:   Info
 rules: 4
  ├─ :__default__     Info
  ├─ :core            Info
  ├─ :io              Warn
  └─ :net
     └─ :http         Error

Hierarchical runtime switches

The same group hierarchy can be used as a lightweight runtime control plane. With the forwarding helpers created by @forward_logger, set_log_level!(group => true) enables the group's default Info-level check, while set_log_level!(group => false) disables it.

function solve(problem)
    clogenabled((:solver, :presolve)) && run_presolve!(problem)

    if clogenabled((:solver, :heuristics))
        run_heuristics!(problem)
    end

    clogenabled((:solver, :cache)) && update_cache!(problem)
end

# Change behavior globally for this module/component.
set_log_level!(
    (:solver, :presolve) => true,
    (:solver, :heuristics) => false,
    (:solver, :cache) => true,
)
solve(problem)

This lets deeply nested behavior be switched at runtime without passing verbose, enable_*, or other configuration flags through every function in the call chain. The switches can control logging, diagnostics, optional computations, algorithmic branches, caching strategies, or any other behavior you choose to place behind clogenabled.

Without forwarding helpers, use the logger-explicit forms instead:

set_log_level!(logger, (:solver, :heuristics) => false)
clogenabled(logger, (:solver, :heuristics))
# Query the level after hierarchical fallback.
get_log_level(logger, (:solver, :heuristics))

Since v2.0.0, set_log_level! accepts only Pair arguments after the logger (or only pairs in its forwarded form); the old separate group and level arguments are no longer supported.

Core APIs

clog

Emits a log record for a group at a given level.

clog(logger, group, level, msg...; kwargs...)

Arguments:

  • logger::AbstractLogger — any logger instance. Typically you pass a ComponentLogger configured with per-group rules and a sink (e.g. PlainLogger). Many codebases also define forwarding helpers to avoid threading the logger explicitly (see below).
  • group::Union{Symbol,NTuple{N,Symbol}} — a Symbol or a tuple of symbols, e.g. :core or (:net, :http).
  • level::Union{Integer,LogLevel} — prefer integers (no need to import Logging). We immediately convert with LogLevel(level).
    • Mapping (integers first): -1000 (Debug), 0 (Info), 1000 (Warn), 2000 (Error).

Example:

clog(logger, :core,  0, "starting job"; jobid=42)   # 0 = Info
clog(logger, :io, 1000, "retrying I/O"; attempt=3)  # 1000 = Warn

clogenabled

Checks whether records at level are enabled for group.

Example:

if clogenabled(logger, :core, 1000)  # guard expensive work
    stats = compute_expensive_stats()
    clog(logger, :core, 1000, "stats ready"; stats)
end

@clog

What if generating the log message itself incurs a computational cost? @clog provides for the lazy construction of messages: it evaluates expressions only when the corresponding log level is enabled.

@clog logger group level msg...

Example:

using Statistics: mean, std

arr = randn(1000)
sumsq = 0.0
for i in eachindex(arr)
    x = arr[i]
    sumsq += x^2
    @clog :core 1 begin
        meanval = mean(arr[1:i])
        stdval = std(arr[1:i])
        # The return value will be used as the log message
        "i=$i, x=$x, mean=$(meanval), std=$(stdval), sumsq=$(sumsq)"
    end
end

If the final message expression evaluates to nothing, no log record is emitted.

@forward_logger

@forward_logger logger

This creates module-local forwarding methods for clog, cdebug, cinfo, cwarn, cerror, clogenabled, set_sink, set_log_level!, get_log_level, with_min_level, @clog, @cdebug, @cinfo, @cwarn, and @cerror macros. The logger expression is bound where @forward_logger is declared, including when the generated macro is imported by another module.

For example, the generated clog is conceptually equivalent to the following module-local forwarding method: clog(args...; kwargs...) = ComponentLogging.clog(logger, args...; kwargs...)

with_min_level

Temporarily sets the logger’s global minimum level and restores it on exit.

For example, to benchmark compute_sumsq() without any logging‑related work:

with_min_level(logger, 2000) do
    @benchmark compute_sumsq()
end

set_sink

set_sink(logger, sink) returns a new ComponentLogger with the supplied sink and the current rules from logger.

logger = ComponentLogger(:io => 1000) # the default sink is ConsoleLogger(Debug)
new_logger = set_sink(logger, PlainLogger())

PlainLogger

PlainLogger is roughly a Base.CoreLogging.SimpleLogger without the [Info:-style prefixes. Its output looks like print, subject to its configured min_level filter. When wrapped by ComponentLogger, a record is enabled only when both the component rules and the sink's own filtering allow it.

PlainLogger and ComponentLogger are independent. You can also include("src/PlainLogger.jl") to use PlainLogger on its own.

Example:

using ComponentLogging, Logging

logger = PlainLogger()
with_logger(logger) do
    @info "Hello, Julia!"
end

Output:

Hello, Julia!
@ Main README.md :203

PlainLogger uses show with MIME"text/plain" to display arrays with two or more dimensions, as it improves matrix readability. For other values, it prints them directly using print.

with_logger(logger) do
    @warn rand(1:9, 3, 3)
end

Output:

3×3 Matrix{Int64}:
 8  5  6
 3  4  9
 7  8  5
@ Main README.md :218

Benchmarking

ComponentLogging is continuously benchmarked over time. Current and historical performance results are available on the benchmark dashboard.

Similar Packages

Memento.jl is a flexible, hierarchical logging framework that brings its own ecosystem of loggers, handlers, formatters, records, and IO backends. Loggers are named (e.g., "Foo.bar"), form a hierarchy with propagation to a root logger, and are configured via config!, setlevel!, and by attaching handlers (file, custom formatters, etc.).

HierarchicalLogging.jl defines a Base.Logging-compatible HierarchicalLogger that associates loggers with hierarchically-related objects (e.g., module → submodule). Each node has a LogLevel that can be set with min_enabled_level!, which also recursively updates children; you can attach different underlying loggers (e.g., ConsoleLogger) to different parts of the tree.

ComponentLogging.jl takes a different approach: it is a lightweight, performance-oriented layer over Base.CoreLogging built around explicit component loggers and hierarchical group rules. It routes accepted records to any AbstractLogger sink, keeps filtered paths extremely cheap, and also exposes the same hierarchy as a lightweight runtime control plane through clogenabled and Boolean level switches.

  • Choose Memento.jl for a self-contained logging framework with hierarchical named loggers and a rich handler/formatter system.
  • Choose HierarchicalLogging.jl for stdlib-compatible hierarchical control over hierarchically related objects with recursive level management.
  • Choose ComponentLogging.jl for explicit hierarchical control, very low filtered-call overhead, direct composition with AbstractLogger sinks, and a hierarchy that can double as a runtime control plane.

About

High-performance hierarchical logging for Julia modules, with fine-grained runtime control and near-zero filtered-call overhead.

Resources

Stars

4 stars

Watchers

0 watching

Forks

Releases

Contributors

Languages