Skip to content

gh-132372: Speed up logging.config existing logger handling#150242

Open
esadomer wants to merge 2 commits into
python:mainfrom
esadomer:codex/find-stronger-cpython-speedup
Open

gh-132372: Speed up logging.config existing logger handling#150242
esadomer wants to merge 2 commits into
python:mainfrom
esadomer:codex/find-stronger-cpython-speedup

Conversation

@esadomer
Copy link
Copy Markdown

@esadomer esadomer commented May 22, 2026

gh-132372

This speeds up logging.config.fileConfig() and logging.config.dictConfig() when they need to handle many existing loggers while disable_existing_loggers is active.

The old code kept the existing logger names in a sorted list, but still used repeated list membership, index(), tail scans, and remove() calls for every configured logger. This keeps the sorted list for child logger ordering, uses a set for membership/removal tracking, and uses bisect_left() to jump directly to the dotted child logger range.

It also batches the manager-wide logging cache clear when preserving child loggers. Previously each preserved child called Logger.setLevel(NOTSET), and each call cleared the cache for every logger.

Validation:

  • PCbuild\build.bat -d -p x64
  • PCbuild\amd64\python_d.exe -m test test_logging -j0
  • PCbuild\amd64\python_d.exe Tools\patchcheck\patchcheck.py
  • git diff --check

Benchmark:

Loaded upstream/main:Lib/logging/config.py and the patched Lib/logging/config.py into the same Python 3.12.8 release interpreter, then timed only the dictConfig() call while resetting identical existing logger state outside the timer.

old n=100  mean=   5.742 ms stdev=  0.160 ms
new n=100  mean=   2.255 ms stdev=  0.051 ms
speedup n=100: 2.55x

old n=500  mean= 133.837 ms stdev=  6.404 ms
new n=500  mean=  42.336 ms stdev=  0.586 ms
speedup n=500: 3.16x

old n=1000 mean= 537.785 ms stdev= 44.765 ms
new n=1000 mean= 170.014 ms stdev=  2.729 ms
speedup n=1000: 3.16x

@eendebakpt
Copy link
Copy Markdown
Contributor

The performance increase is moderate, but the PR reduces code duplication so I am +1.

@esadomer On your benchmark I could get a much larger speedud with this small commit 2b9ce22 (on top of your work). I have not fully tested this, but the speedup is > 5x. If you are interested you could test it further and pull into your branch.

@esadomer esadomer force-pushed the codex/find-stronger-cpython-speedup branch from d599b2e to c1a3122 Compare May 23, 2026 06:21
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
@esadomer esadomer force-pushed the codex/find-stronger-cpython-speedup branch from c1a3122 to 02cf98f Compare May 23, 2026 06:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants