perf: memoize __getattr__ on JVMView / JavaPackage / JavaClass (issue #557)#14
perf: memoize __getattr__ on JVMView / JavaPackage / JavaClass (issue #557)#14Tagar wants to merge 4 commits into
Conversation
) Each scenario maps 1:1 to a planned cold-start improvement so a per- PR CodSpeed delta is unambiguous. Without these, the wins from the upcoming PRs would be invisible to the dashboard. * XA-3 — attribute_walk_jvm_java_lang_System 1 000 walks of `gateway.jvm.java.lang.System`. Each walk re-issues 3 reflection round-trips because JVMView / JavaPackage / JavaClass __getattr__ don't memoize today. Cache canary for the upcoming Python attribute-cache PR — should collapse to ~0 round-trips after first walk once memoization lands. * XB — gateway_reconnect_against_running_jvm 50 fresh JavaGateway() instances against the fixture JVM. Measures per-connection setup cost: Java accept() loop allocates 14 command class instances per connection (GatewayConnection.java:196-208), Python runs _create_connection + socket setup. Lever for the Java-side command-prototype-cache PR. * XC — callback_infra_overhead_unused X1-1 workload (10 000 currentTimeMillis() calls) with CallbackServer started but no callbacks invoked. Delta vs X1-1 is the steady-state overhead of running the callback server alongside the main gateway. Target of the lazy-CallbackClient PR. * XD — full_cold_start_subprocess_first_call subprocess.Popen -> first call -> shutdown, one cycle per round. Slow per-iteration (~350 ms on M-series + JDK 21, multi-second on slower hosts); CodSpeed adapts iteration count. Target of the JVM-flag opt-ins (TieredStopAtLevel=1, AppCDS, CRaC). Implementation notes: XA / XB / XC follow the existing MacroScenario contract and run through the macro_gateway fixture in codspeed_macros.py (shared JVM, setup once, measure many). XD owns its JVM lifecycle inside measure() — it must NOT share the fresh_jvm()-managed fixture because both default to port 25333. So XD is registered in a new _COLD_START_SCENARIOS list parametrized through a new test_cold_start_scenario function. XD is also omitted from ALL_MACRO_CLASSES so the framework runner doesn't try to share its JVM either. Local measurements on M-series + JDK 21 (Temurin 21.0.10): XA-3: ~94 ms / 1000 walks = 94 us per walk (3 RTTs each) XB: ~21 ms / 50 reconnects = 425 us per reconnect XC: ~236 ms / 10k calls = 23.6 us per call XD: ~355 ms per full cold-start cycle Co-authored-by: Isaac
The XD ``test_cold_start_scenario`` introduced in #12 used a blind ``time.sleep(0.25)`` followed by a direct ``JavaGateway()`` connect attempt. On CodSpeed CI runners (and any host slower than a modern laptop) the JVM hadn't yet bound its listen socket when the connect fired, surfacing as ``ConnectionRefusedError: [Errno 111] Connection refused`` and failing the whole codspeed workflow. Switch XD's body to the existing ``fresh_jvm`` context manager with ``readiness_retries=5``. ``fresh_jvm`` polls connection readiness with ``check_connection`` and only proceeds once the JVM is actually accepting calls — same convention the rest of the test suite uses. This also drops a bunch of try/finally boilerplate since ``fresh_jvm`` handles spawn/shutdown. Locally on M-series + JDK 21: XD median ~355 ms (unchanged from post-#12 baseline). The variance is higher in cold-start measurement by nature; what matters is the elimination of hard failures on slower hosts. Co-authored-by: Isaac
Tagar
force-pushed
the
c1-python-attribute-caches
branch
from
1c24b7d to
5f5a9e5
Compare
Merging this PR will improve performance by ×45
Performance Changes
Tip Curious why this is faster? Comment Comparing 
Footnotes
|
|
@codspeedbot explain why this is faster |
The explanation doesn't require code changes — my response here will be posted as the PR comment. Here's the analysis: Why this PR is fasterThe bottleneck: repeated JVM reflection round-tripsIn py4j, every Python attribute access along a chain like
Every single access pays 3 socket round-trips, even though the JVM's answer is always the same — The fix: per-instance
|
XD (full cold start) was producing ~2.7 s per round on CodSpeed CI, which exhausts the per-benchmark wall-time budget after one sample. With only one sample, CodSpeed reports 0% noise and "no measurable impact" for every cold-start PR — including any genuine 10-100 ms improvement that would otherwise be detectable. This made #15 (GatewayConnection background-warm) invisible on the dashboard even though the optimization itself is correct. Replace the legacy readiness model: 0.25 s sleep + 1 retry x 2.0 s = 2.25 s ceiling with tight polling: 0 s sleep + 300 retries x 0.05 s = 15 s ceiling fast hosts now succeed in ~50-200 ms; slow CI in ~1-2 s; ceiling still generous enough for any reasonable runner. With XD per-round down to ~250-500 ms, CodSpeed fits 4-8 samples per benchmark budget and can compute a usable CV. Also drops the unconditional startup_sleep: the rationale ("let the OS reuse the listen port") no longer applies — the JVM-side GatewayServer.startSocket() uses SO_REUSEADDR (already on master), so bind() succeeds immediately even over TIME_WAIT. Backwards compatibility: callers that need the old sleep pattern can pass ``startup_sleep=0.25`` explicitly. Local measurement (M-series + JDK 21): XD before: median ~355 ms, max ~2.4 s (variance dominated by retries) XD after: median ~260 ms, max ~420 ms (proper tight polling) Co-authored-by: Isaac
…4j#557) Cold-start work, C1 of the issue py4j#557 series. Today every attribute walk along ``gateway.jvm.X.Y.Z.method`` re-issues one reflection round-trip per level — JVMView, JavaPackage, and JavaClass all have ``__getattr__`` methods that call out to the JVM without caching the result. After the first walk, the JVM-side answer is stable for the JVM's lifetime, so the next 999 walks pay the same round-trip cost for no new information. This change adds a per-instance dict cache on each of: * ``JVMView._attr_cache`` — top-level names resolved on ``.jvm.X`` * ``JavaPackage._attr_cache`` — names resolved on ``.X.Y`` * ``JavaClass._members`` — static members on ``.System.currentTimeMillis`` Only successful resolutions are cached. Misses (Py4JError) keep raising so that a later ``java_import`` or classpath change can still surface a previously-missing class. The ``UserHelpAutoCompletion`` sentinel path and direct return values on ``JavaClass.__getattr__`` are NOT cached because they each have caller-visible semantics that caching would change. Thread safety mirrors the existing ``_statics`` / ``_dir_sequence_and_cache`` convention: plain dict assignments are atomic under CPython's GIL, worst case is double-fill with the same value. The existing comments in those caches explicitly accept this tradeoff. Local measurement on the new XA-3 scenario (M-series + JDK 21): before: ~94 ms / 1000 walks = ~94 us per walk (3 reflection RTTs) after: ~269 us / 1000 walks = ~0.27 us per walk (dict get) Approx 350x speed-up on the steady-state attribute-walk path; the first walk in any chain still pays the full reflection cost. X1-1 unchanged (10 000 currentTimeMillis() calls, ~232 ms either way). Co-authored-by: Isaac
Tagar
force-pushed
the
c1-python-attribute-caches
branch
from
5f5a9e5 to
a74edbe
Compare
1 participant
Summary
Cold-start work, C1 of the issue py4j/py4j#557 series.
Today every attribute walk along
gateway.jvm.X.Y.Z.methodre-issues one reflection round-trip per level —JVMView,JavaPackage, andJavaClassall have__getattr__methods that call out to the JVM without caching the result. After the first walk, the JVM-side answer is stable for the JVM's lifetime, so the next 999 walks pay the same round-trip cost for no new information.This change adds a per-instance dict cache on each of:
JVMView_attr_cache.jvm.XJavaPackage_attr_cache.X.YJavaClass_members.System.currentTimeMillisLocal CodSpeed delta (M-series + JDK 21)
The first walk in any chain still pays the full reflection cost; subsequent walks are free dict lookups. Steady-state attribute walking is now amortized.
Semantics & safety
java_importor classpath change can still surface a previously-missing class.UserHelpAutoCompletionsentinel path is NOT cached onJVMView— that helper returns a fresh instance per access by design.JavaClass.__getattr__(constants, static non-final fields) are NOT cached — there's no contract that the JVM-side value can't change across calls. OnlyJavaMember(methods) andJavaClass(nested classes) results are cached._statics/_dir_sequence_and_cacheconvention: plain dict assignments are atomic under CPython's GIL, worst case is double-fill with the same value. The existing comments in those caches explicitly accept this tradeoff.Diff
py4j-python/src/py4j/java_gateway.py: +52 / -6Test plan
finalizer,protocol,signals)java_gateway_test+client_server_testpass locally; 14 failures are pre-existingGatewayLauncherTestjar-path issues unrelated to this change (confirmed on byteoak/master baseline)Note on CodSpeed
The CodSpeed run on byteoak master is currently red because of XD's CI-runner startup race (see #13 for the fix). C1's own CodSpeed measurement (the XA-3 delta) will be visible after #13 lands and master CodSpeed goes green again. C1 and #13 are otherwise independent (different files, different concerns).