Skip to content

Improve startup cache loading performance#9307

Merged
mbien merged 1 commit intoapache:masterfrom
mbien:cache-loading-perf
Apr 8, 2026
Merged

Improve startup cache loading performance#9307
mbien merged 1 commit intoapache:masterfrom
mbien:cache-loading-perf

Conversation

@mbien
Copy link
Copy Markdown
Member

@mbien mbien commented Mar 29, 2026

  • use Data*Streams instead of Object*Streams
  • simplify file format (no XML)
  • code renovation and minor smaller optimizations

result are about 4x faster cache loading times (writing probably too but its not relevant for UX)

minor:

  • List -> Set for field solely used for contains() in inner loop

in numbers:

readCache(): 113ms -> 28ms
calculateParents(): 128ms -> 30ms

or see the readCache() and calculateParents() marked in purple

before:
startup_cacheio_0

after:
startup_cacheio_1

third PR for some more startup optimizations pending. (first was #9303)

@mbien mbien added this to the NB30 milestone Mar 29, 2026
@mbien mbien added Code cleanup Label for cleanup done on the Netbeans IDE performance Platform [ci] enable platform tests (platform/*) ci:dev-build [ci] produce a dev-build zip artifact (7 days expiration, see link on workflow summary page) labels Mar 29, 2026
@mbien mbien force-pushed the cache-loading-perf branch from 0047b0a to 34d1d39 Compare March 29, 2026 17:08
@mbien mbien force-pushed the cache-loading-perf branch from 34d1d39 to e28b5f1 Compare April 1, 2026 22:09
@mbien
Copy link
Copy Markdown
Member Author

mbien commented Apr 2, 2026

tested a few failure modes. e.g using a corrupted cache or one from an older NB version and it got reset on start.

Details

e.g cache containing invalid data would show:

INFO [org.netbeans.core.startup.ModuleList]: Cannot read cache
java.io.UTFDataFormatException: malformed input around byte 6
	at java.base/java.io.DataInputStream.readUTF(DataInputStream.java:641)
	at java.base/java.io.DataInputStream.readUTF(DataInputStream.java:550)
	at org.netbeans.core.startup.ModuleList.readProps(ModuleList.java:652)
[catch] at org.netbeans.core.startup.ModuleList.readCache(ModuleList.java:626)
	at org.netbeans.core.startup.ModuleList$ReadInitial.run(ModuleList.java:1583)
	at org.openide.filesystems.EventControl.runAtomicAction(EventControl.java:102)
	at org.openide.filesystems.FileSystem.runAtomicAction(FileSystem.java:494)
	at org.netbeans.core.startup.ModuleList.readInitial(ModuleList.java:147)
	at org.netbeans.core.startup.ModuleSystem.readList(ModuleSystem.java:280)
	at org.netbeans.core.startup.Main.getModuleSystem(Main.java:171)
	at org.netbeans.core.startup.Main.getModuleSystem(Main.java:141)
	at org.netbeans.core.startup.Main.start(Main.java:307)
	at org.netbeans.core.startup.TopThreadGroup.run(TopThreadGroup.java:99)
	at java.base/java.lang.Thread.run(Thread.java:1583)

but there are various checks before that so it would fail silently in most cases without even getting to the point where it is loaded.

@mbien mbien marked this pull request as ready for review April 2, 2026 06:00
@mbien mbien force-pushed the cache-loading-perf branch from e28b5f1 to 54c0790 Compare April 2, 2026 08:06
@BradWalker
Copy link
Copy Markdown
Member

Nice work!

Copy link
Copy Markdown
Contributor

@matthiasblaesing matthiasblaesing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have minimal nitpick, that you might consider. Apart from that thanks for the work.

Copy link
Copy Markdown
Contributor

@matthiasblaesing matthiasblaesing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with this. Exposure is minimized and if we need to change the implementation later, we can that this way. Thank you.

 - use Data*Streams instead of Object*Streams
 - simplify file format (no XML)
 - code renovation and other minor optimizations

results in 4x faster cache loading times (readCache() and
calculateParents() methods)

minor:
 - ModuleManager.EnableContext: List -> Set for field solely used for
   contains() in inner loop
 - Module: data loading can synchronize on instance instead of class
   (DATA_LOCK no longer static)

note:
The IO utility class calls Dependency#create using MethodHandle to
avoid having to make it public API or introduce a split package.
@mbien mbien force-pushed the cache-loading-perf branch from f636af8 to f673ada Compare April 8, 2026 18:04
@mbien
Copy link
Copy Markdown
Member Author

mbien commented Apr 8, 2026

thanks for the patience. I think we converged to a pretty good (+maintainable) solution.

@mbien mbien merged commit 4e7bd08 into apache:master Apr 8, 2026
30 checks passed
@lkishalmi
Copy link
Copy Markdown
Contributor

Nice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:dev-build [ci] produce a dev-build zip artifact (7 days expiration, see link on workflow summary page) Code cleanup Label for cleanup done on the Netbeans IDE performance Platform [ci] enable platform tests (platform/*)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants