DOCSEARCHbyAlgolia
Submit Docs

Bartosz Jabłoński - IT Blog and Travel

website·Indexed 30 days ago

https://bartas93.github.io

A personal technical blog by Backend Cloud Developer Bartosz Jabłoński. Content covers JVM object memory allocation, HotSpot VM configurations, JOL (Java Object Layout), regular expressions, Python scripts for YouTube downloading, and managing Kubernetes in Google Cloud.

Tokens
6.1K
Snippets
25
Records
43
Agent score
62%

What's inside Bartosz Jabłoński Blog

  1. Camply Platform Technical Architecture Overview

    Camply is a Polish marketplace for children's and teenagers' camps. The platform features a search engine based on over 10 criteria using Query DSL, handling complex logic for geographical departure coordinates and dynamic price ranges (combining base trip costs with location-based transport costs).

  2. Implement a reliable mailing system with RabbitMQ

    To prevent system errors and avoid spam classification, Camply implemented a custom mailing system based on RabbitMQ queuing. This ensures that all emails are reliably delivered to customers by decoupling the mailing process from the main application flow.

  3. Download YouTube videos and audio using Python and pytube

    To download MP3 or MP4 files from YouTube playlists or individual videos without using ad-heavy online tools, you can use a Python script leveraging the pytube library. This approach allows for bulk downloading from playlists and avoids invasive advertisements.
    # Install the required library
pip install pytube

  4. Download YouTube videos and audio using Python and pytube

    To avoid invasive advertisements and the limitation of downloading videos individually from web-based tools, you can use the pytube library in Python to create a script that downloads mp3 or mp4 files from a YouTube movie link or playlist.
    # Install the pytube library
pip install pytube

  5. Install and set up the YouTube Downloader script

    To use the YouTube downloader, clone the repository, ensure Python 3.11+ is installed, and install the required dependencies from the requirements file.
    git clone https://gitlab.com/bart_as93/youtube-downloader.git
cd youtube-downloader
pip install -r requirements.txt

  6. Download YouTube videos and audio using pytube

    To avoid invasive advertisements and the limitation of downloading individual videos from online tools, you can use a Python script powered by the pytube library. This allows for downloading mp3 or mp4 files from a provided link to a movie or a playlist.
    # Install the required library
pip install pytube

  7. Analyze JVM object memory layout using JOL

    The Java Object Layout (JOL) library is a toolbox used to analyze the memory layout of objects in the JVM, including their memory footprint and references. It leverages Unsafe, JVMTI, and the Serviceability Agent (SA) to decode object internals.
    <dependency>
    <groupId>org.openjdk.jol</groupId>
    <artifactId>jol-core</artifactId>
    <version>0.10</version>
</dependency>

  8. Manage Compressed References (Compressed Oops)

    On 64-bit JVMs, references normally take 8 bytes. To reduce memory usage, 'Compressed Oops' (Ordinary Object Pointers) are used by default, reducing reference size to 4 bytes using a 3-bit shift.

    • Heap Limit: Compressed references are generally effective for heaps up to 32 GB.
    • Performance Caveat: Increasing the maximum heap (-Xmx) beyond 32 GB (e.g., from 31 GB to 33 GB) can cause the JVM to switch from 4-byte to 8-byte references, potentially increasing the total heap occupancy by up to 40%.
    • Disable Compression: Use the flag -XX:-UseCompressedOops to force 8-byte references.

  9. Manage JVM Compressed References and Heap Size limits

    By default, 64-bit CPUs use Compressed References to reduce the size of object references from 8 bytes to 4 bytes. This is achieved via a bit shift (3 places) leveraging the 8-byte alignment of objects.

    Critical Constraint: The maximum heap size for Compressed References is 32 GB. If the maximum heap (-Xmx) is increased beyond 32 GB (e.g., from -Xmx31g to -Xmx33g), the JVM switches from 4-byte to 8-byte references for every object. This can lead to a significant increase in heap memory usage—potentially up to 40% depending on the data—due to the increased size of every reference on the heap.

    # Example of a heap size that supports Compressed References
-Xmx31g

# Example of a heap size that triggers 8-byte references (increasing memory overhead)
-Xmx33g

  10. Manage JVM heap size and Compressed References (Compressed OOPs)

    On 64-bit CPUs, the JVM uses 'Compressed References' by default to reduce the size of object references from 8 bytes to 4 bytes. This optimization is possible as long as the maximum heap size (-Xmx) is 32 GB or less.

    If the heap is increased beyond 32 GB (e.g., moving from -Xmx31g to -Xmx33g), the JVM switches from 4-byte to 8-byte references. This can lead to a significant increase in total heap occupancy—potentially up to 40% depending on the data—because every single object reference on the heap now requires double the space.

    # Example of a heap size that maintains 4-byte compressed references
java -Xmx31g -jar app.jar

# Example of a heap size that triggers 8-byte references, potentially increasing memory usage by ~40%
java -Xmx33g -jar app.jar

  11. Manage JVM compressed references (Compressed OOPs)

    The JVM uses compressed Ordinary Object Pointers (OOPs) to allow 32-bit references on 64-bit machines, enabling heaps up to 32 GB. This is achieved by shifting addresses 3 bits to the left, leveraging the fact that objects are 8-byte aligned.

    • Default Behavior: Reference compression is enabled for heaps $\le$ 32 GB. If the heap exceeds 32 GB, compression is automatically disabled, and references increase to 8 bytes.
    • Manual Disable: Use the flag -XX:-UseCompressedOops to force 8-byte references.
    • Increasing Heap Limit with Compression: To use compressed references with heaps larger than 32 GB (up to 64 GB), increase the object alignment using -XX:ObjectAlignmentInBytes=16.

    Caveat: Increasing ObjectAlignmentInBytes increases the memory waste (padding) for every object, which may offset the benefits of compression.

    # Disable compressed OOPs (references become 8 bytes)
java -XX:-UseCompressedOops -jar app.jar

# Increase alignment to 16 bytes to allow compressed OOPs up to 64GB heap
java -XX:ObjectAlignmentInBytes=16 -jar app.jar

  12. Manage JVM Compressed References and Heap Limits

    The JVM uses Compressed References (Compressed OOPs) to reduce the size of object references from 8 bytes to 4 bytes on 64-bit CPUs, provided the maximum heap size is kept under 32 GB. This is achieved via bit shifting, leveraging the 8-byte alignment of objects.

    Critical Caveat: Increasing the maximum heap (-Xmx) beyond 32 GB (e.g., moving from -Xmx31g to -Xmx33g) disables Compressed References. This causes every object reference in the heap to jump from 4 bytes to 8 bytes, which can lead to a significant increase in total heap usage—potentially up to 40% depending on the data structure.

    # Example of a heap size that maintains 4-byte compressed references
java -Xmx31g -jar app.jar

# Example of a heap size that may trigger 8-byte references and increase memory overhead
java -Xmx33g -jar app.jar