Experimenting with Project Loom EAP and Jiny

Project Looms changes the existing Thread implementation from the mapping of an OS thread, to an abstraction that can either represent such a thread or a virtual thread. In itself, that is an interesting move on a platform that historically put a lot more value on backward-compatibility in comparison to innovation. Compared to other recent Java versions, this feature is a real game-changer. Developers in general should start getting familiar with it as soon as possible.

Loom will decrease any resource consumption that native/kernel thread limited. Which is pretty much every HTTP service. It's going to be huge and everything will be using it. From "big data" frameworks like Spark, REST, databases like Cassandra. It's hard to think of a common use case for Java that won't benefit from virtual threads.

This article walks you through a experiment that uses a Jiny BIO application with Virtual Threads (similar as Golang's Goroutine). Having access to early access builds is the perfect opportunity to take a look what it takes to use virtual threads as worker threads, to see how memory consumption and the number of throughput it could serve over time.

Prepare stuffs

Project Loom is in its early stages which don't allow for exact benchmarking. Instead, the state of the project should be considered to change over time. From the project page:

Early-access (EA) functionality might never make it into a general-availability (GA) release.

EA functionality might be changed or removed at any time.

Involved components:

To run the experiment, you need to use a Loom EAP build (Java 16).

How the application was customized

  • Jiny BIO
public class App {
    public static void main(String[] args) {
        val gson = new Gson();
        val app = HttpServer.port(1234)
                      .useTransformer(gson::toJson);
        app.get("/", ctx -> of("Hello World"));
        app.start();
    }
}
  • Jiny NIO
public class App {
    public static void main(String[] args) {
        val gson = new Gson();
        val app = NIOHttpServer.port(1234)
                      .useTransformer(gson::toJson);
        app.get("/", ctx -> ofAsync("Hello World"));
        app.start();
    }
}
  • Jiny Fiber

It's easy to integrate a BIO Jiny application with Fiber virtual thread, Jiny has a method to customize the worker executor:

public class App {
    public static void main(String[] args) {
        val gson = new Gson();
        val app = HttpServer.port(1234)
                      // Use Fiber thread
                      .setExecutor(Executors.newVirtualThreadExecutor())
                      .useTransformer(gson::toJson);
        app.get("/", ctx -> of("Hello World"));
        app.start();
    }
}

Observations

The measurement uses wrk (yes, there's the coordinated omission problem, but for this case it's something we can live with) for warmup and measurement. There are quite significant differences between using Virtual and Kernel threads.

  • Jiny BIO with 382MB memory consumed and can serve 56705 req/sec
➜ wrk -t8 -c5000 -d30s http://test-wrk.tuhuynh.com
Running 30s test @ http://test-wrk.tuhuynh.com
  8 threads and 5000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.89ms    4.82ms 460.52ms   90.45%
    Req/Sec     7.20k     4.57k   28.15k    77.83%
  1706741 requests in 30.10s, 263.68MB read
  Socket errors: connect 4757, read 826, write 11, timeout 0
Requests/sec:  56705.78
Transfer/sec:      8.76MB
  • Jiny NIO with 249MB memory consumed and can serve 111842 req/sec
➜ wrk -t8 -c5000 -d30s http://test-wrk.tuhuynh.com
Running 30s test @ http://test-wrk.tuhuynh.com
  8 threads and 5000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.15ms    2.48ms 398.85ms   98.82%
    Req/Sec    14.06k     9.76k   41.96k    64.67%
  3357931 requests in 30.02s, 518.78MB read
  Socket errors: connect 4757, read 644, write 40, timeout 0
Requests/sec: 111842.56
Transfer/sec:     17.28MB
  • Jiny BIO (uses Fiber virtual thread) with 248M memory consumed and can serve 106584 req/sec
➜ wrk -t8 -c5000 -d30s http://test-wrk.tuhuynh.com
Running 30s test @ http://test-wrk.tuhuynh.com
  8 threads and 5000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.25ms    1.06ms  77.13ms   95.49%
    Req/Sec    13.40k     6.79k   26.58k    44.29%
  3199300 requests in 30.02s, 494.28MB read
  Socket errors: connect 4757, read 344, write 0, timeout 0
Requests/sec: 106584.14
Transfer/sec:     16.47MB

Conclusion

Virtual threads require less memory than kernel one (249 MB RSS vs 382 MB RSS), and in Blocking IO mode with Fiber the Jiny application can perform similar as Non-Blocking IO Jiny application.

Reference:

With BIO:

With NIO:

With BIO & Fiber:

How about Reactive/Coroutine?

The main question is, now that the JVM API offers an abstraction over OS threads, what will become of other abstractions such as Reactive and coroutines?

Developers who are about to learn about Reactive and coroutines should probably take a step back, and evaluate whether they should instead learn the new Thread API - or not.

jinyframework.com

Bình luận


White
{{ comment.user.name }}
Bỏ hay Hay
{{comment.like_count}}
Male avatar
{{ comment_error }}
Hủy
   

Hiển thị thử

Chỉnh sửa

White

Tu Huynh

12 bài viết.
84 người follow
Kipalog
{{userFollowed ? 'Following' : 'Follow'}}
Cùng một tác giả
White
55 11
Ngày nay khi mà MVC dần trở thành một thứ nhà quê "lạc hậu", APIs được thiết kế chung cho nhiều client app khác nhau (web app, mobile app or deskto...
Tu Huynh viết 2 năm trước
55 11
White
30 4
Việc phát triển web frontend side ở một số hệ thống lớn lâu nay vẫn được làm theo kiểu monoliths (dù có thể backend đã được microservice hoá), ai c...
Tu Huynh viết 2 năm trước
30 4
White
22 3
Bài viết cung cấp chi tiết về cách mà React render hoạt động, và việc sử dụng Context và Redux ảnh hưởng thế nào tới quá trình render của React. ...
Tu Huynh viết 6 tháng trước
22 3
{{like_count}}

kipalog

{{ comment_count }}

bình luận

{{liked ? "Đã kipalog" : "Kipalog"}}


White
{{userFollowed ? 'Following' : 'Follow'}}
12 bài viết.
84 người follow

 Đầu mục bài viết

Vẫn còn nữa! x

Kipalog vẫn còn rất nhiều bài viết hay và chủ đề thú vị chờ bạn khám phá!