微基準測試
Netty 有個名為「netty-microbench」的模組,它會執行一系列微基準測試。它建構在 OpenJDK JMH 之上,這是 HotSpot 最佳的微基準測試解決方案。它備有「電池」,因此您無需額外的相依性即可開始著手操作。
您可以透過 Maven 或直接在您的 IDE 中從命令列執行基準測試。若要使用預設設定執行所有測試,請使用 mvn -DskipTests=false test
。您需要明確設定 skipTests=false
,因為我們不希望在執行常規測試時,執行可能會花費很多時間的微基準測試作為單元測試。
如果一切順利,您將會看見 JMH 在分支數量上執行熱身和基準測試迭代,並向您呈現一個清楚的摘要。以下是基準測試執行的常見樣貌(您會在輸出中看見許多此類結果)
# Fork: 2 of 2
# Warmup: 10 iterations, 1 s each
# Measurement: 10 iterations, 1 s each
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Running: io.netty.microbench.buffer.ByteBufAllocatorBenchmark.pooledDirectAllocAndFree_1_0
# Warmup Iteration 1: 8454.103 ops/ms
# Warmup Iteration 2: 11551.524 ops/ms
# Warmup Iteration 3: 11677.575 ops/ms
# Warmup Iteration 4: 11404.954 ops/ms
# Warmup Iteration 5: 11553.299 ops/ms
# Warmup Iteration 6: 11514.766 ops/ms
# Warmup Iteration 7: 11661.768 ops/ms
# Warmup Iteration 8: 11667.577 ops/ms
# Warmup Iteration 9: 11551.240 ops/ms
# Warmup Iteration 10: 11692.991 ops/ms
Iteration 1: 11633.877 ops/ms
Iteration 2: 11740.063 ops/ms
Iteration 3: 11751.798 ops/ms
Iteration 4: 11260.071 ops/ms
Iteration 5: 11461.010 ops/ms
Iteration 6: 11642.912 ops/ms
Iteration 7: 11808.595 ops/ms
Iteration 8: 11683.780 ops/ms
Iteration 9: 11750.292 ops/ms
Iteration 10: 11769.986 ops/ms
Result : 11650.238 ±(99.9%) 229.698 ops/ms
Statistics: (min, avg, max) = (11260.071, 11650.238, 11808.595), stdev = 169.080
Confidence interval (99.9%): [11420.540, 11879.937]
最後,測試輸出會類似這個樣子(視您的系統設定和組態而定)
Benchmark Mode Samples Mean Mean error Units
i.n.m.b.ByteBufAllocatorBenchmark.pooledDirectAllocAndFree_1_0 thrpt 20 11658.812 120.728 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.pooledDirectAllocAndFree_2_256 thrpt 20 10308.626 147.528 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.pooledDirectAllocAndFree_3_1024 thrpt 20 8855.815 55.933 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.pooledDirectAllocAndFree_4_4096 thrpt 20 5545.538 1279.721 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.pooledDirectAllocAndFree_5_16384 thrpt 20 6741.581 75.975 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.pooledDirectAllocAndFree_6_65536 thrpt 20 7252.869 70.609 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.pooledHeapAllocAndFree_1_0 thrpt 20 9750.225 73.900 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.pooledHeapAllocAndFree_2_256 thrpt 20 9936.639 657.818 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.pooledHeapAllocAndFree_3_1024 thrpt 20 8903.130 197.533 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.pooledHeapAllocAndFree_4_4096 thrpt 20 6664.157 74.163 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.pooledHeapAllocAndFree_5_16384 thrpt 20 6374.924 337.869 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.pooledHeapAllocAndFree_6_65536 thrpt 20 6386.337 44.960 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.unpooledDirectAllocAndFree_1_0 thrpt 20 2137.241 30.792 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.unpooledDirectAllocAndFree_2_256 thrpt 20 1873.727 41.843 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.unpooledDirectAllocAndFree_3_1024 thrpt 20 1902.025 34.473 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.unpooledDirectAllocAndFree_4_4096 thrpt 20 1534.347 20.509 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.unpooledDirectAllocAndFree_5_16384 thrpt 20 838.804 12.575 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.unpooledDirectAllocAndFree_6_65536 thrpt 20 276.976 3.021 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.unpooledHeapAllocAndFree_1_0 thrpt 20 35820.568 259.187 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.unpooledHeapAllocAndFree_2_256 thrpt 20 19660.951 295.012 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.unpooledHeapAllocAndFree_3_1024 thrpt 20 6264.614 77.704 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.unpooledHeapAllocAndFree_4_4096 thrpt 20 2921.598 95.492 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.unpooledHeapAllocAndFree_5_16384 thrpt 20 991.631 49.220 ops/ms
i.n.m.b.ByteBufAllocatorBenchmark.unpooledHeapAllocAndFree_6_65536 thrpt 20 261.718 11.108 ops/ms
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 993.382 sec - in io.netty.microbench.buffer.ByteBufAllocatorBenchmark
您也可以直接從您的 IDE 執行基準測試。如果您已匯入 netty 父項專案,請開啟 microbench
子專案並導航至 src/main/java/io/netty/microbench
名稱空間。在 buffer
名稱空間中,您可以執行 ByteBufAllocatorBenchmark
,就像執行任何其他基於 JUnit 的測試一樣。最大的不同在於(截至目前為止),您只能一次執行完整的基準測試,無法分別執行每個子基準測試。您應該會在主控台中看到與直接透過 mvn
執行時相同的輸出。
撰寫基準測試本身並不難,但寫對了才算。這並非因為 microbench 專案難以使用,而是因為您在撰寫基準測試時需要避免常見的陷阱。好消息是,JMH 套組提供了有用的註解和功能來緩解大部分陷阱。要開始撰寫,您需要讓您的基準測試繼承 AbstractMicrobenchmark
,它會確保測試透過 JUnit 執行,並設定一些預設值
public class MyBenchmark extends AbstractMicrobenchmark {
}
下一步是建立一個方法,並加上 @GenerateMicroBenchmark
註解(和描述性的名稱)
@GenerateMicroBenchmark
public void measureSomethingHere() {
}
現在最佳的方法是查看此處的範例,並取得撰寫正確 JMH 測試的靈感。另外,查看 JMH 主要作者的談話。
預設設定(在 AbstractMicrobenchmark
中找到)為
- 熱身迭代:10
- 測量迭代:10
- 執行緒的數量:2
這些設定可以在執行階段透過系統屬性(warmupIterations
、measureIterations
和 forks
)自訂
mvn -DskipTests=false -DwarmupIterations=2 -DmeasureIterations=3 -Dforks=1 test
請注意,一般建議不要使用這麼少的迭代次數,但有時候或許有助於查看基準測試是否運作正常,然後在稍後的時機執行全面的基準測試。
請注意,您也可以透過註解在每個測試的基礎上自訂這些預設設定
@Warmup(iterations = 20)
@Fork(1)
public class MyBenchmark extends AbstractMicrobenchmark {
}
這可以在每個類別和每個方法(基準測試)的基礎上執行。請注意,命令列引數永遠會優先於註解預設值。