Performance Metrics

As of release 24.11, for each release, USD generates performance metrics using a specific set of assets and specific hardware and software configurations. This page describes what metrics are collected, what hardware and software configurations are used, the actual metrics results, and how to generate the metrics locally.

What We Measure

For a given asset, our performance script captures the following metrics by default (items in bold are reported on this page):

  • Time to load and configure USD plugins for usdview

  • Time to open the stage

  • Time to reset the usdview prim browser

  • Time to initialize the usdview UI

  • Time to render first image (in usdview)

  • Time to shutdown Hydra

  • Time to close the stage

  • Time to tear down the usdview UI

  • Total time to start and quit usdview

  • Time to traverse the prims in the stage

We run 100 iterations for each asset, and capture the minimum and maximum times for that set of iterations. We also calculate the mean time across the 100 iterations.

For each asset, we first warm the filesystem cache by loading the asset in usdview, to ensure we’re not including cache performance issues in our metrics.

All assets used to measure performance are assumed to be available locally. Time to download assets is not measured or included as part of the gathered performance metrics.

What Environment Is Used

This section describes the computing environment used to generate the published performance metrics. The following operating systems and hardware are currently used.

Note

Machine specifications are subject to change. If specifications do change for a give release, historical performance measurements will be run to backfill any outdated data.

Linux

  • OS: CentOS Linux 7

  • CPU: 23 cores of Intel(R) Xeon(R) Platinum 8268 CPU @ 2.90GHz

  • RAM: 117GB

  • GPU: NVIDIA TU102GL (Quadro RTX 6000/8000)

macOS

  • OS: macOS 14.3

  • CPU: Apple M2 Ultra (20 Core)

  • RAM: 192GB

  • GPU: Apple M2 Ultra GPU (76 Core)

Windows

  • OS: Microsoft Windows 10 Enterprise

  • CPU: AMD EPYC 7763 64-Core Processor, 2450 Mhz, 31 Core(s), 31 Logical Processor(s)

  • RAM: 128GB

  • GPU: NVIDIA RTXA6000-24Q

USD Build

For each of the operating systems and hardware platforms listed previously, we build USD with the same build configuration. We use a stock invocation of build_usd.py with the default options (release build, Python components, imaging and USD imaging components, usdview, etc).

Metrics

Metrics are all measured in seconds.

Performance Graphs Per Platform

The following graphs show the time (in seconds) to open and close usdview for each asset. Graphs are provided for Linux, macOS, and Windows platforms (as described in What Environment Is Used).

Note

A small increase or decrease in performance metrics over different releases may not necessarily indicate an overall performance improvement or regression in USD. Best-case minimum measurements are taken over multiple iterations, but these measurements may still be subject to some variation, so use the below results with caution. We are investigating these sources of variation.

There are known issues with obtaining the create_first_image metric on macOS. We will update published metrics when this issue is resolved.

We are investigating the 25.05 Moore Lane performance regression on linux.

_images/linux.svg _images/macos.svg _images/windows.svg

Standard Shader Ball

This asset is designed to be a comprehensive test of a broad array of material properties in a single render. Geometry is expressed using USD, materials are defined using MaterialX, texture maps are provided in OpenEXR format and encoded using the Academy Color Encoding System ACEScg color space.

In our performance sweep we use the mtlx_bubblegum material. Note the image below may not exactly match the image generated during the performance run; the primary intention is not necessarily to exercise specific renderers.

https://raw.githubusercontent.com/usd-wg/assets/main/full_assets/StandardShaderBall/media/example_materials/mtlx_bubblegum.jpg

The shader ball asset can be downloaded here.

Linux Metrics (min/max/mean in seconds)

Metric

24.11

25.02

25.05

Open stage

min: 0.154571
max: 0.168689
mean: 0.16472699999999998
min: 0.145738
max: 0.191218
mean: 0.16476692
min: 0.152122
max: 0.181628
mean: 0.16804495

Render first image

min: 2.80578
max: 2.829025
mean: 2.8189242
min: 2.339219
max: 2.549609
mean: 2.40064236
min: 2.288085
max: 9.063781
mean: 2.4000510999999998

Close stage

min: 0.000569
max: 0.00095
mean: 0.000746
min: 0.000551
max: 0.00172
mean: 0.00083931
min: 0.000589
max: 0.001241
mean: 0.00093926

Shut down Hydra

min: 0.022005
max: 0.022807
mean: 0.0223177
min: 0.028971
max: 0.036495
mean: 0.03046249
min: 0.035442
max: 0.045576
mean: 0.03700483
macOS Metrics (min/max/mean in seconds)

Metric

24.11

25.02

25.05

Open stage

min: 0.096413
max: 0.098173
mean: 0.0973833
min: 0.090059
max: 0.096531
mean: 0.09162102
min: 0.08718
max: 0.093663
mean: 0.0895949

Render first image

min: N/A
max: N/A
mean: N/A
min: N/A
max: N/A
mean: N/A
min: N/A
max: N/A
mean: N/A

Close stage

min: 0.000238
max: 0.000339
mean: 0.0002655
min: 0.000245
max: 0.001005
mean: 0.00029506
min: 0.000236
max: 0.000483
mean: 0.00027492

Shut down Hydra

min: 0.008993
max: 0.013276
mean: 0.0104122
min: 0.0112
max: 0.019225
mean: 0.01475921
min: 0.011858
max: 0.019994
mean: 0.01478677
Windows Metrics (min/max/mean in seconds)

Metric

24.11

25.02

25.05

Open stage

min: 0.270022
max: 0.300575
mean: 0.2809994
min: 0.265009
max: 0.306585
mean: 0.2770631
min: 0.255097
max: 0.301206
mean: 0.26899483

Render first image

min: 4.571696
max: 4.745389
mean: 4.6892396000000005
min: 4.787113
max: 5.128265
mean: 4.85809605
min: 4.852535
max: 11.186214
mean: 5.01563222

Close stage

min: 0.001281
max: 0.002975
mean: 0.0017164
min: 0.001221
max: 0.002109
mean: 0.00140918
min: 0.001013
max: 0.002825
mean: 0.0015245

Shut down Hydra

min: 0.040251
max: 0.043999
mean: 0.0421298
min: 0.052362
max: 0.060537
mean: 0.05581599
min: 0.060713
max: 0.080423
mean: 0.06517961

Kitchen Set

This asset provides a complex kitchen scene.

https://openusd.org/images/Kitchen_set_thumb.png

The Kitchen Set asset can be downloaded here.

Linux Metrics (min/max/mean in seconds)

Metric

24.11

25.02

25.05

Open stage

min: 0.069993
max: 0.107452
mean: 0.0842476
min: 0.079919
max: 0.129695
mean: 0.09578667
min: 0.080612
max: 0.117714
mean: 0.09265372

Render first image

min: 0.279229
max: 0.3028
mean: 0.2903682
min: 0.271989
max: 0.464893
mean: 0.29444296
min: 0.287641
max: 0.357674
mean: 0.30991565

Close stage

min: 0.007057
max: 0.007835
mean: 0.0074928
min: 0.007885
max: 0.014355
mean: 0.00892537
min: 0.008111
max: 0.014272
mean: 0.00921452

Shut down Hydra

min: 0.009247
max: 0.009617
mean: 0.0093791
min: 0.013788
max: 0.020161
mean: 0.01476924
min: 0.013995
max: 0.01997
mean: 0.01467029
macOS Metrics (min/max/mean in seconds)

Metric

24.11

25.02

25.05

Open stage

min: 0.06026
max: 0.09047
mean: 0.0732717
min: 0.068011
max: 0.094733
mean: 0.07521936
min: 0.064019
max: 0.083588
mean: 0.07094971

Render first image

min: N/A
max: N/A
mean: N/A
min: N/A
max: N/A
mean: N/A
min: N/A
max: N/A
mean: N/A

Close stage

min: 0.002246
max: 0.002519
mean: 0.0023587
min: 0.002071
max: 0.003521
mean: 0.00249618
min: 0.002103
max: 0.003613
mean: 0.00251872

Shut down Hydra

min: 0.006481
max: 0.013024
mean: 0.0102488
min: 0.008866
max: 0.020057
mean: 0.01476464
min: 0.00835
max: 0.019442
mean: 0.01349169
Windows Metrics (min/max/mean in seconds)

Metric

24.11

25.02

25.05

Open stage

min: 0.13587
max: 0.177268
mean: 0.1491912
min: 0.111618
max: 0.139086
mean: 0.12787824
min: 0.113966
max: 0.138581
mean: 0.12613241

Render first image

min: 2.862769
max: 3.086248
mean: 2.9800833
min: 2.917131
max: 3.162327
mean: 2.970126
min: 2.817951
max: 3.474417
mean: 2.98876465

Close stage

min: 0.019616
max: 0.024004
mean: 0.0215864
min: 0.017333
max: 0.025273
mean: 0.02057523
min: 0.016401
max: 0.02549
mean: 0.02164378

Shut down Hydra

min: 0.020476
max: 0.023532
mean: 0.0217044
min: 0.021171
max: 0.028106
mean: 0.02360369
min: 0.022153
max: 0.031277
mean: 0.02482973

ALab

ALab is a full production scene created by Animal Logic and contains over 300 assets, complete with high-quality textures and two characters with looping animation in shot context. Supplied as four separate downloads: the full production scene, high-quality textures, shot cameras matching the ALab trailer, and baked procedural fur and fabric for the animated characters.

The metrics have been measured with the base asset merged with the additional “techvars” info.

https://animallogic.com/wp-content/uploads/2022/08/LoopySequence02.gif

The ALab asset can be downloaded here.

Linux Metrics (min/max/mean in seconds)

Metric

24.11

25.02

25.05

Open stage

min: 0.39004
max: 0.516364
mean: 0.4642872
min: 0.4406
max: 0.563632
mean: 0.50131423
min: 0.55127
max: 0.782238
mean: 0.61888838

Render first image

min: 7.897517
max: 8.556198
mean: 8.1684536
min: 6.865629
max: 8.149738
mean: 7.57883956
min: 7.327838
max: 8.2319
mean: 7.66690162

Close stage

min: 0.084709
max: 0.090772
mean: 0.0874594
min: 0.0905
max: 0.269124
mean: 0.09663697
min: 0.091385
max: 0.109916
mean: 0.09734432

Shut down Hydra

min: 0.12083
max: 0.165165
mean: 0.1328269
min: 0.086453
max: 0.115286
mean: 0.09681611
min: 0.089733
max: 0.313577
mean: 0.1043322
macOS Metrics (min/max/mean in seconds)

Metric

24.11

25.02

25.05

Open stage

min: 0.419115
max: 0.44761
mean: 0.4308989
min: 0.447765
max: 0.567854
mean: 0.5199257
min: 0.446114
max: 0.518793
mean: 0.47762163

Render first image

min: N/A
max: N/A
mean: N/A
min: N/A
max: N/A
mean: N/A
min: N/A
max: N/A
mean: N/A

Close stage

min: 0.034878
max: 0.041104
mean: 0.037702
min: 0.033369
max: 0.046855
mean: 0.03937341
min: 0.032355
max: 0.045791
mean: 0.03850542

Shut down Hydra

min: 0.100348
max: 0.120492
mean: 0.10805630000000001
min: 0.091086
max: 0.11464
mean: 0.10125593
min: 0.081101
max: 0.110275
mean: 0.09709076
Windows Metrics (min/max/mean in seconds)

Metric

24.11

25.02

25.05

Open stage

min: 0.453708
max: 0.465223
mean: 0.4604347
min: 0.410067
max: 0.530343
mean: 0.42763175
min: 0.524329
max: 0.616512
mean: 0.55108881

Render first image

min: 11.070572
max: 11.738015
mean: 11.290158
min: 11.124981
max: 12.310513
mean: 11.36062476
min: 10.725459
max: 12.190389
mean: 11.03470107

Close stage

min: 0.337716
max: 0.364083
mean: 0.35308
min: 0.285362
max: 0.353328
mean: 0.31994381
min: 0.295428
max: 0.375903
mean: 0.33473847

Shut down Hydra

min: 0.219807
max: 0.384717
mean: 0.2734629
min: 0.215869
max: 0.452494
mean: 0.30029581
min: 0.203839
max: 0.454649
mean: 0.29897051

Moore Lane

4004 Moore Lane is a fully composed, high-quality scene for the purpose of testing various visual computing issues. The house itself was wrapped around a number of typical problem areas for light transport and noise sampling. This includes things like thin openings in exterior walls, recessed area light sources, deeply shadowed corners, and high-frequency details. The exterior landscape surrounding the house consisted of a relatively simple ecosystem of instanced plants which could provide some additional levels of complexity. In addition to the geometry itself being designed to exacerbate some typical issues, the USD structure itself was created for several layers of testing.

The metrics have been measured using the contained MooreLane_ASWF_0623.usda file.

https://dpel.aswf.io/images/4004/render_20181.jpeg

The Moore Lane asset can be downloaded here.

Linux Metrics (min/max/mean in seconds)

Metric

24.11

25.02

25.05

Open stage

min: 0.075535
max: 0.094023
mean: 0.0791025
min: 0.076916
max: 0.094922
mean: 0.08151592
min: 0.071287
max: 0.097127
mean: 0.07692868

Render first image

min: 11.484618
max: 11.776089
mean: 11.6368226
min: 10.74976
max: 11.18661
mean: 10.92291829
min: 11.959853
max: 13.319375
mean: 12.66771735

Close stage

min: 0.020613
max: 0.02107
mean: 0.0207992
min: 0.023531
max: 0.02652
mean: 0.02451658
min: 0.03185
max: 0.048383
mean: 0.03333876

Shut down Hydra

min: 0.103945
max: 0.138991
mean: 0.1194904
min: 0.060153
max: 0.074561
mean: 0.06535938
min: 0.101514
max: 0.156663
mean: 0.11992382
macOS Metrics (min/max/mean in seconds)

Metric

24.11

25.02

25.05

Open stage

min: 0.085466
max: 0.086856
mean: 0.0859231
min: 0.086742
max: 0.093883
mean: 0.08819937
min: 0.081021
max: 0.087773
mean: 0.0821557

Render first image

min: N/A
max: N/A
mean: N/A
min: N/A
max: N/A
mean: N/A
min: N/A
max: N/A
mean: N/A

Close stage

min: 0.003298
max: 0.00522
mean: 0.0036797
min: 0.003228
max: 0.00909
mean: 0.00396154
min: 0.003233
max: 0.009139
mean: 0.00425161

Shut down Hydra

min: 0.222393
max: 0.28412
mean: 0.2741197
min: 0.098075
max: 0.329771
mean: 0.29719891
min: 0.098344
max: 0.331723
mean: 0.29398898
Windows Metrics (min/max/mean in seconds)

Metric

24.11

25.02

25.05

Open stage

min: N/A
max: N/A
mean: N/A
min: 0.144654
max: 0.195054
mean: 0.16251379
min: 0.129056
max: 0.19494
mean: 0.14662759

Render first image

min: N/A
max: N/A
mean: N/A
min: 14.465477
max: 16.936302
mean: 15.28659683
min: 14.190506
max: 15.70013
mean: 14.69402018

Close stage

min: N/A
max: N/A
mean: N/A
min: 0.062537
max: 0.082089
mean: 0.06802329
min: 0.06399
max: 0.087721
mean: 0.06963007

Shut down Hydra

min: N/A
max: N/A
mean: N/A
min: 0.425902
max: 0.662102
mean: 0.48684717
min: 0.375697
max: 0.482632
mean: 0.42639765

Running Performance Metrics Locally

We encourage developers to run the USD performance metrics to measure performance impacts of OpenUSD code contributions. Performance metrics can also be run to validate local runtime environments and hardware configurations.

Performance metrics are generating using the usdmeasureperformance.py script found in pxr/extras/performance. See the usdmeasureperformance tool docs for more information on the different parameters available.

usdmeasureperformance.py uses usdview and testusdview, so you will need to make sure those are in your current path, or aliased properly.

For gathering the metrics published on this page, the following parameters are used (for each asset):

python usdmeasureperformance.py <asset.usda> -i 100 -a min -o <metrics output filename.yaml>

Optionally, –tracedir <dir> will output trace information to dir that may aid in performance debugging.

Adding Custom Metrics

You can add your own custom metrics and have usdmeasureperformance.py include them as part of the set of metrics that it measures.

To define a custom metric, create a script file that defines a testUsdviewInputFunction() function that will be passed to testusdview. For example, if you wanted to add a metric named “process prims”, that traversed the stage and processed each prim in some way, you might have a processPrimsMetric.py script that looks something like:

from pxr import Usd, UsdUtils, Usdviewq

def testUsdviewInputFunction(appController):
    with Usdviewq.Timer("process prims", True):
        stage = appController._dataModel.stage
        for prim in stage.Traverse():
            # process prim as needed, etc

See also the “traverse stage” example in pxr/extras/performance/explicitMetrics/stageTraversalMetric.py.

To include your custom metrics when running usdmeasureperformance.py, add your metrics script name and metric name as part of the --custom-metrics script parameter. For example, if you wanted to include the “process prims” metric example for “MyTestAsset.usda”, you would use a usdmeasureperformance.py command line similar to:

python usdmeasureperformance.py MyTestAsset.usda --custom-metrics processPrimsMetric.py:'process prims'

usdmeasureperformance.py will look for your custom metric script relative to the directory from which the usdmeasureperformance.py script is run.