[Closed] A Benchmarking Study
I want to start new thread where everyone can post some benchmarking tests to show interesting results of functions or methods performance, memory usage, etc.
As a first sample I would like to share the comparison of 5 different ways of binding rotations of two objects (it’s just a sample). I picked these methods:
script controller
param wire
expression
constraint
expose tm
Here is a result:
(
delete objects
-- script
b1 = box name:"s_target" width:10 length:10 height:10 pos:[-12,-12,0] wirecolor:(color 210 210 0)
b2 = box name:"s_source" width:14 length:14 height:14 pos:[12,-12,-2] wirecolor:(color 250 140 0)
sc = b1.rotation.controller = rotation_Script()
sc.addtarget "source_rotation" b2.rotation.controller
sc.setexpression "source_rotation"
-- wire
b3 = box name:"w_target" width:10 length:10 height:10 pos:[-12,12,0] wirecolor:(color 70 210 0)
b4 = box name:"w_source" width:14 length:14 height:14 pos:[12,12,-2] wirecolor:(color 90 240 0)
paramwire.connect b4.controller[#rotation] b3.controller[#rotation] "rotation"
-- expression
b5 = box name:"x_target" width:10 length:10 height:10 pos:[-12,36,0] wirecolor:(color 0 80 230)
b6 = box name:"x_source" width:14 length:14 height:14 pos:[12,36,-2] wirecolor:(color 0 110 240)
for k=1 to 3 do
(
ex = b5.rotation.controller[k].controller = float_expression()
ex.addscalartarget "source_rotation" b6.rotation.controller[k]
ex.setexpression "source_rotation"
)
-- constraint
b7 = box name:"c_target" width:10 length:10 height:10 pos:[-12,60,0] wirecolor:(color 0 120 120)
b8 = box name:"c_source" width:14 length:14 height:14 pos:[12,60,-2] wirecolor:(color 0 180 180)
cc = b7.rotation.controller = orientation_constraint()
cc.appendtarget b8 100
-- expose tm
b9 = box name:"e_target" width:10 length:10 height:10 pos:[-12,84,0] wirecolor:(color 120 0 120)
b10 = box name:"e_source" width:14 length:14 height:14 pos:[12,84,-2] wirecolor:(color 180 0 180)
ex = exposeTM name:"e_expose" size:10 pos:[12,84,12] wirecolor:(color 220 0 220)
ex.exposeNode = b10
b9.rotation.controller[1].controller = ex.localEulerX.controller
b9.rotation.controller[2].controller = ex.localEulerY.controller
b9.rotation.controller[3].controller = ex.localEulerZ.controller
-- tests
count = 10000
gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do b1.rotation
format "script >> time:% memory:%
" (timestamp() - t1) (m1 - heapfree)
gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do b3.rotation
format "param wire >> time:% memory:%
" (timestamp() - t1) (m1 - heapfree)
gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do b5.rotation
format "expression >> time:% memory:%
" (timestamp() - t1) (m1 - heapfree)
gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do b7.rotation
format "constraint >> time:% memory:%
" (timestamp() - t1) (m1 - heapfree)
gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do b9.rotation
format "expose tm >> time:% memory:%
" (timestamp() - t1) (m1 - heapfree)
)
script >> time:78 memory:3613760L
param wire >> time:26 memory:128L
expression >> time:8 memory:128L
constraint >> time:7 memory:128L
expose tm >> time:8 memory:128L
check the results, and you will be surprised to know how bad #script_controllers leak, or how safe to use ExposeTM node for performance for example…
do you believe that any mxs block with undo OFF always faster than with undo ON?
what is faster .net String.Split or mxs filterString, and why?
what is the difference between using if … do vs if … then?
do you know the fastest way of the drawing bitmap?
why meshop.getpolysusingface is VERY slow and how to get it faster?
…
…
…
and many other questions to know. and many other answers to get.
c’mon people. is it not interesting?
Great idea for a thread! I’d like to point out a bit of unfairness in your benchmark. You are testing a rotation_script vs. 3 float_expressions.
I think this is more fair:
(
delete objects
-- script
b1 = box name:"s_target" width:10 length:10 height:10 pos:[-12,-12,0] wirecolor:(color 210 210 0)
b2 = box name:"s_source" width:14 length:14 height:14 pos:[12,-12,-2] wirecolor:(color 250 140 0)
for k = 1 to 3 do
(
ex = b1.rotation.controller[k].controller = float_script()
ex.addtarget "source_rotation" b2.rotation.controller[k]
ex.setexpression "source_rotation"
)
-- expression
b5 = box name:"x_target" width:10 length:10 height:10 pos:[-12,36,0] wirecolor:(color 0 80 230)
b6 = box name:"x_source" width:14 length:14 height:14 pos:[12,36,-2] wirecolor:(color 0 110 240)
for k=1 to 3 do
(
ex = b5.rotation.controller[k].controller = float_expression()
ex.addscalartarget "source_rotation" b6.rotation.controller[k]
ex.setexpression "source_rotation"
)
-- tests
count = 10000
gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do b1.rotation
format "script >> time:% memory:%
" (timestamp() - t1) (m1 - heapfree)
gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do b5.rotation
format "expression >> time:% memory:%
" (timestamp() - t1) (m1 - heapfree)
)
script >> time:7 memory:1576L
expression >> time:5 memory:112L
And in the case of if/do/then, there is absolutely no performance difference. I assume it gets compiled to the same code.
count = 1000000
(
for k=1 to count do ok --helps to improve benchmark accuracy
gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do if 1 + 1 == 2 do ok
format "if-do >> time:% memory:%
" (timestamp() - t1) (m1 - heapfree)
gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do if 1 + 1 == 2 then ok
format "if-then >> time:% memory:%
" (timestamp() - t1) (m1 - heapfree)
)
if-do >> time:559 memory:112L
if-then >> time:553 memory:112L
count = 10000
(
local str = "Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Praesent nibh lorem, vehicula non dignissim in, imperdiet ac orci.
Nam placerat dui ut orci consequat vitae rhoncus leo adipiscing.
Integer neque est, mollis et consectetur at, eleifend porttitor nisl.
Etiam id est ipsum."
for k=1 to count do ok --helps to improve benchmark accuracy
local token = " "
local tokens = dotNetObject "System.String[]" 1
tokens.setValue " " 0
local filterStr = filterString
local strSplit = dotNetObject "System.String" str
local strSplitOptions = (dotnetClass "System.Stringsplitoptions").none
gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do strSplit.split tokens strSplitOptions
format "String.Split >> time:% memory:%
" (timestamp() - t1) (m1 - heapfree)
gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do filterStr str token
format "filterString >> time:% memory:%
" (timestamp() - t1) (m1 - heapfree)
)
String.Split >> time:1788 memory:25075816L
filterString >> time:326 memory:24274200L
No matter what, it seems filterString will always win. I assume this is because string.split is constantly converting the result to a maxscript string.
using denisT test:
script >> time:64 memory:3657136L
param wire >> time:22 memory:128L
expression >> time:7 memory:128L
constraint >> time:7 memory:520760L
expose tm >> time:9 memory:1231272L
–
about some poly functions, and why they area slow,
depends on how you use them… mxs-help is ambiguous about many things…
I have the impression that they came from editable_mesh,
where if I’m correct, those where the only way,
scanning the whole mesh… checking… checkin…
while e_poly uses more memory, we know why…
and they may seem slow, but happens that most people abuse them,
I think math people would say: its like O(pow N N) instead of O(N)
Most scripts out there(even if ecrypted), that you know they were made very Wrong!,
The problem is that the correct ways is usually complex.
but the benefit is that you can feed them with a whole array.
I suppose max uses that for things like converting selections, for example
About the undo(speed), I suppose it’s not worry if used correctly,
what worries me is memory…
something that people may not know about the undo, is that
internally max ‘puts’ entries into the undo that are grouped in contexts
the number of puts, can consume your whole ram, and probably increasing time,
reducing them is key.
–
lo example when the value is set ‘=’ and when the rotation is applied to the object referenced:
for k=1 to count do b1.rotation = b1.rotation
for k=1 to count do b5.rotation = b5.rotation
script >> time:366 memory:3490392L
expression >> time:85 memory:128L
for k=1 to count do b2.rotation = b2.rotation
for k=1 to count do b6.rotation = b6.rotation
script >> time:299 memory:128L
expression >> time:250 memory:128L
some people think that using of node (addnode method) in a script controller solves a problem of circular dependency. well… it means you don’t see an error message and the controller kinda works.
check the sample, and you will be disappointed:
(
delete objects
clearlistener()
local node, parent
count = 3
global nodes = for k=0 to count-1 collect in parent
(
node = box name:("B" + k as string) width:10 length:10 height:10 pos:[k*15,0,-5] wirecolor:(color 0 (250-k*10) (250-k*10))
if isvalidnode parent do
(
sc = node.pos.controller[1].controller = float_script()
sc.addconstant "id" k
sc.addnode "parent" parent
sc.setexpression ("print (\"id: \" + (id as string)); parent.pos.controller[1].value; " + sc.script)
)
parent = node
node
)
format ">>>>>>
"
move nodes[1] [0,0,0]
)
In this sample I make hierarchical chain (in real life it might be a bone chain).
Position controller of every child depends on its parent’s controller.
Changing of root’s transform should cause the updating of all script controllers. Follow normal logic we have to have 2 updates (because we have 2 controllers without circular dependency, or backward dependency).
But check the print… and you will see 3 updates.
Now change the count parameter to 5… how many updates for 4 children? 10!
How many updates for 10 children (set count to 11)? …55
I showed how a script controller leaks. Now draw your conclusion.
can you be more especific, is this the right thread?
Just in case,
I don’t see your post contradicting mine,
in fact it enforces what I said about bi-directionality.
But there are ways to avoid that, If I understand what you’re trying to say…
using my way,
5 interactions:
>>>>>>
OK
“id: 4”
“id: 3”
“id: 2”
“id: 1”
OK
and for 10 :
>>>>>>
OK
“id: 9”
“id: 8”
“id: 7”
“id: 6”
“id: 5”
“id: 4”
“id: 3”
“id: 2”
“id: 1”
OK
but I have to say, there is no leaking, it’s exactly as it’s supposed to be…