Notifications
Clear all

[Closed] A Benchmarking Study

I want to start new thread where everyone can post some benchmarking tests to show interesting results of functions or methods performance, memory usage, etc.

As a first sample I would like to share the comparison of 5 different ways of binding rotations of two objects (it’s just a sample). I picked these methods:

script controller

param wire

expression

constraint

expose tm

Here is a result:


(
	delete objects
	
-- script 
	b1 = box name:"s_target" width:10 length:10 height:10 pos:[-12,-12,0] wirecolor:(color 210 210 0)
	b2 = box name:"s_source" width:14 length:14 height:14 pos:[12,-12,-2] wirecolor:(color 250 140 0)
	
	sc = b1.rotation.controller = rotation_Script()
	sc.addtarget "source_rotation" b2.rotation.controller
	sc.setexpression "source_rotation"

-- wire 
	b3 = box name:"w_target" width:10 length:10 height:10 pos:[-12,12,0] wirecolor:(color 70 210 0)
	b4 = box name:"w_source" width:14 length:14 height:14 pos:[12,12,-2] wirecolor:(color 90 240 0)
	
	paramwire.connect b4.controller[#rotation] b3.controller[#rotation] "rotation"

-- expression
	b5 = box name:"x_target" width:10 length:10 height:10 pos:[-12,36,0] wirecolor:(color 0 80 230)
	b6 = box name:"x_source" width:14 length:14 height:14 pos:[12,36,-2] wirecolor:(color 0 110 240)
	
	for k=1 to 3 do
	(
		ex = b5.rotation.controller[k].controller = float_expression()
		ex.addscalartarget "source_rotation" b6.rotation.controller[k]
		ex.setexpression "source_rotation"
	)

-- constraint	
	b7 = box name:"c_target" width:10 length:10 height:10 pos:[-12,60,0] wirecolor:(color 0 120 120)
	b8 = box name:"c_source" width:14 length:14 height:14 pos:[12,60,-2] wirecolor:(color 0 180 180)
	
	cc = b7.rotation.controller = orientation_constraint()
	cc.appendtarget b8 100
	
-- expose tm	
	b9 = box name:"e_target" width:10 length:10 height:10 pos:[-12,84,0] wirecolor:(color 120 0 120)
	b10 = box name:"e_source" width:14 length:14 height:14 pos:[12,84,-2] wirecolor:(color 180 0 180)
	ex = exposeTM name:"e_expose" size:10 pos:[12,84,12] wirecolor:(color 220 0 220)
	ex.exposeNode = b10
	b9.rotation.controller[1].controller = ex.localEulerX.controller
	b9.rotation.controller[2].controller = ex.localEulerY.controller
	b9.rotation.controller[3].controller = ex.localEulerZ.controller
	
-- tests
	count = 10000
	
	gc()
	t1 = timestamp()
	m1 = heapfree
	for k=1 to count do b1.rotation
	format "script		>>	time:%	memory:%
" (timestamp() - t1) (m1 - heapfree)

	gc()
	t1 = timestamp()
	m1 = heapfree
	for k=1 to count do b3.rotation
	format "param wire	>>	time:%	memory:%
" (timestamp() - t1) (m1 - heapfree)

	gc()
	t1 = timestamp()
	m1 = heapfree
	for k=1 to count do b5.rotation
	format "expression	>>	time:%	memory:%
" (timestamp() - t1) (m1 - heapfree)

	gc()
	t1 = timestamp()
	m1 = heapfree
	for k=1 to count do b7.rotation
	format "constraint	>>	time:%	memory:%
" (timestamp() - t1) (m1 - heapfree)

	gc()
	t1 = timestamp()
	m1 = heapfree
	for k=1 to count do b9.rotation
	format "expose tm	>>	time:%	memory:%
" (timestamp() - t1) (m1 - heapfree)
)


script		>>	time:78	memory:3613760L
param wire	>>	time:26	memory:128L
expression	>>	time:8	memory:128L
constraint	>>	time:7	memory:128L
expose tm	>>	time:8	memory:128L

check the results, and you will be surprised to know how bad #script_controllers leak, or how safe to use ExposeTM node for performance for example…

21 Replies

do you believe that any mxs block with undo OFF always faster than with undo ON?
what is faster .net String.Split or mxs filterString, and why?
what is the difference between using if … do vs if … then?
do you know the fastest way of the drawing bitmap?
why meshop.getpolysusingface is VERY slow and how to get it faster?



and many other questions to know. and many other answers to get.

c’mon people. is it not interesting?

 lo1

Great idea for a thread! I’d like to point out a bit of unfairness in your benchmark. You are testing a rotation_script vs. 3 float_expressions.
I think this is more fair:

(
	delete objects
	
-- script 
	b1 = box name:"s_target" width:10 length:10 height:10 pos:[-12,-12,0] wirecolor:(color 210 210 0)
	b2 = box name:"s_source" width:14 length:14 height:14 pos:[12,-12,-2] wirecolor:(color 250 140 0)
	
	for k = 1 to 3 do
	(
		ex = b1.rotation.controller[k].controller = float_script()
		ex.addtarget "source_rotation" b2.rotation.controller[k]
		ex.setexpression "source_rotation"
	)	

-- expression
	b5 = box name:"x_target" width:10 length:10 height:10 pos:[-12,36,0] wirecolor:(color 0 80 230)
	b6 = box name:"x_source" width:14 length:14 height:14 pos:[12,36,-2] wirecolor:(color 0 110 240)
	
	for k=1 to 3 do
	(
		ex = b5.rotation.controller[k].controller = float_expression()
		ex.addscalartarget "source_rotation" b6.rotation.controller[k]
		ex.setexpression "source_rotation"
	)

-- tests
	count = 10000
	
	gc()
	t1 = timestamp()
	m1 = heapfree
	for k=1 to count do b1.rotation
	format "script		>>	time:%	memory:%
" (timestamp() - t1) (m1 - heapfree)
	
	gc()
	t1 = timestamp()
	m1 = heapfree
	for k=1 to count do b5.rotation
	format "expression	>>	time:%	memory:%
" (timestamp() - t1) (m1 - heapfree)
)
script		>>	time:7	memory:1576L
expression	>>	time:5	memory:112L
 lo1

And in the case of if/do/then, there is absolutely no performance difference. I assume it gets compiled to the same code.

count = 1000000
(
		
for k=1 to count do ok --helps to improve benchmark accuracy

gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do if 1 + 1 == 2 do ok
format "if-do	>>	time:%	memory:%
" (timestamp() - t1) (m1 - heapfree)

gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do if 1 + 1 == 2 then ok
format "if-then	>>	time:%	memory:%
" (timestamp() - t1) (m1 - heapfree)
)
if-do	>>	time:559	memory:112L
if-then	>>	time:553	memory:112L
 lo1
count = 10000
(
local str = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. 
	Praesent nibh lorem, vehicula non dignissim in, imperdiet ac orci. 
	Nam placerat dui ut orci consequat vitae rhoncus leo adipiscing. 
	Integer neque est, mollis et consectetur at, eleifend porttitor nisl. 
	Etiam id est ipsum."
	
for k=1 to count do ok --helps to improve benchmark accuracy

local token = " "
local tokens = dotNetObject "System.String[]" 1
tokens.setValue " " 0

local filterStr = filterString
local strSplit = dotNetObject "System.String" str
local strSplitOptions = (dotnetClass "System.Stringsplitoptions").none

gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do strSplit.split tokens strSplitOptions
format "String.Split	>>	time:%	memory:%
" (timestamp() - t1) (m1 - heapfree)

gc()
t1 = timestamp()
m1 = heapfree
for k=1 to count do filterStr str token
format "filterString	>>	time:%	memory:%
" (timestamp() - t1) (m1 - heapfree)
)
String.Split	>>	time:1788	memory:25075816L
filterString	>>	time:326	memory:24274200L

No matter what, it seems filterString will always win. I assume this is because string.split is constantly converting the result to a maxscript string.

Interesting stuff chaps, thanks.

using denisT test:

script >> time:64 memory:3657136L
param wire >> time:22 memory:128L
expression >> time:7 memory:128L
constraint >> time:7 memory:520760L
expose tm >> time:9 memory:1231272L

about some poly functions, and why they area slow,
depends on how you use them… mxs-help is ambiguous about many things…
I have the impression that they came from editable_mesh,
where if I’m correct, those where the only way,
scanning the whole mesh… checking… checkin…
while e_poly uses more memory, we know why…
and they may seem slow, but happens that most people abuse them,
I think math people would say: its like O(pow N N) instead of O(N)
Most scripts out there(even if ecrypted), that you know they were made very Wrong!,
The problem is that the correct ways is usually complex.
but the benefit is that you can feed them with a whole array.
I suppose max uses that for things like converting selections, for example

About the undo(speed), I suppose it’s not worry if used correctly,
what worries me is memory…
something that people may not know about the undo, is that
internally max ‘puts’ entries into the undo that are grouped in contexts
the number of puts, can consume your whole ram, and probably increasing time,
reducing them is key.

lo example when the value is set ‘=’ and when the rotation is applied to the object referenced:

for k=1 to count do b1.rotation = b1.rotation
for k=1 to count do b5.rotation = b5.rotation

script >> time:366 memory:3490392L
expression >> time:85 memory:128L

for k=1 to count do b2.rotation = b2.rotation
for k=1 to count do b6.rotation = b6.rotation

script >> time:299 memory:128L
expression >> time:250 memory:128L

some people think that using of node (addnode method) in a script controller solves a problem of circular dependency. well… it means you don’t see an error message and the controller kinda works.

check the sample, and you will be disappointed:


 (
 	delete objects
 	clearlistener()
 	local node, parent
 	count = 3
 	global nodes = for k=0 to count-1 collect in parent 
 	(
 		node = box name:("B" + k as string) width:10 length:10 height:10 pos:[k*15,0,-5] wirecolor:(color 0 (250-k*10) (250-k*10))
 			
 		if isvalidnode parent do
 		(
 			sc = node.pos.controller[1].controller = float_script()
 			sc.addconstant "id" k
 			sc.addnode "parent" parent
 			sc.setexpression ("print (\"id: \" + (id as string)); parent.pos.controller[1].value; " + sc.script)
 		)	
 		parent = node
 		node
 	)
 	format ">>>>>>
"	
 	move nodes[1] [0,0,0]
 )
 

In this sample I make hierarchical chain (in real life it might be a bone chain).
Position controller of every child depends on its parent’s controller.
Changing of root’s transform should cause the updating of all script controllers. Follow normal logic we have to have 2 updates (because we have 2 controllers without circular dependency, or backward dependency).
But check the print… and you will see 3 updates.

Now change the count parameter to 5… how many updates for 4 children? 10!
How many updates for 10 children (set count to 11)? …55

I showed how a script controller leaks. Now draw your conclusion.

can you be more especific, is this the right thread?

Just in case,
I don’t see your post contradicting mine,
in fact it enforces what I said about bi-directionality.
But there are ways to avoid that, If I understand what you’re trying to say…

using my way,
5 interactions:
>>>>>>
OK
“id: 4”
“id: 3”
“id: 2”
“id: 1”
OK

and for 10 :
>>>>>>
OK
“id: 9”
“id: 8”
“id: 7”
“id: 6”
“id: 5”
“id: 4”
“id: 3”
“id: 2”
“id: 1”
OK

but I have to say, there is no leaking, it’s exactly as it’s supposed to be…

could you show the code of your way, please?

Page 1 / 2