[Closed] Pflow – custom particle positions [need advice]

I suggest you to read the topic “how to make it faster” in the maxscript reference.

Anyway try to change your for loop with this one:

for thePointData in pos do
  
  (
  
  --give birth to a new particle
  
  pCont.AddParticle()
  pCont.particleIndex = pCont.NumParticles()
  pCont.particleAge = 0
  pCont.particlePosition = thePointData*1000
  pCont.particleSpeed = [0, 0, 0]
  )

keep in mind that script operator are always slower than normal operators.

Thanks for directions !

I’ve splitted my 3D array in 3 1D arrays.
I’m also using append to try to avoid memory leaks.

Speed is one of my concerns but just to be sure i’m not following a wrong path : a one million elements array in maxscript is something crazy or standard ?

I’ve worked a bit further in my scripts and the problem comes from the file reading part (PFlow seems ok handling a million particles).
It seems that reading 150 000 positions from a binary files is pushing max to its limits.
the source file is 20 Mo, i would have guessed that was cake for max. I must do something wrong or asking too much, here is the code :

    
 	   chemin = getOpenFileName caption:"Choose a file"
 	   f = fopen chemin "rb"
 	   
 	/* Read Simulation Header*/
 	fseek f 4 #seek_cur /*  skip fortran header*/
 	npart = readBEL f
 	fseek f 8 #seek_cur 
 	procid = readBEL f
 	fseek f 8 #seek_cur
 	
 	/* Simulation bounds */
 	xminproc = readBEF f
 	xmaxproc = readBEF f
 	yminproc = readBEF f
 	ymaxproc = readBEF f
 	zminproc = readBEF f
 	zmaxproc = readBEF f
    
	fseek f 8 #seek_cur
 	
 	/* Particles --------------*/
 	
 	/* Positions */
 	maxn = maxPart.value
 	for n = 1 to maxn do 
 	(
 			data = readfloat f 
 			append posx data
 			bloc = readfloat f 
 			append posy data
 			bloc = readfloat f 
 			append posz data
 	)
 

Pflow can handle a million particles without a problem, I suspect that the intermediate storage of particle data might be the cause for crashes.

Have you thought of moving the scripted utility that reads the binary file into memory to the Birth Script operator? All you would have to do is read a particle position at a time and create it as you are already doing, but without the whole global array.

Also, have you tried increasing the MAXScript Memory (HeapSize) under Customize>Preferences>MAXScript? I would expect MAXScript to do this automatically, but you never know. Btw, is this in 32 or 64 bit Max and what version?

Yet another alternative would be Krakatoa in Evaluation mode.
Krakatoa provides a MAXScript Interface that can save particles to its own PRT file format.
The PRT file format is compressed and can be read very fast using the Krakatoa PFlow Operators. Of course, this might be slightly outside of your technical level, but if you want to try it out, I could give you some guidance.

Here is a short overview of the process:

*In your scripted utility, instead of writing to an array, you simply write to PRT via this interface:
http://software.primefocusworld.com/software/support/krakatoa/krakatoaparticleostream_maxscript_value.php
*You run that script just once to create a PRT representation of your data and you are ready to use Krakatoa.
*In Max, you create a PRT Loader object and pick the new PRT file.
*In PFlow, you add a Krakatoa PRT Loader Birth and pick the PRT Loader from the scene.
*Done.

You can get the evaluation version of Krakatoa from
http://software.primefocusworld.com/software/products/krakatoa/download/

Oh thanks. Many tips here.
I’ll try this week-end to read the file directly in the birth script event and then evaluate converting my files to the PRT Krakatoa format for faster reading, which seams quite straitforward.

Thanks a lot.

Just in case it can be helpfull to other users looking for the same info (and because i often read forum but don’t contribute a lot i realise) i tested the 2 methods suggested by Bobo :

I – reading the million-particles file directly in the birth script and creating particles on the fly
It works! Max doesn’t crash, and the particles are created.
Cons : Working on the Pflow is quite slow and rendering is very slow and painfull on a duo core windows 64bit system. I’m not used to optimizations though, there must be some ways to speed it i’m sure.

II – using krakatoa PRT file format
Wow.
I just tested. Loading a million particle with the PRT loader object just takes a second. I can inspect in real time my million particles cloud. I’m so impressed i had to write something here
I’ve made a render test. Works like a charm too. Really impressive.
Cons : Krakatoa requires a license. I’ll try to convince my client to purchase one.

Pierre-Gérard David

You made my day!

Some notes on the licensing thing:
*The only limitations of the Evaluation version are that resolutions above 480×360 render with a watermark, network rendering is disabled and some bonus PFlow tools like the very fast Krakatoa Collision and the Krakatoa Geometry Test are disabled. Everything else works and you can use it as much as you want. For free.
*We will provide you with a free evaluation license for 15 days if you submit a request. That could be enough to finish a project, just in case your client cannot be convinced to spend the $$$. Of course I would hope he would, as it pays our salaries
*At that point, we would hope you would be so in love with Krakatoa that you would budget it for future projects or at least suggest it as option to other clients, thus increasing awareness of its awesomeness…

Seriously though, I am so glad these approaches work for you.

Also note that with Krakatoa 1.5, this gives you an extra level of flexibility to tweak the appearance of your particles since you can modify any channels using the MagmaFlow editor, deform particles using regular Max modifiers, cull particles in areas defined by geometry volumes and so on. Be sure to read through the documentation for some inspiration:

Deforming, Culling, Recycling of PFlow Partricles:
http://software.primefocusworld.com/software/support/krakatoa/using_prt_loader_birth_and_update.php

Practical use of MagmaFlow:
http://software.primefocusworld.com/software/support/krakatoa/magmaflow_practical_example_gijoe.php

MagmaFlow Examples:
http://software.primefocusworld.com/software/support/krakatoa/particle_channels_editor_examples.php

Hi Bobo,

Clever, wish I had the time to evaluate it. What is the Krakatoa Magaflow Editor UI made with? it looks a bit like Kees’ Helium UI.

Hey ! it’s me again.

your tool is really awesome

Can i request some more guidance ?

1 . I can’t find the macroscripts page (dead link) on the prime focus website which is supposed to describe how to install the krakatoa toolbar. How do i do that ?

2. i have created a maxscript wich writes a PRT file from my custom source file. It is a snapshot of my million particles’s positions AND velocities at a given time.
   Is there a quick way (PFlow/krakatoa combination?) to animate those particles, based on the written velocities ? (to just make them “continue” their movement)

Best regards,

Pierre-Gérard

Strange, not sure where that page went.
Check out the offline CHM of the Help – it has the page.
http://software.primefocusworld.com/downloads/Krakatoa_1_5_1_38002_CHM.zip

Yes, but due to some earlier design decisions, it requires a little jumping through hoops

*When saving the Particles, do not save the Velocity into a “Velocity” channel, instead save into ANY float16[3] channel like a “Mapping10” or a custom channel (you can add new custom channels to the system in the Channels rollout – for example I created one called “MyVelocity”). But for speed and simplicity, let’s assume you will dump the velocity into the Mapping10 channel.
*In the more general case when saving an actual scene particle system like PFlow etc, you can add a global KCM in the Global Render Values rollout and add a MagmaFlow that copies all values from the Velocity channel into your channel. When saving the particle frame, your channel in the PRT will be populated with the actual velocity data.

*Now when loading, create a PRT Loader, set to 100% in viewport.
*Pick the single frame you saved.
*Check “Limit To Custom Range” and hit the “Range” button to set start and end frame to the same frame (0).
RESULT: The particles will appear and stay on all frames.

*Add a Krakatoa Channels Modifier.
*Set the Input to Position, set the Output to Position
*Add a Mapping10 Channel Input.
*Select the Mapping10 Input and the Position Input and hit + on the Numeric Pad to add the two together.
RESULT: The particles will now appear offset significantly since the Speed is stored in units per second.

*Add a Script Input and enter “currenttime.frame/FrameRate” in the script field.
*Select both the Mapping10 Channel Input and the Script Input and hit * on the Numeric Pad to add a Multiply Operator.
RESULT: Now the Velocity is scaled to the correct frame rate and moving the time slider/rendering will produce particles moving along the same velocity vector over time.

*At this point we can add a second KCM and set it to Mapping10 Input -> Velocity Output.
*Switch the PRT Loader to “Display Velocities” instead of “Large Dots” and you will see the actual velocities.

Why do we have to do all this?

Early on before there was MagmaFlow, the design decision was made that loading a single frame with valid velocities and displaying it over multiple frames CANNOT produce motion blur. So if you save the Velocity channel as usual to frame 0000 and either check “Load Single Frame Only” or “Limit To Custom Range”, the particles will render on every frame, but the Velocity will be internally reset to [0,0,0] because in fact they cannot be moving

Now that MagmaFlow could move them by integrating the position and velocity data over time, this design decision comes to bite us in the butts. But, and there is a but, since we support arbitrary channels, there is usually some workaround like the one above to save them

Hope this helps.