Notifications
Clear all

[Closed] Submit to Deadline

I want my script to automatically send jobs to Deadline, is there any easy way to send it to deadline? Like creating a cmd line to send to the deadline submitter or something…

8 Replies
2 Replies
(@bobo)
Joined: 11 months ago

Posts: 0

Deadline ships with a library of MAXScript functions (SubmitToDeadline_Functions.ms) found in the Repository under submission\3dsmax. You can fileIn() that file and then call the functions to prepare the submission files and perform the actual submission, or you can copy&paste from that file to make your own independent script that does (a porition of) what these functions do.

Note that for example Krakatoa has code that loads those functions and then uses them, so the KrakatoaGUI.ms file would also be a source of interest if you want to see how it is done.

I am quite sure I wrote a short example of a simple batch submitter (sending a folder of MAX files to Deadline or something like that) a while ago for some customer, will try to find it when I get to work.

In general, a Deadline submission requires the following files:
*The Submit Info File which describes the submission data – what plugin will be used for rendering, what the pools and priority should be set to etc.
*The Job Info file which contains info about the job itself – mostly the properties found in the 3dsMax tab of the Monitor for tweaking things like what camera to render, where the output should go etc.
*The MAX file itself.
These 3 files are then passed as arguments to the DeadlineCommandBG.exe file and the job gets submitted. But using the functions mentioned above, these files can be created and sent much easier…

When creating a MAXScript Job on Deadline (a job that runs a custom script instead of rendering the scene), a fourth .MS file would be passed as command line argument and a property in the submission files would be set to denote this is a script job and what the script’s name is. If the job has to perform MAX file saving operations or anything related to the UI, it is also possible to set the job to run in Workstation mode (pulling a Max license!) and you can batch-process/construct/modify scenes and save them on the network. For example, a while ago we had a job that would export XSI files to OBJ, then another dependent MXS job would import these and produce a MAX scene out of the exported data, thus batch-converting scenes overnight…

(@davewortley)
Joined: 11 months ago

Posts: 0

That would be perfect if you can find it. Many thanks.

Here it comes.
It looks scary, but half of the code is remarks, and half of the rest is the function to detect where the repository is (you can hard-code that if you want).
You can expand it by adding more SMTDSetting properties – you can see the full list of SMTD Settings by typing

show SMTDSettings

after having evaluated the SubmitMaxToDeadline_Functions.ms file.

1 Reply
(@davewortley)
Joined: 11 months ago

Posts: 0

Thanks Bobo, will check this out in the morning. I can hard-code the repository as we have that mapped over UNC. And with regards to it looking scary… It’s nothing on the 2000 lines of code i’ve already got in the script it’s going into! Definetly a new personal best for me!

Thanks Bobo works like a dream.

I’ve just editted the code to work from a predefined Repository location and it just submits the current open Max File. Here it is for anyone who needs it…

--Make sure the file is saved before submitting
 saveMaxFile quiet:true
 
 	(
 		global SMTDSettings  --this is the global variable that will contain the submission settings of SMTD
 		global SMTDFunctions --this is the global variable that will contain the submission functions of SMTD
 		
 		local theNetworkRoot = "" --this variable will hold the network root path - the path to the Repository
 		local theHomeRoot = "" --this variable will hold the local home folder of Deadline
 	
 
 		--------------------------------------
 		--MAIN SUBMISSION CODE
 		--------------------------------------
 			theNetworkRoot = "\\\\deadline.yourserver.com\\DeadlineRepository" --and we assign the network root value to our variable for later use.
 			format "Network Root is '%'
" theNetworkRoot 
 						
 			--First, we load the script file with the function definitions
 			local remoteScript = theNetworkRoot + "\\submission\\3dsmax\\SubmitMaxToDeadline_Functions.ms"  --this is the file name on the Repository
 			local localScript = getDir #scripts + "\\SubmitMaxToDeadline_Functions.ms" --to speed things up, we will copy it locally first, then we will load it
 			if doesFileExist remoteScript then --if the remote file exists,
 			(
 				deleteFile localScript --we delete the local file in case it exists already
 				copyFile remoteScript localScript --then we copy the remote file over
 				fileIn localScript quiet:true --and execute it (equivalent to Main Menu > MAXScript > Run Script...)
 				
 				--At this point, SMTDFunctions should have been initialized to the latest version found on the Repository.
 				--Now we have to make sure all defaults (both global and local) are loaded, otherwise the SMTDSettings struct would contain factory defaults
 				SMTDFunctions.loadSettings()
 														
 				---### use currently open file
 				f = maxfilepath + maxfilename
 				
 				SMTDSettings.JobName = getFileNameFile f + " [BATCH SUBMISSION BETA]" --we could for example set the job name to the name of the max file and any custom text we want
 				SMTDSettings.Comment = "Submitted from the batch Custom Submitter" --we could set the comment field
 				SMTDSettings.Priority = 50 --we can override the priority, for example we could increment the priority based on the order of the files or something else
 	
 				--Note that the simple act of loading SMTDFunctions has also initialized a SMTDPaths struct with default paths - type 'show SMTDPaths' to see what paths are available
 				local SubmitInfoFilename = SMTDPaths.tempdir + "\\max_submit_info.job"  --we define the submit info file name which will contain info about the submission settings
 				local JobInfoFilename = SMTDPaths.tempdir  + "\\max_job_info.job" --and the job info file which will contain info about the job's settings (most of them can be changed in the job's Properties dialog of the Monitor
 	
 				SMTDFunctions.CreateSubmitInfoFile SubmitInfoFilename --we then call the function to create a submit info file based on the current SMTDSettings values
 				
 				--We also call the function to create a job info file. It will contain all info about the current scene like renderer settings, list of cameras etc.
 				--This function accepts optional overrides for the render output filename, the tiles to render and the camera to render. 
 				--We can specify these if we know them: renderOutputOverride:"" tileString:"" forceCamera:"" 
 				SMTDFunctions.CreateJobInfoFile JobInfoFilename  --renderOutputOverride: could be set to a string that points at the network location to write the images from the scene being submitted instead of using the renderer's output settings!
 					
 				--Now that we have all the files we need for a submission (the scene file, the submit info and the job info file), we can call the submitter!
 				--To do this, we will have to create an argument list to pass to DeadlineCommandBG
 				local initialArgs = "\"" + SubmitInfoFilename + "\" \"" + JobInfoFilename  + "\" \"" + f + "\" " 
 				
 				--We call the DeadlineCommandBG with our arguments and the timeout (which you can customize)
 				local retcode = SMTDFunctions.waitForCommandToComplete initialArgs SMTDSettings.TimeoutSubmission
 				
 				--Once the command returns, we can try to get the message from the message file. Even if it failed, the file could contain userul error info...
 				local renderMsg = SMTDFunctions.getRenderMessage() 
 				SMTDFunctions.getJobIDFromMessage renderMsg --we can also extract the job ID from the render message - it will be stored in SMTDSettings.DeadlineSubmissionLastJobID
 				
 				if retCode == #success then --if the return code from the call was #success, we show a positive message
 				(
 					format "File % submitted successfully as Job %.

%

" (filenameFromPath f) SMTDSettings.DeadlineSubmissionLastJobID renderMsg
 					SMTDFunctions.CopyExternalFilesToRepository() --copy files to main job
 				)	
 				else --if the code was anything else, we print the message as error message
 					format "Remaining Frames Job Submission FAILED.

%" renderMsg 
 						
 		)--end if root resolved
 	)--end script

–Alternatively, we could submit the file without opening it, but then we would have to provide a custom job info file without real insight into the file’s content, which can be tricky, but a lot faster to execute

Bobo, what info do we get from opening the file, if say I had 10 files I wanted to be submitted to Deadline, all very similar, could it be possible to get the info required from just opening one file and then submitting all the others based on this? Would be much much quicker if we didn’t have to wait for each pass to submit individually, and just let it batch submit once all passes were setup without opening each one.

rgds

Dave

1 Reply
(@bobo)
Joined: 11 months ago

Posts: 0

The info taken from the file is related to getting the frame range, render output path, Max version, all renderer settings etc. The functions that create the job and submit info files use the current scene to extract all that data. If you want to submit several Max files without opening them, it is possible, but you will have to generate the .job files yourself and put whatever info you think is needed into them without looking inside the Max files. Use the .job files created by the current script as templates and try to provide all that data via UI controls or hard-code them.

Dave,
How about selecting all the Max files you want to batch submit in the deadline monitor via the “Submit 3dsMax Job to Deadline” py script? This py submission script handles multiple Max files and provides some basic functionality which is generic between scene files. You could execute the py code directly from within Max using Blur’s python wrapper if this is necessary.
Mike