tutorial is provided by the folks at Martian
Labs. There is no warrantee, written or implied, pertaining
to its function or correctness & no guarantee that it will even
work or keep your systems in good working order. In other words,
use at your own risk.
we start, the basic idea to rendering Houdini scenes with Dr Queue
is to write out IFD (Instantaneous Frame Description) files and
render each of them from the command line with Houdini's Mantra
renderer. Dr Queue lends itself to such a task using the "General
Job" which allows you to execute a shell script on all machines
on the queue. Like all things Houdini, there exists more than one
way to accomplish this task. We will present what we feel is the
most clear & concise method that can be used by beginners and
modified for professionals.
environment setup for our basic example should be only that which
is covered in the Dr Queue installation. As an overview:
Dr Queue server needs to exist.
drqueue files need to be in one, central location to which all
machines have read/write access.
machine needs to run the appropriate slave program for its OS.
the time the slave is launched, all Houdini environment variables
must be set. (Note: once the slave is launched,
all env variables set at the time of launch will remain set and
unchanged until the slave is killed regardless of how variables
are changed/removed/added after launch)
machine must have the environment variables DRQUEUE_ROOT and DRQUEUE_MASTER
machines must be properly licensed to run mantra.
accomplish all the environment variable settings by launching the
slave program as the user "render" who has all the correct
settings in his login script (.bashrc or .tcshrc in our case).
Houdini, for the simple example, we'll assume no external referencing
to shaders or plugins. Dr Queue will render IFD files, which can
be created from a typical mantra output operator. To create the
drop a typical mantra output node in the output network editor.
the parameters of the output operator by hitting the 'P' key while
your mouse is over the output network editor.
an appropriate output picture location. (Note:
"$F4" is padded $F - it's the same as doing `padzero(4,$F)`
& yields 0001, 0002, 0003, etc.
"Render Any Frame" to "Render Frame Range"
and check that the range is correct.
the "Generate Script File" option.
an appropriate location for the IFD files.
then just hit the render button. This should generate a directory
full of IFD files. To help understand the process, you can now open
a terminal window (or Houdini Command prompt if you're in Windows
land) and test a single frame render with the command:
should render out frame 1 of your scene to the file /path/to/project/pic/test_scene.0001.tif.
Assuming that works, you're ready to setup a Dr Queue job. If the
test did not work, you'll need to double check paths for both picture
and ifd. You can also get a little more information by using "mantra
-V1a" as opposed to just "mantra".
up a Dr Queue Job
in mind that a Dr Queue "general job" simply runs commands
on each node in the queue, it should make sense that the command
we used above as our test will also be the one we want Dr Queue
execute on each machine. To do this, first open up the drqueue manager.
(To ensure your settings are correct, you may want to check the
"Computers" tab to see that at least one computer is showing
up, it should look something like this.
If you don't see any computers in the list, make sure that the slaves
are running and that your DRQUEUE environment variables are set
correctly on your workstation).
the Dr Queue Manager "Jobs" tab, right click anywhere
in the "Job Status" selection window and chose
"New Job". This should bring up "New
Job" dialog. For our mantra render, do the following:
the "Kind of job" is set to "General"
the job some meaningful name (for display in the manager).
in the command you want to be executed. The one non-standard Dr
Queueism here is that the padded frame number needs to be substituted
with "$DRQUEUE_PADFRAME" (see this
image as a guide).
the frame range to match the scene.
check other options & hit the "Submit" button at
the bottom of the dialog (you may have to scroll the dialog to
see the button)
that should do it. Your job should now show up in the Job Status
selection window of the Jobs tab in Dr Queue manager. You can watch
frame logs and check render stats by right clicking your job and
choosing "details," where right-clicking on frame and
choosing "watch frame log" will bring up the log. All
frames should then render to /path/to/project/pic/test_scene.$F4.tif
By default, houdini looks in two places for shaders and/or plugins
- $HFS and ~/houdiniX.X. If you have [non-embedded] shaders in your
scene, in order for all machines on the queue to use them, you can
either copy all the files in your ~/houdiniX.X to the render user's
homedir on each machine (probably not the best way to do it) or
you can modify the HOUDINI_PATH variable so that it looks for these
If you would like to be able to change a path (or any other environment
variable) at any time after you launch the slave, you can do it
at render time by adding a shell script to the general job's render
command like so:
/new/path; mantra < ./ifd/foo.$DRQUEUE_PADFRAME.ifd
follows, then, that any number of things can happen in that shell
script & therein lies the flexibility that one will need to
use Dr Queue in production. At Martian Labs, we run two shell scripts
before the mantra command that set up a slurry of environment variables
for the particular job/shot we're working on and then changes directories
to the correct location so that everything remains standardized
across the project.
The more humans become involved in the process, the more likely
it is that errors will be introduced. It's often the case that the
render command will get mistyped in the tedium of typing out a long
string. We have come up with a couple methods for automating the
shell script that outputs a correct render command for Dr Queue
command line submission program.
made a *rough* version of the shell script that's available here.
Execution should be self explanatory.
made a *rough* version of the command line submission program available
here: martianSendJob.h martianSendJob.c.
Please note: This source code is COMPLETELY UNSUPPORTED
& is only known to work at Martian Labs (though there are no
proprietary operations made by the code, so it's possible it will
work elsewhere) with Dr Queue v 0.63. It is up to the reader to
compile the code (done by modifying the Dr Queue makefile) &
ensure that it is functioning properly. Another note: It is also
possible that the general sendjob submission in the current source
tree works properly in which case this source code will be of little
use, however, at the time we last upgraded, general sendjobs were