Instructions on how to get a PROOF parallel analysis to work on a
multi-core and/or multi-CPU machine.
Author: Anthony Palladino
Date: 02 Sep 2010

-----------------------------------------------------------------

PROOF works with TSelectors.
In this example, I show how we should use TSelectors that inherit from
the usertree, instead of writing our methods directly in those generated 
files. The reason is to ease the amount of work necessary when we modify
the tree structure.



Open a .root file:

]$ root -l
root [0] TFile *f1 = TFile::Open("/net/group1/data2/$USER/pen08/results/result84619.root")


Make a skeleton TSelector from usertree:

root [1] usertree->MakeSelector("usertreeSkeleton")
Info in <TTreePlayer::MakeClass>: Files: usertreeSkeleton.h and usertreeSkeleton.C generated from TTree: usertree


Create an analysis class that inherits from usertreeSkeleton.
I called this pen08Selector
Reimplement the constructor, destructor, Begin, Process and Terminate methods.


Create a PAR package containing the generated usertreeSkeleton class.
First make a directory structure as follows:

[ap4ax@node6 myselector]$ tree libusertreeSkeleton/
libusertreeSkeleton/
|-- Makefile
|-- PROOF-INF
|   |-- BUILD.sh
|   |-- SETUP.C
|   `-- SETUP.C~
|-- usertreeSkeleton.C
`-- usertreeSkeleton.h

1 directory, 6 files

Now create the package:

[ap4ax@node6 myselector]$ tar -czf libusertreeSkeleton.par libusertreeSkeleton

Now delete the directory (it will cause problems later because a symlink must be created with the same name):

[ap4ax@node6 myselector]$ rm -rfv libusertreeSkeleton



Let's see what we got:

[ap4ax@node6 myselector]$ ll
total 88K
-rw-r--r-- 1 ap4ax ap4ax 8.5K Sep  2 09:38 libusertreeSkeleton.par
-rw-r--r-- 1 ap4ax ap4ax 2.8K Sep  2 09:27 pen08Selector.C
-rw-r--r-- 1 ap4ax ap4ax  546 Sep  2 09:26 pen08Selector.h
-rw-r--r-- 1 ap4ax ap4ax 3.3K Sep  2 09:17 usertreeSkeleton.C
-rw-r--r-- 1 ap4ax ap4ax  56K Sep  2 09:17 usertreeSkeleton.h


Now copy two files from the tutorials directory:

[ap4ax@node6 myselector]$ /home/ap4ax/root_dev/tutorials/proof/getProof.C .
[ap4ax@node6 myselector]$ /home/ap4ax/root_dev/tutorials/proof/runProof.C .


Modify runProof.C such that it runs the pen08Selector... see my file ;) from the tar.gz archive.



Now run it:

[ap4ax@node1 myselector]$ root -l
root [0] .L runProof.C+
root [0] runProof("pen08")

or since "pen08" is the default argument, we can simply run it as follows:

[ap4ax@node6 myselector]$ root -l runProof.C+

If you want to run this on a batch node, or via 
condor (only distributed over the 4 cores of ONE node, for now) we can:

[ap4ax@node6 myselector]$ run.new "root -l runProof.C+"





You can then modify runProof.C to run any Selector you have, by simply adding additional code
to handle arguments such as:

root [0] runProof("pen08_makeBushForLikelihood")
root [0] runProof("pen08_makeHistos")