Data Processing

• Developing Job Submission Scripts
  Creating scripts to manage job submission both reduces the effort needed to process the data and the chances that mistakes will be made either if processing the wrong data (or not processing the right data) or missing processing errors.

  A scripting language such as Bourne Shell, perl or python are most suitable and are normally used in conjunction with some sort of database

• Use Batch Processing Systems (Farms)
  A typical modern HEP experiment needs far more computing power than a single processor can deliver. So the solution is to set up an array of processors with a batch queuing system that takes user jobs, farms them to optimise the use of the processors, and return the output to the user.

  Locally we have -

  • Access to a farm at: Oxford
  • The GRID overview and documentation