One aim of the project is the development of an automatic reduction pipeline in order to reduce data of Wide Field CCD facilities, e.g. WFI@ESO/MPG 2.2 m telescope,
OmegaCam@VST. Data of these multi CCD cameras need new ways in observing and reducing. In contrast to single chip CCDs one has to think about how to close gaps
between the single CCDs efficiently or how to to bring data from different CCDs with different charakteristics on the same photometric level for calibration. A second
problem of wide field observations is the huge amount of data produced by the Mosaic cameras (e.g. ~30GB per night for the WFI) which will increase to more than
100 GB for OmegaCam. For efficient reduction and calibration of the data it is necessary to have a well working pipeline on a high performace computer system. For our
project we use a cluster consisting of 32
nodes. Every node is build of an Athlon XP 2.8 GHz processor with 1 GB working space and an 120 GB hard disk. Additionally a master computer is used consisting of a
dual processor board with 2 x 2.8 GHz Atholon XP processors, 2 GB ram space as well as a raid5 hard disk array of 4 TB storage capacity. The single disks of the raid
array are connected via fibre channel. For backup of the final data products we use a DVD+/-RW burner, a SCSI-DLT tape drive (80GB), as well as a SCSI-DAT-IV tape
drive. As IO-server we use a P4 2.8 GHz computer with 512 MB ram working space and an 120 GB hard disk. The data transfer will be done via an USB2.0 connection.
The nodes are connected with the master surver via a Gigabit Ethernet.
With this high end cluster in combination with our developed automatic reduction pipeline it is now possible to reduce
efficently large amounts of wide field data. For this purpuse the the philosophy of the pipeline is to split the multi CCD images into single files for every single CCD. The
work on single CCDs is very fast and efficient. Therefore the number of nodes in the cluster is choosen to be n times the number of CCDs. That makes it possible to use 1
node to work on data of one CCD. With our cluster we are able to work on 4 WFI or 2 OmegaCam data sets at the same time. this makes the reduction of large surveys
very efficient.
Additionally we also implemented our pipeline on several stand alone dual processor machines
(e.g. 2 x 2.4 GHz, 4 GB ram space, 2 TB raid array). Also in this
configuration the pipeline is able to reduce relatively efficiently surveys up to 1 - 2 deg2 field.
