Australia Telescope Compact Array

How to conduct an RFI survey with the ATCA

WARNING: THIS PAGE NEEDS TO BE UPDATED FOR THE NEW CABB CORRELATOR.

This document describes how to complete an RFI survey at the Australia Telescope Compact Array. It covers the basic principles, creating SCHED files, taking and reducing the data and the presentation of the results. It is fairly detailed, to allow non-system/astronomy type operators to follow the procedure.

All files pertaining to RFI surveys since 2004 are stored in /DATA/KAPUTAR_1/operations/rfi/survey.

General principles

RFI surveys attempt to measure non-astronomical emmision in the observing bands of the telescope. As the observing bands are very wide (several GHz) and the instantaneous bandwidth of the array is comparitively small, 8-128 MHz, it is necessary to use an unusual correlator mode.

The ATCA correlator comprises of 15 blocks, containing 8 modules (we'll ignore the auto-correlation blocks at this stage). The 15 blocks are for the 15 baselines (6 antennas: 1-2, 1-3, ... 5-6), the 8 modules in each arefor the different products (XX, XY, YX, YY - for two frequencies). The resources are limited, but scale linearly for channels and bandwidth. Thus doubling the bandwidth means halving the channels (an effective increase in spectral resolution by a factor of four).

By discarding some of the products, it is possible to double bandwidth/channels but without sacrificing the other. In the modes we use for the RFI survey, we discard all but one polarisation, and then discard all but one baseline. This single baseline is processed by 4 blocks (= 32 modules) and results in 64 MHz of bandwidth and 1024 channels.

It is not possible to concatenate more than 4 blocks as it leads to timing delay problems. It can be done with rewiring the patch cables from the correlator distributor, but it kills the "existing" arrangement, and is best left alone.

In the past, RFI surveys were always conducted with baseline 4-5. More recently, correlator configurations have been created for two more potentially short baselines involving two neighbouring antennas, namely 3-4 and 1-2. The correlation configurations available for baseline 4-5 are:

      BL45_64_1024
      BL45_32_2048

For RFI surveys, we use the former, but for detailed surveying of a particular line, the other might be useful. Consequently, the new correlator configurations for RFI surveys follow the same naming convention:

      BL12_64_1024
      BL34_64_1024

The actual surveys are done a "band" at a time. The bands at the ATCA are:

    L    "20cm"     1170  -  1807 MHz
    S    "13cm"     2223  -  2632 MHz
    C     "6cm"     3900  -  6860 MHz
    X     "3cm"     8000  - 10117 MHz

The wavelength designations are very general; C-band, for instance, almost covers an octave. Also, in future, provision may need to be made for K-, Q- and W-bands.

The survey is done by observing at 32 MHz intervals all the way across the band, one after the other. This is handled automatically by the ATCA's scheduling and observing programs.

Creating the schedule files

The schedule files are long and tedious to create. To simplify the process in the past a DCL command procedure was used to create them. However, since there is an existing set of files already, it is now easiest to use the sched program (or an editor) to change the entry for the correlator configuration, which is usually the only parameter that will change. The existing sched files are stored in subdirectory rfi/survey/sched and are called:

    l_rfi.sch
    s_rfi.sch
    c_rfi.sch
    x_rfi.sch

Taking the data

For taking the data it is easiest to first stow the array, then detach all antennas and re-attach only those that are needed. To set those up for the RFI survey observations, two setup files have been created, one for the L and S band, the other for the C and X band.

The SCHED files used should be loaded into CAOBS and the relevant antennas attached and disabled. Also ensure an appropriate reference antenna is selected. The following example is for baseline 4-5, for setting up L and S band survey observations (replace "ls" in the setup filename by "cx" for C/X band setup):

CAOBS> stow
CAOBS> detach all
CAOBS> set file rfi_survey_ls_setup
CAOBS> set ref ca04
CAOBS> attach ca04
CAOBS> attach ca05
CAOBS> enable ca04
CAOBS> track 1
CAOBS> stop
CAOBS> disable ca04
CAOBS> enable ca05
CAOBS> track 2
CAOBS> stop
CAOBS> disable ca05

The setup file has brought the appropriate receiver on-axis. The two scans track positions az,el=90,89.5 (scan 1) and az,el=270,89.5 on the sky (scan 2). Pointing the two antennas at two positions offset from each other by 1 degree decorrelates emission from the sky. At 89.5 degrees the elevation for both antennas is identical, making the conditions under which they obtain their data (from terrestrial sources close to the horizon) as similar as possible.

CAOBS> set file l_rfi
CAOBS> track 1

Load the sched file for the L band RFI survey itself. Give the attenuation time to settle down before starting the survey:

CAOBS> stop
CAOBS> corr closef
CAOBS> start 1

Opening a new datafile after all the preparations are complete makes the data reduction later easiest (see below).

The data can be checked on VIS using the frequency display. E.g.:

VIS>  sel 45aa
VIS>  af-t
VIS>  a-f

Duration of a complete RFI survey

An RFI survey for all four centimetric wavebands, including an overhead of about 45 minutes for executing the setup files, takes about 5:30 hours in total. The approximate net times for each waveband are:

  L band   19 scans   0:28 h
  S band   13 scans   0:19 h
  C band   93 scans   2:19 h
  X band   67 scans   1:40 h
  --------------------------
  TOTAL   192 scans   4:46 h

Reducing the data

Either perform the data reduction on a personal machine first (Miriad and gnuplot will be required) or, as a person with access to the operations area, run the programs directly on kaputar. The entire reduction of RFI survey data is scripted. The required scripts can be found in the subdirectory rfi/survey/scripts. The main script to run is

rfi_survey.csh

In this script specify where the input data files are located for the RFI surveys in each subband. The script will run the following tasks for each of the four bands separately:

  Task:   atlod
  in       = 97-04-22_2224.rfi
  out      = multi.uv
  ifsel    = 1
  restfreq =  
  options  = relax,unflag,nopflag
  nfiles   =  
  nscans   =

On starting, because the antennas were disabled, all the data in the RPFITS file will be flagged. Therefore, it must be unflagged before proceeding. The simplest way to do this is with UVFLAG as given in this example:

  Task:   uvflag
  vis      = multi.uv
  select   = source(rfi)
  line     =  
  edge     =  
  flagval  = unflag
  options  =  
  log      =

Now split the multi.uv dataset into frequency specific datasets. This is done using the uvsplit task.

  Task:   uvsplit
  vis      = multi.uv
  select   =  
  options  =

You should now have a directory full of files called src.freq (e.g. rfi.2447). A problems ensues due to this file naming convention, because a subsequent task will want these files sorted correctly by name, but will then file when reaching values of 10 GHz or above (rfi.10xxx). For that reason, the next step in the script is to rename all file with four-digit frequency extensions to five-digits, with a leading zero. This is done by a separate script called

rfi_rename.csh

that will be executed by rfi_survey.csh.

Next the output files need to have their visibility data extracted into ASCII text files. Once this is done, the data in these text files can be concatenated and sorted.

The extraction process is done using Miriad's UVSPEC task.

  Task:   uvspec
  vis      = rfi.2367
  select   = ant(4)(5)
  line     = channel,512,256,1,1
  stokes   = xx
  interval = 20
  hann     =  
  offset   =  
  options  =  
  axis     = freq,ampl
  yrange   =  
  device   = /xs
  nxy      = 1,1
  log      = temp.log

To make things easier, a script called

rfi_reduce.csh

has been written. It will vary, depending on the exact bandwidth and correlator configuration used, but outlines the general principles of the reduction and provides a model for the user to make their own reduction script. In it one needs to change the entries for the antenna selection parameter if a baseline other than 4-5 was used. The script is listed here:

#!/bin/csh

echo " "
echo "Miriad script for the reduction of spectral line"
echo "data taken for radio frequency interference (RFI)"
echo "surveys. D.J.McKay, 1995"
echo " "

echo "Setting source to RFI"
set src="rfi"
echo "src = ${src}"

echo "Removing old temporary files"
'rm' results.dat
'rm' -r file_list.temp

echo "Creating file list"
'ls' -C1 -d ${src}.*  > file_list.temp
'ls' -l file_list.temp

echo "About to enter the main loop"
foreach file (`cat file_list.temp`)
  echo "Processing file ${file}"
  'rm' -r temp.file

  uvspec vis=${file} \
         interval=20 \
         axis=frequency,amplitude \
         device=/xs \
         nxy=1,1 \
         log=temp.log \
         "select=ant(4)(5)" \
         stokes=xx \
         line=channel,512,256,1,1

  echo "Sorting data"
  sort temp.log > temp.sort
  echo "Concatenating results"
  'mv' results.dat temp.dat
  cat temp.dat temp.sort > results.dat
  echo "Cleanup for file ${file}"
  'rm' temp.dat
  'rm' temp.log
  'rm' temp.sort
  echo "Processing for file ${file} complete"
  echo " "

end

echo " "
echo "Complete"

The output of this script is a file called results.dat, which will contain, in two columns, the frequency and flux for the different channels of all the data taken. This can then be plotted for presentation.

To avoid confusion between different epochs, it is recommended to rename the output file results.dat. The current convention is to use names such as

survYYMM.dat

where YYMM is a four-digit representation of year and month (e.g. 0702 for February 2007). This is done as the last step in script

rfi_survey.csh;

the final output filename is a user input in that script.

Plotting the data

Of course, while UVSPEC is running, you will see plots being displayed to screen (unless you set the PGPLOT device to /null or a file). However, this will only be one bandpass at a time, and the aim is to get a single wide-band plot.

This can be done with the gnuplot package. gnuplot is a standard package for plotting scientific data that is delivered as part of linux. A script has been prepared to make this easy, called

gnuplot.csh.

Again, this script is run as part of

rfi_survey.csh.

gnuplot.csh calls an input parameter file called

gnuplot_rfisurvey.txt.

In this file one needs to make a general replacement of the name string of the output files for the current epoch RFI survey, e.g.

surv0609 ==> surv0702

The final output postscript plot files for each individual waveband are called

survYYMM_[lscx].ps,

while the file mosaicking all bands into one array is called

survYYMM.ps.

Original: Derek McKay (3-Apr-1995)
Modified: Michael Dahlem (2-Mar-2007)