How to conduct an RFI survey with the ATCA
This document describes how to complete an RFI survey at the Australia Telescope Compact Array. It covers the basic principles, creating SCHED files, taking and reducing the data and the presentation of the results. It is fairly detailed, to allow non-system/astronomy type operators to follow the procedure.
All files pertaining to RFI surveys since 2004 are stored in /DATA/KAPUTAR_1/operations/rfi/survey.
General principles
RFI surveys attempt to measure non-astronomical emmision in the observing bands of the telescope. As the observing bands are very wide (several GHz) and the instantaneous bandwidth of the array is comparitively small, 8-128 MHz, it is necessary to use an unusual correlator mode.The ATCA correlator comprises of 15 blocks, containing 8 modules (we'll ignore the auto-correlation blocks at this stage). The 15 blocks are for the 15 baselines (6 antennas: 1-2, 1-3, ... 5-6), the 8 modules in each arefor the different products (XX, XY, YX, YY - for two frequencies). The resources are limited, but scale linearly for channels and bandwidth. Thus doubling the bandwidth means halving the channels (an effective increase in spectral resolution by a factor of four).
By discarding some of the products, it is possible to double bandwidth/channels but without sacrificing the other. In the modes we use for the RFI survey, we discard all but one polarisation, and then discard all but one baseline. This single baseline is processed by 4 blocks (= 32 modules) and results in 64 MHz of bandwidth and 1024 channels.
It is not possible to concatenate more than 4 blocks as it leads to timing delay problems. It can be done with rewiring the patch cables from the correlator distributor, but it kills the "existing" arrangement, and is best left alone.
In the past, RFI surveys were always conducted with baseline 4-5. More recently, correlator configurations have been created for two more potentially short baselines involving two neighbouring antennas, namely 3-4 and 1-2. The correlation configurations available for baseline 4-5 are:
BL45_64_1024
BL45_32_2048
For RFI surveys, we use the former, but for detailed surveying of a
particular line, the other might be useful. Consequently, the new
correlator configurations for RFI surveys follow the same naming
convention:
BL12_64_1024
BL34_64_1024
The actual surveys are done a "band" at a time. The bands at the ATCA are:
L "20cm" 1170 - 1807 MHz
S "13cm" 2223 - 2632 MHz
C "6cm" 3900 - 6860 MHz
X "3cm" 8000 - 10117 MHz
The wavelength designations are very general; C-band, for instance,
almost covers an octave. Also, in future, provision may need to be
made for K-, Q- and W-bands.
The survey is done by observing at 32 MHz intervals all the way across the band, one after the other. This is handled automatically by the ATCA's scheduling and observing programs.
Creating the schedule files
The schedule files are long and tedious to create. To simplify the process in the past a DCL command procedure was used to create them. However, since there is an existing set of files already, it is now easiest to use the sched program (or an editor) to change the entry for the correlator configuration, which is usually the only parameter that will change. The existing sched files are stored in subdirectory rfi/survey/sched and are called:
l_rfi.sch
s_rfi.sch
c_rfi.sch
x_rfi.sch
Taking the data
For taking the data it is easiest to first stow the array, then detach all antennas and re-attach only those that are needed. To set those up for the RFI survey observations, two setup files have been created, one for the L and S band, the other for the C and X band.The SCHED files used should be loaded into CAOBS and the relevant antennas attached and disabled. Also ensure an appropriate reference antenna is selected. The following example is for baseline 4-5, for setting up L and S band survey observations (replace "ls" in the setup filename by "cx" for C/X band setup):
CAOBS> stow CAOBS> detach all CAOBS> set file rfi_survey_ls_setup CAOBS> set ref ca04 CAOBS> attach ca04 CAOBS> attach ca05 CAOBS> enable ca04 CAOBS> track 1 CAOBS> stop CAOBS> disable ca04 CAOBS> enable ca05 CAOBS> track 2 CAOBS> stop CAOBS> disable ca05The setup file has brought the appropriate receiver on-axis. The two scans track positions az,el=90,89.5 (scan 1) and az,el=270,89.5 on the sky (scan 2). Pointing the two antennas at two positions offset from each other by 1 degree decorrelates emission from the sky. At 89.5 degrees the elevation for both antennas is identical, making the conditions under which they obtain their data (from terrestrial sources close to the horizon) as similar as possible.
CAOBS> set file l_rfi CAOBS> track 1Load the sched file for the L band RFI survey itself. Give the attenuation time to settle down before starting the survey:
CAOBS> stop CAOBS> corr closef CAOBS> start 1Opening a new datafile after all the preparations are complete makes the data reduction later easiest (see below).
The data can be checked on VIS using the frequency display. E.g.:
VIS> sel 45aa VIS> af-t VIS> a-f
Duration of a complete RFI survey
An RFI survey for all four centimetric wavebands, including an overhead of about 45 minutes for executing the setup files, takes about 5:30 hours in total. The approximate net times for each waveband are:L band 19 scans 0:28 h S band 13 scans 0:19 h C band 93 scans 2:19 h X band 67 scans 1:40 h -------------------------- TOTAL 192 scans 4:46 h
Reducing the data
Either perform the data reduction on a personal machine first (Miriad and gnuplot will be required) or, as a person with access to the operations area, run the programs directly on kaputar. The entire reduction of RFI survey data is scripted. The required scripts can be found in the subdirectory rfi/survey/scripts. The main script to run isrfi_survey.cshIn this script specify where the input data files are located for the RFI surveys in each subband. The script will run the following tasks for each of the four bands separately:
Task: atlod in = 97-04-22_2224.rfi out = multi.uv ifsel = 1 restfreq = options = relax,unflag,nopflag nfiles = nscans =
On starting, because the antennas were disabled, all the data in the RPFITS file will be flagged. Therefore, it must be unflagged before proceeding. The simplest way to do this is with UVFLAG as given in this example:
Task: uvflag vis = multi.uv select = source(rfi) line = edge = flagval = unflag options = log =
Now split the multi.uv dataset into frequency specific datasets. This is done using the uvsplit task.
Task: uvsplit vis = multi.uv select = options =You should now have a directory full of files called src.freq (e.g. rfi.2447). A problems ensues due to this file naming convention, because a subsequent task will want these files sorted correctly by name, but will then file when reaching values of 10 GHz or above (rfi.10xxx). For that reason, the next step in the script is to rename all file with four-digit frequency extensions to five-digits, with a leading zero. This is done by a separate script called
rfi_rename.cshthat will be executed by rfi_survey.csh.
Next the output files need to have their visibility data extracted into ASCII text files. Once this is done, the data in these text files can be concatenated and sorted.
The extraction process is done using Miriad's UVSPEC task.
Task: uvspec vis = rfi.2367 select = ant(4)(5) line = channel,512,256,1,1 stokes = xx interval = 20 hann = offset = options = axis = freq,ampl yrange = device = /xs nxy = 1,1 log = temp.log
To make things easier, a script called
rfi_reduce.cshhas been written. It will vary, depending on the exact bandwidth and correlator configuration used, but outlines the general principles of the reduction and provides a model for the user to make their own reduction script. In it one needs to change the entries for the antenna selection parameter if a baseline other than 4-5 was used. The script is listed here:
#!/bin/csh
echo " "
echo "Miriad script for the reduction of spectral line"
echo "data taken for radio frequency interference (RFI)"
echo "surveys. D.J.McKay, 1995"
echo " "
echo "Setting source to RFI"
set src="rfi"
echo "src = ${src}"
echo "Removing old temporary files"
'rm' results.dat
'rm' -r file_list.temp
echo "Creating file list"
'ls' -C1 -d ${src}.* > file_list.temp
'ls' -l file_list.temp
echo "About to enter the main loop"
foreach file (`cat file_list.temp`)
echo "Processing file ${file}"
'rm' -r temp.file
uvspec vis=${file} \
interval=20 \
axis=frequency,amplitude \
device=/xs \
nxy=1,1 \
log=temp.log \
"select=ant(4)(5)" \
stokes=xx \
line=channel,512,256,1,1
echo "Sorting data"
sort temp.log > temp.sort
echo "Concatenating results"
'mv' results.dat temp.dat
cat temp.dat temp.sort > results.dat
echo "Cleanup for file ${file}"
'rm' temp.dat
'rm' temp.log
'rm' temp.sort
echo "Processing for file ${file} complete"
echo " "
end
echo " "
echo "Complete"
The output of this script is a file called results.dat,
which will contain, in two columns, the frequency and flux for the
different channels of all the data taken. This can then be plotted for
presentation.
To avoid confusion between different epochs, it is recommended to rename the output file results.dat. The current convention is to use names such as
survYYMM.datwhere YYMM is a four-digit representation of year and month (e.g. 0702 for February 2007). This is done as the last step in script
rfi_survey.csh;the final output filename is a user input in that script.
Plotting the data
Of course, while UVSPEC is running, you will see plots being displayed to screen (unless you set the PGPLOT device to /null or a file). However, this will only be one bandpass at a time, and the aim is to get a single wide-band plot.This can be done with the gnuplot package. gnuplot is a standard package for plotting scientific data that is delivered as part of linux. A script has been prepared to make this easy, called
gnuplot.csh.Again, this script is run as part of
rfi_survey.csh.gnuplot.csh calls an input parameter file called
gnuplot_rfisurvey.txt.In this file one needs to make a general replacement of the name string of the output files for the current epoch RFI survey, e.g.
surv0609 ==> surv0702The final output postscript plot files for each individual waveband are called
survYYMM_[lscx].ps,while the file mosaicking all bands into one array is called
survYYMM.ps.
Original: Derek McKay (3-Apr-1995)
Modified: Michael Dahlem (2-Mar-2007)