2.4. Running a schedule

2.4.1. How can I calculate how much data will be produced?

The data rate during CABB observations can be calculated with the formula:

Equation 2.3. CABB data rate per cycle

where

• $\mathrm{b}_{\mathrm{cycle}}$ is the number of bytes produced per integration cycle,

• $\mathrm{b}_{\mathrm{corr}}$ is the number of bytes per correlation product, and

• $N_{\mathrm{IFs}}$ is the number of IFs,

• $N_{\mathrm{channels}}(i)$ is the number of channels in IF $i$,

• $N_{\mathrm{Stokes}}(i)$ is the number of Stokes quantities in IF $i$,

• $N_{\mathrm{products}}(i)$ is the number of correlation products in IF $i$,

• $N_{\mathrm{bins}}(i,j,k)$ is the number of bins in IF $i$, Stokes parameter $j$ and correlation product $k$.

The data-rate per second is thus:

where $t_{\mathrm{cycle}}$ the cycle time of the correlator.

A few example calculations are given below. In all cases it is assumed that $\mathrm{b}_{\mathrm{corr}}=8$ bytes (in all modes, 4 bytes are recorded for each of the complex visibilities, and 4 bytes are recorded for the weight component), and $t_{\mathrm{cycle}}=10$ seconds.

2.4.1.1. How much data is produced for a typical continuum observation?

For a typical continuum observation, we have:

• 2 IFs ($N_{\mathrm{IFs}}=2$), each with

• 2049 channels ($N_{\mathrm{channels}}(1-2)=2049$),

• 4 Stokes parameters (XX, YY, XY, YX; $N_{\mathrm{Stokes}}(1-2)=4$),

• 21 correlator products (6 auto-correlation and 15 cross-correlation spectra; $N_{\mathrm{products}}(1-2)=21$).

• Each auto-correlation same-polarisation (XX, YY) spectrum has 2 bins (noise diode on and noise diode off; $N_{\mathrm{bins}}(1-2,1-2,1-6)=2$).

• Each auto-correlation cross-polarisation (XY, YX) spectrum has 1 bin $N_{\mathrm{bins}}(1-2,3-4,1-6)=1$).

• Each cross-correlation spectrum has 1 bin ($N_{\mathrm{bins}}(1-2,1-4,7-21)=1$).

Putting this in Equation 2.3 we get

This works out to be a data rate of 0.300 MiB per second, or roughly 1.055 GiB per hour. Note: we use the binary prefixes MiB and GiB (the mebibyte and gibibyte) for data sizes. See Wikipedia for a better explanation.

Since CABB data files will automatically close when they reach 4.0 GiB in size, it would take approximately 3 h 47 m to fill one file (not accounting for the overhead consumed by the header information).

2.4.1.2. How much data is produced for an observation with all zooms individually recorded?

For an observation where each of the 16 zooms in each IF is recorded, and no zooms are consolidated, we have:

• 34 IFs ($N_{\mathrm{IFs}}=34$). Two of these are the continuum IFs, each having

• 2049 channels ($N_{\mathrm{channels}}(1-2)=2049$),

• 4 Stokes parameters (XX, YY, XY, YX; $N_{\mathrm{Stokes}}(1-2)=4$),

• 21 correlator products (6 auto-correlation and 15 cross-correlation spectra; $N_{\mathrm{products}}(1-2)=21$).

• Each auto-correlation same-polarisation (XX, YY) spectrum has 2 bins (noise diode on and noise diode off; $N_{\mathrm{bins}}(1-2,1-2,1-6)=2$).

• Each auto-correlation cross-polarisation (XY, YX) spectrum has 1 bin $N_{\mathrm{bins}}(1-2,3-4,1-6)=1$).

• Each cross-correlation spectrum has 1 bin ($N_{\mathrm{bins}}(1-2,1-4,7-21)=1$).

• The other 32 IFs are the zoom bands, each having

• 2049 channels ($N_{\mathrm{channels}}(3-34)=2049$),

• 4 Stokes parameters (XX, YY, XY, YX; $N_{\mathrm{Stokes}}(3-34)=4$),

• 21 correlator products (6 auto-correlation and 15 cross-correlation spectra; $N_{\mathrm{products}}(3-34)=21$).

• Each spectrum has 1 bin ($N_{\mathrm{bins}}(3-34,1-4,1-21)=1$).

Putting this in Equation 2.3 we get

This works out to be a data rate of 4.352 MiB per second, or roughly 15.3 GiB per hour. Since CABB data files will automatically close when they reach 4.0 GiB in size, it would take approximately 15 m to fill one file (not accounting for the overhead consumed by the header information).

2.4.1.3. How much data is produced for a pulsar-binning observation?

For an observation using CABBs pulsar-binning mode, we have:

• 4 IFs ($N_{\mathrm{IFs}}=4$). Two of these are the normal continuum IFs, each having

• 2049 channels ($N_{\mathrm{channels}}(1-2)=2049$),

• 4 Stokes parameters (XX, YY, XY, YX; $N_{\mathrm{Stokes}}(1-2)=4$),

• 21 correlator products (6 auto-correlation and 15 cross-correlation spectra; $N_{\mathrm{products}}(1-2)=21$).

• Each auto-correlation same-polarisation (XX, YY) spectrum has 2 bins (noise diode on and noise diode off; $N_{\mathrm{bins}}(1-2,1-2,1-6)=2$).

• Each auto-correlation cross-polarisation (XY, YX) spectrum has 1 bin $N_{\mathrm{bins}}(1-2,3-4,1-6)=1$).

• Each cross-correlation spectrum has 1 bin ($N_{\mathrm{bins}}(1-2,1-4,7-21)=1$).

• The other two IFs are the pulsar-binned continuum bands, each having

• 513 channels (each of them being the average of 4 of the normal continuum channels) ($N_{\mathrm{channels}}(3-4)=513$),

• 4 Stokes parameters (XX, YY, XY, YX; $N_{\mathrm{Stokes}}(3-4)=4$),

• 15 correlator products (15 cross-correlation spectra; $N_{\mathrm{products}}(3-4)=15$).

• Each cross-correlation spectrum has 32 bins across the pulse period ($N_{\mathrm{bins}}(3-4,1-4,1-15)=32$).

Putting this in Equation 2.3 we get

This works out to be a data rate of 1.803 MiB per second, or roughly 6.339 GiB per hour. Since CABB data files will automatically close when they reach 4.0 GiB in size, it would take approximately 37 m to fill one file (not accounting for the overhead consumed by the header information).

2.4.2. How can I quickly determine how much data will be produced?

Since all modes give you the continuum IFs, you start off with one of two base data rates, per cycle. For the 1 MHz modes, the base data rate is:

bytes per IF. For the 64 MHz modes, the base data rate is:

bytes per IF.

For each pulsar-binned IF you have (only available in 1 MHz mode) you add:

For each single zoom you have (i.e. zooms with a width of 1) you add:

For each consolidated zoom you have (i.e. zooms with a width of 2 or more) you begin by adding:

Then, for each zoom band in the consolidated zoom, you add:

For example, if you had a consolidated zoom with a width of 5, it would produce

bytes per cycle.

Example: If you were using the 1 MHz/64 MHz hybrid CABB configuration, and had one zoom of width 1 and one consolidated zoom of width 12 in the 64 MHz IF your total data usage would be:

Doing the sum gives $\mathrm{b}_{\mathrm{cycle}} = 1.192224\times10^{7}$ bytes per cycle. Assuming a 10 second cycle time, this would generate 3.997 GiB per hour, which would give you roughly one new CABB data file every hour.

2.4.3. Do I have to do the observations myself?

Users of the ATCA are expected to organise the execution of the observations. Any member of the team that proposed the experiment is eligible to observe from the Marsfield SOC, or any other researcher that the primary investigator might like to nominate. If the researcher has not previously observed with the ATCA and CABB, or has not observed within the past 12 months, they will have to travel to the SOC to be trained.

2.4.3.1. Do I need to be trained to use the ATCA?

Yes. It is recommended that observers unfamiliar with the ATCA arrive at the SOC at least two full weekdays before their observations, in order for them to become familiar with how to use the array and who to talk to if things go wrong.

2.4.3.2. What if we can't make it to the SOC?

If it is not possible for any of your team members to come to the SOC for your allocated observing time, but you have visited SOC sometime in the 12 months prior to your observations, you may be granted permission to do your observations remotely (Section 2.4.4).

2.4.3.3. Can't someone do the observations for me?

We expect that observers are members of the team that proposed the experiment. Said another way, if you require someone to do your observations, we fully expect that that person would become a collaborator, and would be included on any publications derived from those observations. If this is what you require, you will need to alert ATNF staff well ahead of your scheduled time (4 weeks at least).

2.4.3.4. I was planning to come, but I've just had an emergency. What can I do?

We understand that plans can change on short timescales and without warning. We recommend first trying to recruit another member of your team to do the observations, but should that not be possible, please alert ATNF staff.

It may be possible to do a short-notice scheduling swap with another project, leaving you to do your observations at a later date. If this is not possible, we may be able to find a replacement observer, with the understanding that whomever does the observing will become a collaborator on the experiment, and will receive due credit on any publications derived from the observations.

2.4.4. Can I do my observations from my home institution?

Remote observing is available for experienced ATCA observers. We will consider granting a request for remote observations if the observer:

• Has observed from the SOC at least once in the 12 months prior to the date of the proposed remote observations, or

• Usually observes more than 3 times per semester, or on at least 10 days per semester, and has observed from the SOC at least once in the 18 months prior to the date of the proposed remote observations.

Applications to do a remote observation must be made at least 2 weeks before the observations. This is so observatory staff can check for possible network disruptions during your observations, and advise you if remote observing will not be possible. The later you leave your remote observing application, the less time you will have to change your plans should your request for remote observations be refused.

Permission to do your observations remotely is given at the discretion of science operations staff, and you should not presume that permission will be forthcoming. Should permission be refused and you are not able to come to the observatory, you may need to find a local collaborator to do the observations for you. Your observing “friend” will not generally be available to do your observations. The duty astronomer is also under no obligation to observe for your project if you are unable to.

To apply to make an observation remotely, use the “Booking” tab on the ATCA observing PORTAL, which will look something like Figure 2.2. First, fill out the “Booking details” form (which is revealed by clicking on that button), and hit the “Book” button in the “Status” column.

Remote observers must have their own ATNF UNIX account and remember their password for this account. The ATCA website has pages describing current remote observing conditions and instructions for observing remotely.

To be sure of a successful remote observing session, all observers are required to test their remote observing setup at least a week before their observations.

At a minimum, you will require the following for remote observing:

• A computer with SSH software, capable of running a VNC viewer, on a high bandwidth connection,

• A telephone near to the computer, that can make calls to, and accept calls from the observatory.

Remote observations will be easier if you also have:

• Two or more monitors attached to their computer,

• A computer with X-Windows software,

• A hands-free speakerphone or headset.

To test your setup, you should contact the duty astronomer and ask to be allowed to login and perform the test. If the test will not disturb current observations, you should follow the instructions on remote observing found in Chapter 3 to ensure that you can monitor and control all the required VNC sessions. If problems are encountered here, and the duty astronomer can't help you resolve them, contact the observatory staff with a description of the problems and ask for advice.