In the 2017APR semester, a rapid response mode was introduced. This is designed to allow the ATCA to respond almost instantly to a request for a NAPA over-ride. This section describes how to use this mode.
The ATCA rapid response mode is only available to proposers who have submitted a
NAPA proposal for the current semester. If you are interested in being granted
permission to use the rapid response mode for your NAPA proposal, please talk
to Jamie Stevens (<Jamie.Stevens@csiro.au>).
The rapid response mode is designed to take a user-made schedule, and get it onto the telescope as soon as possible after the request has been made. Getting the telescope to stop one schedule and start another is very easy; determining when "as soon as possible" actually starts is the tricky part. The rest of this section describes the process which takes place to perform an over-ride observation.
You submit a NAPA science case to the ATNF TAC, which gives you a ranking grade for the semester.
You ask for, and are supplied with an authorisation token which will allow you to request an over-ride through the rapid response service.
At some point during the semester, you decide to trigger your NAPA proposal. You (or more likely your software) creates a CABB schedule that observes both the target of interest and a nearby gain calibrator.
You submit this schedule to the ATCA rapid response web service.
The service checks that you are an authorised user, and rejects the request if not.
It then checks that the minimum amount of time that you have requested is still available to be scheduled for your project, considering the amount of time you requested from the TAC, and the amount of time you have used before this request. If you do not have sufficient time, the request is rejected.
A start time is determined where the source is above the ATCA lower elevation limit, and that the schedule can be executed for at least the minimum required amount of time while still being above the horizon. The search for this suitable start time is limited to either 100 hours after the request submission, or to a user-specified duration, if it is less than 100 hours. The 100 hour limit is designed to allow for an automated response even for long weekends where the proposers may not be available for a manual over-ride, while the user-specified duration can indicate that an observation that can't be made within some time frame would no longer be scientifically interesting.
If a suitable start time cannot be found, the request is rejected.
An estimate is made of when the flux-density and bandpass calibrators should be observed, given the start time. The schedule is altered to insert those scans.
If the request has not been rejected by this point, the request is approved.
At the start time determined in the previous step, several additional checks are made.
All scheduled scans will be above the ATCA lower elevation limit at their expected start time, for at least the minimum required time.
The array is not stowed due to weather conditions.
CABB is configured in either 1 MHz continuum mode, 1 MHz zooms mode, or 64 MHz zooms mode.
If any of these conditions is not met, then the start time of the project is delayed from between 5 seconds and 10 minutes, at which point each of the checks are made again.
When all checks from the previous step are satisfied, a final set of schedule alterations are made.
The schedule is shortened if necessary, to fit in the time available at this start time. The amount of time scheduled will always be at least the minimum required.
If the user has requested to use the same frequency configuration as the project being over-ridden, and
the band currently being observed (or that was last observed) is not 7mm or 3mm,
the array is not currently split between two bands,
and the project that is currently being observed (or that was last observed) is not C999,
then the over-ride schedule is altered to use the same frequency configuration as is currently listed. Otherwise, the frequency configuration is left as specified in the request.
If the slew from the current position to the first scan of the schedule is less than 20 seconds, and the delays need to be reset because the user-specified frequencies are being used instead of the currently configured frequencies, an additional “Normal” scan of 40 seconds is inserted at the beginning of the schedule. This scan is named “setup” so it can easily be identified and discarded during later data reduction.
The first scan in the schedule is altered, in the following ways.
The command
focus defaultis added, to ensure the focus is set correctly.If the scheduled frequencies are not the same as those that were observed before the over-ride, the command
corr reset delaysis added, and the tv-channels are set to be 1020 to 1022 in each IF.
If CABB is currently configured in 64 MHz zooms mode, several scans are added at the beginning of the schedule to do an automatic delay calibration for the user-specified frequencies. A description of how this is done can be found at this link.
The altered schedule is copied into the right place, caobs detects this new schedule file and immediately begins executing it. Over-ride schedules are started at scan 1, and continue through until the last scan, and will not loop.
While the schedule is being executed, an automatic monitoring system will watch that the schedule is progressing, and will attempt to automatically restart the schedule in the event that it gets stopped.
All scheduled slots that have a project code starting with “C” can be over-ridden by the rapid response mode, provided that your TAC score is sufficiently high and CABB is in the correct mode.
Blocks of time scheduled for reconfigures, maintenance and VLBI cannot be over-ridden.
Any over-ride project that has a score higher than or equal to the lowest score given to a scheduled project may over-ride any other scheduled project. The theory for this is that if we knew at the beginning of the semester when a NAPA would trigger, and that NAPA had a score about the lowest score that was actually scheduled, we would have attempted to schedule it at the appropriate time and scheduled the displaced project at another time, regardless of the score of the displaced project.
As part of the project to develop this rapid response mode, a library for creating and manipulating CABB schedules was developed. It can be found on GitHub, and is free for use.
Because many of the existing libraries that astronomers use to deal with transient event streams are written in Python, the choice was made to focus first on providing a Python library for schedule manipulation. This library is targetted toward Python 2.7, but will work with 3.8.
Documentation on how to use the library in your own code is part of the GitHub repository.
To ensure that only those projects that have been pre-approved can perform over-rides on
the ATCA, any request for over-ride time needs to be accompanied by
a valid authorisation token. This can only be obtained by emailing
<Jamie.Stevens@csiro.au>.
There are two types of authorisation token available. The first is a test token, which allows you to submit a request, and go through the entire process of the over-ride, but not actually get time on the telescope. This is designed to be used while you are trying to develop and perfect your automated trigger routines. You may issue as many requests as you like with a test token.
The second token will allow your request to allocate time on the telescope. This type of token will not be issued until you have demonstrated that your code can properly submit a test request. You will need to work with Jamie Stevens during this process.
Authorisation tokens are valid only for a specific project and semester, and will be rejected if used outside those bounds.
The authorisation token itself is a JSON Web Token. You will be supplied with a normal ASCII file that will need to be passed back to the service whenever you make a request.
All requests to trigger an over-ride observation are submitted through a web service. The exact format of the request won't be documented here, and we suggest that you use the Python code we provide when you are building your trigger response software.
The library which makes it easy to submit over-ride requests can be found on GitHub, along with documentation and examples.
If your request is unable to be scheduled given the bounds you provided, you will receive your rejection in the response from the web service.
If the web service doesn't reject your request, your schedule will enter the queue to be executed. At this point, two emails will be sent to this effect. As with all emails sent by the rapid response service, they will be from Jamie.Stevens@csiro.au and will go to all the members of the NAPA team (as specified on its OPAL coversheet), along with all the members of each project that will need to be displaced.
The first email will have the subject “ATCA rapid response mode triggered for Cxxxx ”, where of course the real NAPA project code will be specified instead of “ Cxxxx”. The second email will have the subject “Your ATCA rapid response over-ride request has been queued for execution.”.
The first email will look something like this.
The ATCA rapid response mode was requested by the NAPA project C008 at UTC 2017-03-31 04:14:14.
This over-ride has been scheduled to begin at UTC 2017-03-31 04:14:14 and end at approximately UTC 2017-03-31 04:27:06.
This over-ride has been assigned the ID xaap6yuacy10ltg9lc3g.
This will displace time from the project C007.
The observations will be automatically started without any user intervention. While the observations are running, please do not intervene unless advised to by observatory staff.
You will receive another email when the observations have started, and another when the over-ride observations have ended. Messages to the same effect will also appear in the ATCA Portal chat.
Please send any feedback about this system by replying to this email.
As you can see, the time of the request, and the expected start and end times are all listed, along with the unique ID assigned to the over-ride, and a list of all the projects that will be displaced.
The next email will look similar to the following.
This email is from the ATCA rapid response system.
What follows is a log of the progress of your requested over-ride experiment.
Project code: C008
Over-ride ID: xaap6yuacy10ltg9lc3g
2017-03-31_04:14:14: Your experiment has been placed in the queue, and should start at 2017/3/31 04:14:14. You will receive another email at that time.
If you have any concerns about this experiment, please reply to this email.
This second email introduces the time-stamped log of the progress of the over-ride. All subsequent email communication will contain this log, as it was when the email was sent. The log will have entries for all the important actions made by the rapid response service, and entries that will be useful for later data reduction.
At the start time, the service will either start or delay the schedule, and in either case an email will be sent. If the schedule is started, the email will have the subject “An ATCA rapid response over-ride is now being observed”, while if the schedule is delayed the subject will be “Your ATCA rapid response over-ride request has been delayed”; the reasons for the delay will be included in the log. If the schedule is started, the name of the schedule will be given in the email, and you may load it in the CABB web scheduler to see how the service has altered it.
No emails will be sent while the schedule is running, except if the service encounters a problem. In that case, you will get an email with the subject “There has been a problem while executing your ATCA rapid response over-ride”.
When the schedule finishes, the service will send out one final email, with the subject “ATCA rapid response over-ride is now complete”. This email contains the full log, which will include the name of the RPFITS files that contain the data.
If for some reason the service is unable to perform the scheduled over-ride (for example, if the antennas are weather stowed for an extended period of time), then the schedule might expire and be removed from the queue. If this happens you will get an email with the subject “Your ATCA rapid response over-ride request has expired”.
The schedule you make depends on what type of observation you are trying to perform. What follows are some general recommendations for rapid response schedules.
Select the frequencies that will give you the best science, rather than ask for the frequency configuration that is being used when the telescope is over-ridden. There is no actual need to have a perfectly calibrated array when doing an observation with the ATCA. As long as the delays can be measured while observing a strong calibrator, you will be able to correct for those delays in Miriad. If you specify the frequencies you want, the service will ensure that the delays are set back to their defaults at the start of your observation, which will always produce a dataset that can be successfully calibrated.
To ensure that the observations can be monitored effectively with default delays, the tvchannel range is limited to 1020 - 1022 in each IF. This 3 MHz of bandwidth near the band centre is sufficient to measure the system temperature and to show the calibrator's flux density in vis. You will not need to specify this when creating the schedule, as the service will automatically do this for you.
If measuring very early prompt emission is important to your science, ensure that you visit the target source first, before your first scan on the gain calibrator. Observing your target for a few minutes before going to the gain calibrator should not be a problem, as the gains can be reliably extrapolated at cm wavelengths. Also, if you are observing in the 16cm band, there is a very good chance you will have a source in your field that is bright enough to allow for self-calibration.
Keep your scans short so the service can easily cut it to fit the earliest available slot. If authorised to do so, the service can automatically shorten the schedule you provide in order to fit it into the earliest possible available slot; how it does this is described in Section 2.5.9. If you keep your scans short (ie. don't have hour long scans between visits to your calibrator) then the shortener will act much more reliably.
Do not include a flux density calibrator in your schedule. The service will automatically add the appropriate flux density calibrator to your schedule, depending on frequency and sidereal time.
The service shortens a schedule in four stages.
The hard end time of the available slot is determined. This may be because the source has set, or some sort of maintenance or reconfigure is scheduled. All scans that end after that hard end time are immediately deleted from the schedule, except if it is the flux density calibrator scan that the service added.
As mentioned in Section 2.5.8, this step can cause problems if your scans are too long. Consider a scenario where the scan that ends after the hard end time is scheduled to run for 1 hour. Even if the hard end time is only 10 minutes from the scheduled end of this scan, the entire scan is cut, and you have lost a potential 50 minutes of observing time.
If the schedule is still too long at this point, all scans on the nominated calibration source may be shortened to have a dwell length of 1 minute. This is done from the first scan and continues until all the calibrator scans have been shortened, or the schedule now finishes before the hard end time.
If the schedule is still too long at this point, then the scan on the flux density calibrator is shortened by 1 minute.
If the schedule is still too long at this point, then the source scans are shortened. The service calculates the amount of time that needs to be cut for the schedule to fit in the slot, and then subtracts as much time as possible from the last remaining source scan. If the last remaining source scan is longer than the cut time, then the scan survives, otherwise it, and the following calibrator scan, are removed entirely. This process repeats until the schedule finishes before the hard end time.
If at the end of this process, the schedule length falls below the specified minimum useful time required, the start time will be delayed and the schedule restored to its original specification.