The Web Interface

The Crab dashboard allows the status of the jobs to be monitored. On this page, the job status column will change color to indicate the status, and it will flash while the job is running. Clicking on the status will lead to the most recent output recorded for the job.

The host and user columns contain links leading to a summary page of the cron jobs for a given user or host. From this page, the links below each table can be used to show deleted jobs, and to display the raw crontab as last imported.

Clicking on a job ID or command link leads to the job information page, giving a summary of the job’s parameters and a table of the most recent events. Clicking the status of any job finish event leads to the corresponding output.

Job Configuration

Below the summary on the job information page, there is a link allowing the job’s configuration to be edited. If a job is deleted, then its configuration is considered to be orphaned. In this case, when configuring a job for which no configuration exists, the system will offer a list of orphaned configurations for re-linking. This should be used when the job is actually the continuation of a previous job. Note that notifications which are attached to specific jobs are linked via the configuration. Therefore re-linking the configuration will re-attach all associated notifications.

However this problem can generally be avoided by giving the jobs suitable names via the CRABID parameter. Crab will then be able to recognize jobs by name even if the command string changes.

The grace period specifies how close to the scheduled time the job must start in order not to be considered missed. The time-out is the maximum expected duration of the job. If it runs for longer than this, it will be marked as stopped with timed-out (error) status. Note that the job may actually still be running when this status is displayed. If the job is restarted, or reported as already running, during the time-out period, then the time-out is reset. If either of these timing parameters are left blank then the default values of 2 minutes grace period and 5 minutes time-out will be used.

Regular expression patterns used to determine success or failure and to identify warnings can be given. These patterns are compared to the standard output and standard error of the job when it finishes, but do not override a more severe status. For example if a job is reported as finishing with failure, then it will be logged as such even if the success or warning patterns match. If none of the patterns match then the status is logged as it was reported, unless a success pattern was defined. If the success pattern does not match then the status will be failure if the was no failure pattern or unknown if there was a failure pattern which did not match.

The “Inhibit execution” checkbox can be use to temporarily request that a job not be run. This setting is stored in the Crab server and passed to the client when it reports that a job is being started. Note that there is no guarantee that the job will not be run while this option is selected: the client could fail to connect to the server before starting the job, or it could choose to ignore the inhibit setting. The crabsh wrapper shell reads a configuration parameter allow_inhibit from the crabsh section of the cran.ini file to determine whether inhibit requests should be honored. (The default value is true, i.e. it will not run the job if it receives the inhibit flag in response to its job starting message.)

The job configuration page also allows jobs to be marked as deleted. Normally this would be done by importing a new crontab without that job in it, but having this available on the web interface is useful in situations such as the host being inaccessible. Note that if a start or finish event is received from the job, but the Crab server is still able to identify it, then the job should be automatically marked as not deleted.

There is also the option to alter the job identifier. However care must be taken to also update it in the job itself, for example via the CRABID parameter in the crontab. If the identifier is changed via the web server but not in the job, then the Crab server will identify it as a new job the next time it receives a start or finish report from it.

Notifications

Crab includes a configurable notifications system, which currently supports sending notification messages by email. Notifications can either be attached to a specific job, or configured by host name and/or by user name.

A link below the summary on the job information page allows notifications to be attached to that job. Check-boxes for each notification can be used to select which severity of events should be featured, and whether the job output should be included. The schedule box should contain a cron-style schedule specification (e.g. 0 12 * * *), and if left blank, will default to the value given in the crabd.ini file, allowing all notification schedules to be managed in one place. Notifications will only be sent if there are relevant events, so it is possible to request almost-immediate error warnings by including a schedule of * * * * * and selecting errors only.

The add and delete links can be used to add and remove notifications, but the changes are not saved until the Configure button is clicked.

The drop-down menu which appears when the mouse is positioned over the Crab heading at the top of each page includes a link to the main notifications page. This allows notifications to be configured by host name and/or by user name. Notifications will include any jobs where the host and user match the specified values, but if either is left blank, then it will match all entries.

Additional Job Actions

Depending on the state of a job, additional links may appear below the summary on the job information page. These are:

  • “Clear status”: this appears when the job is in a warning or error state. Selecting this option sets the job state to “Cleared”, which you can use to acknowledge the problem. The job’s status will then be shown in green on the dashboard.
  • “Resume inhibited job”: this appears when the inhibit setting has been selected on the job configuration page. The link provides a convenient means of removing the inhibit setting.