Storage

crab.store

class crab.store.CrabStore
get_jobs(host=None, user=None, **kwargs)

Fetches a list of all of the cron jobs, excluding deleted jobs by default.

Optionally filters by host or username if these parameters are supplied. Other keyword arguments (e.g. include_deleted, crabid, command, without_crabid) are passed to the _get_jobs method.

delete_job(id_)

Mark a job as deleted.

undelete_job(id_)

Remove deletion mark from a job.

update_job(id_, **kwargs)

Updates job information.

Keyword arguments are passed on to the private _update_job method, and can include: crabid, command, time, timezone.

log_start(host, user, crabid, command)

Inserts a job start record into the database.

Returns a dictionary including a boolean value indicating whether the job inhibit setting is active or not.

log_finish(host, user, crabid, command, status, stdout=None, stderr=None)

Inserts a job finish record into the database.

The output will be passed to the write_job_output method, unless both stdout and stderr are empty.

get_job_config(id_)

Retrieve configuration data for a job by ID number.

write_job_output(finishid, host, user, id_, crabid, stdout, stderr)

Writes the job output to the store.

This will use the outputstore’s corresponding method if it is defined, otherwise it writes to this store.

get_job_output(finishid, host, user, id_, crabid)

Fetches the standard output and standard error for the given finish ID.

The result is returned as a two element list. Returns a pair of empty strings if no output is found.

This will use the outputstore’s corresponding method if it is defined, otherwise it reads from this store.

get_crontab(host, user)

Fetches the job entries for a particular host and user and builds a crontab style representation.

Please see crab.util.crontab.write_crontab for more details of how the crontab is constructed.

save_crontab(host, user, crontab, timezone=None, allow_filter=True)

Takes a list of crontab lines and uses them to update the job records.

It looks for the CRABID and CRON_TZ variables, but otherwise ignores everything except command lines. It also checks for commands starting with a CRABID= definition, but otherwise inserts them into the database as is.

If “allow_filter” is True (as is the default) then cron jobs are skipped if they have a specified user name or client host name which does not match the given host or user name.

Returns a list of warning strings.

check_job(*args, **kwargs)

Ensure that a job exists in the store.

Acquires the lock and then calls the private _check_job method.

write_raw_crontab(host, user, crontab)
get_raw_crontab(host, user)

crab.store.db

class crab.store.db.CrabDBLock(conn, error_class, cursor_args={}, ping=False)
class crab.store.db.CrabStoreDB(lock, outputstore=None)

Crab storage backend using a database.

Currently written for SQLite but since it uses the Python DB API it should be possible to generalize it by altering the queries based on the database type where necessary.

log_alarm(id_, status)

Inserts an alarm regarding a job into the database.

This is for alarms generated interally by crab, for example from the monitor thread. Such alarms are currently stored in an separate table and do not have any associated output records.

get_job_info(id_)

Retrieve information about a job by ID number.

write_job_config(id_, graceperiod=None, timeout=None, success_pattern=None, warning_pattern=None, fail_pattern=None, note=None, inhibit=False)

Writes configuration data for a job by ID number.

Returns the configuration ID number.

disable_inhibit(id_)

Disable the inhibit setting for a job.

This is a convenience routine to simply disable the inhibit job configuration parameter without having to read and write the rest of the configuration.

get_orphan_configs()

Make a list of orphaned job configuration records.

get_job_finishes(id_, limit=100, finishid=None, before=None, after=None, include_alreadyrunning=False)

Retrieves a list of recent job finish events for the given job, most recent first.

Can optionally find a particular finish, or finishes before or after a certain finish. In the case of finishes after a certain finish, the most recent event will be last.

ALREADYRUNNING events are only reported if the include_alreadyrunning argument is set.

get_job_events(id_, limit=100, start=None, end=None)

Fetches a combined list of events relating to the specified job.

Return events, newest first (with finishes first for the same datetime). This ordering allows us to apply the SQL limit on number of result rows to find the most recent events. It gives the correct ordering for the job info page.

get_events_since(startid, alarmid, finishid)

Extract minimal summary information for events on all jobs since the given IDs, oldest first.

get_fail_events(limit=40)

Retrieves the most recent failures for all events, combining the finish and alarm tables.

This method has to include a list of status codes to exclude since the filtering is done in the SQL. The codes skipped are CLEARED, LATE, SUCCESS, ALREADYRUNNING and INHIBITED.

delete_old_events(datetime_)

Delete events older than the given datetime.

get_notifications()

Fetches a list of notifications, combining those defined by a config ID with those defined by user and/or host.

get_job_notifications(configid)

Fetches all of the notifications configured for the given configid.

get_match_notifications(host=None, user=None)

Fetches matching notifications which are not tied to a configuration entry.

write_notification(notifyid, configid, host, user, method, address, time, timezone, skip_ok, skip_warning, skip_error, include_output)

Adds or updates a notification record in the database.

delete_notification(notifyid)

Removes a notification from the database.

crab.store.file

class crab.store.file.CrabStoreFile(dir)

Store class for cron job output.

This backend currently implements only the write_job_output and get_job_output methods, to allow it to be used as an “outputstore” along with CrabStoreDB.

write_job_output(finishid, host, user, id_, crabid, stdout, stderr)

Write the cron job output to a file.

The only parameter required to uniquely identify the event associated with this output is the “finishid”, but the host, user and job identifiers are also provided to allow hierarchical storage.

Only writes a stdout file (extension set by self.outext, by default txt), and a stderr file (extension self.errext, default err) if they are not empty.

get_job_output(finishid, host, user, id_, crabid)

Find the file containing the cron job output and read it.

As for write_job_output, only the “finishid” is logically required, but this method makes use of the host, user and job identifiers to read from a directory hierarchy.

Requires there to be an stdout file but allows the stderr file to be absent.

write_raw_crontab(host, user, crontab)

Writes the given crontab to a file.

get_raw_crontab(host, user)

Reads the given user’s crontab from a file.

crab.store.mysql

crab.store.sqlite

class crab.store.sqlite.CrabStoreSQLite(filename, outputstore=None)