pyfarm.master.api.jobs module¶
Jobs¶
This module defines an API for managing and querying jobs
-
class
pyfarm.master.api.jobs.
JobIndexAPI
[source]¶ Bases:
flask.views.MethodView
-
get
()[source]¶ A
GET
to this endpoint will return a list of all jobs.-
GET
/api/v1/jobs/ HTTP/1.1
¶ Request
GET /api/v1/jobs/ HTTP/1.1 Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json [ { "title": "Test Job", "state": "queued", "id": 1 }, { "title": "Test Job 2", "state": "queued", "id": 2 } ]
Statuscode 200: no error -
-
methods
= ['GET', 'POST']¶
-
post
()[source]¶ A
POST
to this endpoint will submit a new job.-
POST
/api/v1/jobs/ HTTP/1.1
¶ Request
POST /api/v1/jobs/ HTTP/1.1 Accept: application/json { "end": 2.0, "title": "Test Job 2", "jobtype": "TestJobType", "data": { "foo": "bar" }, "software_requirements": [ { "software": "blender" } ], "start": 1.0 }
Response
HTTP/1.1 201 CREATED Content-Type: application/json { "time_finished": null, "time_started": null, "end": 2.0, "time_submitted": "2014-03-06T15:40:58.335259", "jobtype_version": 1, "jobtype": "TestJobType", "jobqueue": None "start": 1.0, "priority": 0, "state": "queued", "parents": [], "hidden": false, "project_id": null, "ram_warning": null, "title": "Test Job 2", "tags": [], "user": null, "by": 1.0, "data": { "foo": "bar" }, "ram_max": null, "notes": "", "batch": 1, "project": null, "environ": null, "requeue": 3, "software_requirements": [ { "min_version": null, "max_version": null, "max_version_id": null, "software_id": 1, "min_version_id": null, "software": "blender" } ], "tag_requirements": [ { "tag": "workstation", "negate": true } ], "id": 2, "ram": 32, "cpus": 1, "children": [] }
Statuscode 201: a new job item was created Statuscode 400: there was something wrong with the request (such as invalid columns being included) Statuscode 404: a referenced object, like a software or software version, does not exist Statuscode 409: a conflicting job already exists -
-
-
class
pyfarm.master.api.jobs.
JobNotifiedUsersIndexAPI
[source]¶ Bases:
flask.views.MethodView
-
get
(job_name)[source]¶ A
GET
to this endpoint will return a list of all users to be notified on events in this job.-
GET
/api/v1/jobs/[<str:name>|<int:id>]/notified_users/ HTTP/1.1
¶ Request
GET /api/v1/jobs/Test%20Job%202/notified_users/ HTTP/1.1 Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json [ { "id": 1, "username": "testuser", "email": "testuser@localhost" } ]
Statuscode 200: no error Statuscode 404: job not found -
-
methods
= ['GET', 'POST']¶
-
post
(job_name)[source]¶ A
POST
to this endpoint will add the specified user to the list of notified users for this job.-
POST
/api/v1/jobs/[<str:name>|<int:id>]/notified_users/ HTTP/1.1
¶ Request
POST /api/v1/jobs/Test%20Job/notified_users/ HTTP/1.1 Accept: application/json { "username": "testuser" "on_success": true, "on_failure": true, "on_deletion": false }
Response
HTTP/1.1 201 CREATED Content-Type: application/json { "id": 1 "username": "testuser" "email": "testuser@example.com" }
Statuscode 201: a new notified user entry was created Statuscode 400: there was something wrong with the request (such as invalid columns being included) Statuscode 404: the job or the specified user does not exist -
-
-
class
pyfarm.master.api.jobs.
JobSingleNotifiedUserAPI
[source]¶ Bases:
flask.views.MethodView
-
delete
(job_name, username)[source]¶ A
DELETE
to this endpoint will remove the specified user from the list of notified users for this job.-
DELETE
/api/v1/jobs/[<str:name>|<int:id>]/notified_users/<str:username> HTTP/1.1
¶ Request
DELETE /api/v1/jobs/Test%20Job/notified_users/testuser HTTP/1.1 Accept: application/json
Response
HTTP/1.1 204 NO_CONTENT
Statuscode 204: the notified user was removed from this job or wasn’t in the list in the first place Statuscode 404: the job or the specified user does not exist -
-
methods
= ['DELETE']¶
-
-
class
pyfarm.master.api.jobs.
JobSingleTaskAPI
[source]¶ Bases:
flask.views.MethodView
-
get
(job_name, task_id)[source]¶ A
GET
to this endpoint will return the requested task-
GET
/api/v1/jobs/[<str:name>|<int:id>]/tasks/<int:task_id> HTTP/1.1
¶ Request
GET /api/v1/jobs/Test%20Job%202/tasks/1 HTTP/1.1 Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json { "time_finished": null, "agent": null, "attempts": 0, "frame": 2.0, "agent_id": null, "job": { "id": 1, "title": "Test Job" }, "time_started": null, "state": "running", "project_id": null, "id": 2, "time_submitted": "2014-03-06T15:40:58.338904", "project": null, "parents": [], "job_id": 1, "hidden": false, "children": [], "priority": 0 }
Statuscode 200: no error -
-
methods
= ['GET', 'POST']¶
-
post
(job_name, task_id)[source]¶ A
POST
to this endpoint will update the specified task with the data in the request. Columns not specified in the request will be left as they are. The agent will use this endpoint to inform the master of its progress.-
POST
/api/v1/jobs/[<str:name>|<int:id>]/tasks/<int:task_id> HTTP/1.1
¶ Request
PUT /api/v1/job/Test%20Job/tasks/1 HTTP/1.1 Accept: application/json { "state": "running" }
Response
HTTP/1.1 200 OK Content-Type: application/json { "time_finished": null, "agent": null, "attempts": 0, "failures": 0, "frame": 2.0, "agent_id": null, "job": { "id": 1, "title": "Test Job" }, "time_started": null, "state": "running", "project_id": null, "id": 2, "time_submitted": "2014-03-06T15:40:58.338904", "project": null, "parents": [], "job_id": 1, "hidden": false, "children": [], "priority": 0 }
Statuscode 200: the task was updated Statuscode 400: there was something wrong with the request (such as invalid columns being included) -
-
-
class
pyfarm.master.api.jobs.
JobTasksIndexAPI
[source]¶ Bases:
flask.views.MethodView
-
get
(job_name)[source]¶ A
GET
to this endpoint will return a list of all tasks in a job.-
GET
/api/v1/jobs/[<str:name>|<int:id>]/tasks HTTP/1.1
¶ Request
GET /api/v1/jobs/Test%20Job%202/tasks/ HTTP/1.1 Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json [ { "hidden": false, "id": 3, "attempts": 0, "priority": 0, "time_started": null, "time_submitted": "2014-03-06T15:49:51.892228", "frame": 1.0, "time_finished": null, "job_id": 2, "project_id": null, "state": "queued", "agent_id": null }, { "hidden": false, "id": 4, "attempts": 0, "priority": 0, "time_started": null, "time_submitted": "2014-03-06T15:49:51.892925", "frame": 2.0, "time_finished": null, "job_id": 2, "project_id": null, "state": "queued", "agent_id": null } ]
Statuscode 200: no error -
-
methods
= ['GET']¶
-
-
class
pyfarm.master.api.jobs.
SingleJobAPI
[source]¶ Bases:
flask.views.MethodView
-
delete
(job_name)[source]¶ A
DELETE
to this endpoint will mark the specified job for deletion and remove it after stopping and removing all of its tasks.-
DELETE
/api/v1/jobs/[<str:name>|<int:id>] HTTP/1.1
¶ Request
DELETE /api/v1/jobs/1 HTTP/1.1 Accept: application/json
Response
HTTP/1.1 204 NO_CONTENT
Statuscode 204: the specified job was marked for deletion Statuscode 404: the job does not exist -
-
get
(job_name)[source]¶ A
GET
to this endpoint will return the specified job, by name or id.-
GET
/api/v1/jobs/[<str:name>|<int:id>] HTTP/1.1
¶ Request
GET /api/v1/jobs/Test%20Job%202 HTTP/1.1 Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json { "ram_warning": null, "title": "Test Job", "state": "queued", "jobtype_version": 1, "jobtype": "TestJobType", "environ": null, "user": null, "priority": 0, "time_finished": null, "start": 2.0, "id": 1, "notes": "", "notified_users": [] "ram": 32, "tags": [], "hidden": false, "data": { "foo": "bar" }, "software_requirements": [ { "software": "blender", "software_id": 1, "min_version": null, "max_version": null, "min_version_id": null, "max_version_id": null } ], "tag_requirements": [ { "tag": "workstation", "negate": true } ], "batch": 1, "time_started": null, "time_submitted": "2014-03-06T15:40:58.335259", "requeue": 3, "end": 4.0, "parents": [], "cpus": 1, "ram_max": null, "children": [], "by": 1.0, "project_id": null }
Statuscode 200: no error Statuscode 404: job not found -
-
methods
= ['DELETE', 'GET', 'POST']¶
-
post
(job_name)[source]¶ A
POST
to this endpoint will update the specified job with the data in the request. Columns not specified in the request will be left as they are. If the “start”, “end” or “by” columns are updated, tasks will be created or deleted as required.-
POST
/api/v1/jobs/[<str:name>|<int:id>] HTTP/1.1
¶ Request
PUT /api/v1/jobs/Test%20Job HTTP/1.1 Accept: application/json { "start": 2.0 }
Response
HTTP/1.1 201 OK Content-Type: application/json { "end": 4.0, "children": [], "jobtype_version": 1, "jobtype": "TestJobType", "time_started": null, "tasks_failed": [], "project_id": null, "id": 1, "software_requirements": [ { "software": "blender", "min_version": null, "max_version_id": null, "software_id": 1, "max_version": null, "min_version_id": null } ], "tags": [], "environ": null, "requeue": 3, "start": 2.0, "ram_warning": null, "title": "Test Job", "batch": 1, "time_submitted": "2014-03-06T15:40:58.335259", "ram_max": null, "user": null, "notes": "", "data": { "foo": "bar" }, "tag_requirements": [ { "tag": "workstation", "negate": true } ], "ram": 32, "parents": [], "hidden": false, "priority": 0, "cpus": 1, "state": "queued", "by": 1.0, "time_finished": null }
Statuscode 200: the job was updated Statuscode 400: there was something wrong with the request (such as invalid columns being included) -
-
-
class
pyfarm.master.api.jobs.
SingleTaskOnAgentFailureAPI
[source]¶ Bases:
flask.views.MethodView
-
delete
(job_id, task_id, agent_id)[source]¶ A
DELETE
to this endpoint will remove the specified agent from the list of agents that failed to execute this task-
DELETE
/api/v1/jobs/<int:id>/tasks/<int:task_id>/failed_on_agents/<str:agent_id> HTTP/1.1
¶ Request
DELETE /api/v1/jobs/12/tasks/1234/failed_on_agents/02f08241-c556-4355-9e5e-33243d8c4577 HTTP/1.1
Response
HTTP/1.1 204 NO_CONTENT
Statuscode 204: the given agent was removed from the list of agents that failed to execute this task Statuscode 404: the job, task or agent does not exist -
-
methods
= ['DELETE']¶
-
-
class
pyfarm.master.api.jobs.
TaskFailedOnAgentsIndexAPI
[source]¶ Bases:
flask.views.MethodView
-
get
(job_id, task_id)[source]¶ A
GET
to this endpoint will return a list of all agents that failed to execute this task-
GET
/api/v1/jobs/<int:job_id>/tasks/<int:task_id>/failed_on_agents/ HTTP/1.1
¶ Request
GET /api/v1/jobs/12/tasks/1234/failed_on_agents/ HTTP/1.1 Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json [ { "id": "02f08241-c556-4355-9e5e-33243d8c4577", "hostname": "agent1" } ]
Statuscode 200: no error Statuscode 404: job or task not found -
-
methods
= ['GET', 'POST']¶
-
post
(job_id, task_id)[source]¶ A
POST
to this endpoint will add the specified agent to the list of agents that failed to execute this task-
POST
/api/v1/jobs/<int:id>/tasks/<int:task_id>/failed_on_agents/ HTTP/1.1
¶ Request
POST /api/v1/jobs/12/tasks/1234/failed_on_agents/ HTTP/1.1 Accept: application/json Content-Type: application/json { "id": "02f08241-c556-4355-9e5e-33243d8c4577" }
Response
HTTP/1.1 201 CREATED Content-Type: application/json { "id": "02f08241-c556-4355-9e5e-33243d8c4577", "hostname": "agent1" }
Statuscode 201: a new entry was created Statuscode 400: there was something wrong with the request (such as invalid columns being included) Statuscode 404: the job, task or agent specified does not exist -
-
-
pyfarm.master.api.jobs.
parse_requirements
(requirements)[source]¶ Takes a list dicts specifying a software and optional min- and max-versions and returns a list of
JobRequirement
objects.Raises TypeError if the input was not as expected or ObjectNotFound if a referenced software or version was not found.
Parameters: requirements (list) – A list of of dicts specifying a software and optionally min_version and/or max_version.
Raises: - TypeError – Raised if
requirements
is not a list or if an entry inrequirements
is not a dictionary. - ValueError – Raised if there’s a problem with the content of at least one of the requirement dictionaries.
- ObjectNotFound – Raised if the referenced software or version was not found
- TypeError – Raised if
-
pyfarm.master.api.jobs.
schema
()[source]¶ Returns the basic schema of
Job
-
GET
/api/v1/jobs/schema HTTP/1.1
¶ Request
GET /api/v1/jobs/schema HTTP/1.1 Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json { "batch": "INTEGER", "by": "NUMERIC(10, 4)", "cpus": "INTEGER", "data": "JSONDict", "end": "NUMERIC(10,4)", "environ": "JSONDict", "hidden": "BOOLEAN", "id": "INTEGER", "jobtype": "VARCHAR(64)", "jobtype_version": "INTEGER", "jobqueue": "VARCHAR(255)", "notes": "TEXT", "priority": "INTEGER", "project_id": "INTEGER", "ram": "INTEGER", "ram_max": "INTEGER", "ram_warning": "INTEGER", "requeue": "INTEGER", "start": "NUMERIC(10,4)", "state": "WorkStateEnum", "time_finished": "DATETIME", "time_started": "DATETIME", "time_submitted": "DATETIME", "title": "VARCHAR(255)", "user": "VARCHAR(255)" }
Statuscode 200: no error -