v 14.0 Third Party 996
Download for v 14.0 Deploy on Odoo.sh
Availability
Odoo Online
Odoo.sh
On Premise
Odoo Apps Dependencies Discuss (mail)
Lines of code 1723
Technical Name celery
LicenseLGPL-3
Websitehttps://www.novacode.nl
Versions 13.0 9.0 10.0 11.0 12.0 14.0
You bought this module and need support? Click here!
Availability
Odoo Online
Odoo.sh
On Premise
Odoo Apps Dependencies Discuss (mail)
Lines of code 1723
Technical Name celery
LicenseLGPL-3
Websitehttps://www.novacode.nl
Versions 13.0 9.0 10.0 11.0 12.0 14.0

Celery: Distributed Task Queue

Odoo integration of the Python #1 asynchronous task queue

Features:
  • Put model-methods on the Celery Task Queue.
  • Monitor and manage the Task Queue in Odoo.
  • All Exceptions are catched and available as State=Failure with Exception message/trace shown.
  • Requeue of Failed and Pending (stale) tasks.
  • No complex installation and setup requirements are needed, except the Celery setup.

Check out the links below, for more info about Celery.

Celery website

http://celeryproject.org
Celery installation

http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html

Put a model-method on the Celery Task Queue

Just Python code to call from an Odoo model

Example

Model and method:

  • model: "celery.example"
  • method: "schedule_import"

Celery Options:

Shall be provided to kwargs of the call_task method

celery = {
    'queue': 'high.priority',
    'countdown': 5,
    'retry': True,
    'retry_countdown_setting': 'MUL_RETRIES_SECS',
    'retry_countdown_multiply_retries_seconds': 60,
    'retry_policy': {
        'interval_start': 30
    }
}

Calling the task:

self.env["celery.task"].call_task("celery.example", "schedule_import", celery=celery)

Celery Options

All Celery options are optional (not required).

Option Description Celery Documentation
queue Name of the Celery/worker queue, the task shall be routed to. Routing Tasks: http://docs.celeryproject.org/en/latest/userguide/routing.html
countdown The countdown is a shortcut to set ETA by seconds into the future. ETA and Countdown: http://docs.celeryproject.org/en/latest/userguide/calling.html#eta-and-countdown
retry Set to True to enable the retry of sending task messages. Message Sending Retry: http://docs.celeryproject.org/en/latest/userguide/calling.html#message-sending-retry
retry_countdown_setting Specify whether and how to increase the Retry Countdown by:
  • ADD_SECS:
    countdown = countdown + retry_countdown_add_seconds
  • MUL_RETRIES:
    countdown = countdown * request.retries
  • MUL_RETRIES_SECS:
    countdown = request.retries * retry_countdown_multiply_retries_seconds
This is a custom option which affects the Celery countdown option.
retry_countdown_add_seconds Specify the seconds (integer field) to add to the Retry Countdown. This is a custom option which affects the Celery countdown option.
retry_countdown_multiply_retries_seconds Specify the seconds (integer field) to multiply with request retries, which becomes the Retry Countdown. This is a custom option which affects the Celery countdown option.
retry_policy Options when retrying publishing a task message in the case of connection loss or other connection errors. Message Sending Retry: http://docs.celeryproject.org/en/latest/userguide/calling.html#message-sending-retry

Extra kwargs

Extra kwargs are optional (not required).

Kwarg Description
transaction_strategy Specifies when the task shall apply (ORM create and send to Celery MQ):
  • after_commit:
    Apply task after commit of the main/caller transaction (default setting).
  • immediate:
    Apply task immediately from the main/caller transaction, even if it ain't committed yet.

Monitor and control the Celery Task Queue

List of queued tasks

Task Failure info

Tasks waiting for Retry

Task waiting for Retry with Failure info

Requeue a Failed Task

Requeue multiple Failed Tasks

Installation and configuration

Celery and Message broker

Celery Installation

http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html

Odoo configuration

All you need is to determine the Odoo Celery user and setup the credentials for XML-RPC authentication.

This enables the Celery process to authenticate in Odoo, by the XML-RPC server.

To support different kind of deployment and build tools, the credentials can either be setup as:

  • (OS) Environment variables of the user running the Odoo server process: ODOO_CELERY_USER and ODOO_CELERY_PASSWORD

  • (OS) Environment variables for third-party broker: ODOO_CELERY_BROKER, ODOO_CELERY_BROKER_HEARTBEAT, ODOO_CELERY_WORKER_PREFETCH_MULTIPLIER

  • Put in the odoo.conf file, e.g:

    celery_user = Odoo-User
    celery_password = Odoo-User-Password

    See example.odoo.conf, visit link: https://github.com/novacode-nl/odoo-celery/blob/14.0/celery/example.odoo.conf

  • Put in the odoo.conf file, under section [celery] e.g:

    [celery]
    user = Odoo-User
    password = Odoo-User-Password

    See example_section_celery.odoo.conf, visit link: https://github.com/novacode-nl/odoo-celery/blob/14.0/celery/example_section_celery.odoo.conf

Start the Celery (Odoo) worker server

Check the Celery Documentation for more ways to start and manage the server/proces. E.g. by Supervisor

The Celery (Odoo) worker => Python file odoo.py, which is located directly under the "celery" module directory.

Start the worker (default/celery queue) on command line, whereas "odoo" references to the Python file:

# celery -A odoo worker --loglevel=info

Start a worker as a named queue:

# celery -A odoo worker --loglevel=info -Q high.priority -n high.priority

"Celery Example" module

Demo of 2 implemented example tasks, queued by 2 buttons

Check out the "Celery Example" module

  • After installation, go to the menu "Celery / Example Task".
  • Click button(s) "Queue create Line" shown in screensshot, which puts a task on the queue.
  • Check the queue (menu: Celery / Tasks). Check the form of the Example record - new Lines had been created.
Celery Examples module

https://apps.odoo.com/apps/modules/11.0/celery_example/

Changelog

0.27

  • Improvement: Add tracking for field exc_info (Exception Info).
    (GitHub PR: https://github.com/novacode-nl/odoo-celery/pull/38)
  • Imrpvovement: Implement broker connection based on env.vars
    (GitHub PR: https://github.com/novacode-nl/odoo-celery/pull/41)

0.26

  • Fix: AccessDenied error is raised if no password was passed to the function, but this exception was not imported.
    (GitHub PR: https://github.com/novacode-nl/odoo-celery/pull/37)

0.25

  • Fix relating to 0.21 (support inactive Celery User). Inactive user not allowed anymore, since recent Odoo 14.0
    Override the res.users access check method: only for Celery user (from config) remove the active check.

0.24

  • Fix crash (upon install, update) due to faulty config of Window Action "Flag/update task: Stuck" (action_celery_task_set_state_stuck) - introduced by porting to Odoo 14.0.

0.23

  • Fix cancellation of stucked tasks(s) wasn't possible.
  • Fix requeue of stucked tasks(s) wasn't possible.

0.22

  • Upon processing task rpc_run_task (method), only update the related record fields (res_model, res_ids) if these have a value set in the method its result.
    This enables the API to store these related record fields upfront, when the task is created.

0.21

  • Some security hardening: support inactive Celery User, which refuses regular (web) authentication/login.

0.20

  • [FIX] In call_task() the search of celery.task.setting results in Access Denied by ACLs for operation: read, model: celery.task.setting.
    Solved by calling the celery.task.setting search() with sudo().

0.19

  • [FIX] Related record: res_ids (now ListSerialized field).
  • Also change kwargs field to sensible class(name): TaskSerialized to KwargsSerialized.

0.18

  • Configurable (database) transaction strategy, when the task shall apply (ORM create and send to Celery MQ).
  • From now on - by default a task shall apply after commit of the main/caller transaction, instead of immediately in the main/caller transaction.

0.17

  • Task scheduling added - being able to schedule tasks in a specified time interval or certain day(s) of the week.
  • A new task state - Scheduled, is handled by an Odoo cron job - "Celery: Handle Scheduled Tasks".

0.16

  • Configurable celery queues added to task settings.

0.15

  • Scheduled (cron) cleanup of tasks - with optionally specifying/overriding: (1) timedelta (days=90, hours, minutes, seconds), (2) states to keep and (3) batch_size=100.
  • Create database index for the State Time (state_date) field.

0.14

  • Create database index for the Reference field (ref).

0.13

  • Get XML-RPC URL from ir.config.parameter (celery.celery_base_url) by Settings.

0.12

  • Also support to get the Celery (Odoo) user and password from the odoo.conf file, under section [options] too.

0.11

  • Put task into a new state "Retrying", right before the Retry starts.

0.10

  • Renamed task retry settings: 'MUL_RETR' to 'MUL_RETRIES', 'MUL_RETR_SECS' to 'MUL_RETRIES_SECS'.

0.9

  • Hide (Odoo) password in the "retry" payload used by the MQ broker for XML-RPC authentication.
    GitHub issue: https://github.com/novacode-nl/odoo-celery/issues/17

0.8

  • Fix task retry countdown/interval ignored. For more info see https://github.com/novacode-nl/odoo-celery/issues/14
  • Add task retry countdown settings, to increase countdown during retries ('ADD_SECS', 'MUL_RETRY', 'MUL_RETRY_SECS').
  • Search view of stuck tasks: (1) Add/show field reference (2) search-view with filters and grouping.
  • Track task changes of fields model, method, handle_stuck
  • Disable create/copy in the task form-view.

0.7

  • Task Reference, which serves:
    • Easier searching for tasks by Reference.
    • Check (ORM query) before call_task() call, to prevent redundant creation of recurring Pending tasks.

0.6

Introduction of "Stuck" tasks. A stuck task could be caused, for example by: a stopped/restarted Celery worker, message broker, or server.
  • Settings: to specify when a task (model, method) is Stuck after seconds from Started or Retry (states).
  • Stuck Tasks Report: which shows Stuck (not completed) tasks.
  • Cron or manually put tasks in "Stuck" state.
  • Ability to cancel (Pending or Stuck) tasks, which never completed.
  • Track and messaging about state change. Chatter on form view.

0.5

  • Routing tasks to specific (named) queues.

0.4

  • FIX: Store task state (Started, Retry) before execution.

0.3

  • Hide (Odoo) password in payload used by the broker for XML-RPC authentication.
    GitHub issue: https://github.com/novacode-nl/odoo-celery/issues/4

0.2

  • Task state information.

0.1

  • Initial version.

Please log in to comment on this module

  • The author can leave a single reply to each comment.
  • This section is meant to ask simple questions or leave a rating. Every report of a problem experienced while using the module should be addressed to the author directly (refer to the following point).
  • If you want to start a discussion with the author, please use the developer contact information. They can usually be found in the description.
Please choose a rating from 1 to 5 for this module.
thank you for this module
by
Pascal de Sélys
on 6/8/22, 3:31 PM

Thanks for this module that allows to use celery with odoo 


There are no comments yet!