Celery by dummy

Publish in

Software

4 views

Please download to get full document.

View again

of 12
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Share
Description
1. By Os Shiowattana (@djshiow) 2. Who am I ● Os Shiowattana (@djshiow) ● Software engineer for ~10 years ● Currently at Origami Logic ○ We’re looking for great…
Transcript
  • 1. By Os Shiowattana (@djshiow)
  • 2. Who am I ● Os Shiowattana (@djshiow) ● Software engineer for ~10 years ● Currently at Origami Logic ○ We’re looking for great developers ;-) ● Python experience: 3 months ○ Teaching is a good way to learn
  • 3. Outline ● What is Celery? ● What is it for? ● Simple Example http://www.myhdiet.com/sharons-raw-celery-soup/
  • 4. What is celery? ● Task queue ○ Distributed ○ Asynchronous ● Create tasks and add it to the queue ● Workers pick them up & work on them
  • 5. How does it work? Broker Worker Worker Worker …. Producer Result Store
  • 6. What is it for? ● Long running asynchronous task ● Execute a task “reliably” ○ Asynchronous + Retries ○ e.g. Interactions with external APIs ● Scheduling Periodic tasks
  • 7. How does Origami Logic use it? ● Fetch marketing metrics from all your social network daily ○ Periodic ○ Reliable (with retries) / Traceable ○ Distributed (multiple machines) & Scalable ○ Rate limits
  • 8. Instagram ● Populate follower feed >7 M followers Source: http://blogs.vmware.com/vfabric/2013/04/how-instagram-feeds-work-celery-and-rabbitmq.html
  • 9. Simple Example ● Adding two numbers asynchronously !! ● Setup: ○ Install RabbitMQ ○ Install Celery From: http://celery.readthedocs.org/en/latest/getting-started/first-steps-with-celery.html
  • 10. First, define Celery app & tasks from celery import Celery #setup app app = Celery('tasks', backend='amqp' , broker='amqp://guest@localhost//') #Define task @app.task def add(x, y): return x + y Then, run celery worker celery -A tasks worker --loglevel=info Decorator to create a task class
  • 11. Lastly, trigger the task from tasks import add #run the add task asynchronously 3 seconds later #passing (4,4) as param add.apply_async((4, 4) , countdown=3) [2015-03-15 14:34:10,397: INFO/MainProcess]Received task: tasks.add[b68b0109-7e99-473b-8991-1da8adaa50e8] eta:[2015-03-15 14:34:13.385448+00:00] [2015-03-15 14:34:14,836: INFO/MainProcess] Tasktasks.add[b68b0109-7e99-473b-8991-1da8adaa50e8] succeeded in 0.0209855769999s: 8
  • 12. Reference ● http://www.slideshare.net/idangazit/an-introduction-to-celery ● http://blogs.vmware.com/vfabric/2013/04/how-instagram-feeds-work- celery-and-rabbitmq.html ● http://celery.readthedocs.org/en/latest/
  • Related Search
    We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks