Concurrency

• Concurrency is a way to structure the program to do many things at once, but it may not be executed all at once (not necessarily parallelism)
• Patterns: threading, multi-processing, and async programming.
• A common pattern: looping indefinitely while taking items from a queue and processing each with a function, the loop ends when the queue produces a sentinel value. In this case, sentinels are called poison pills.
• Two queues needed, one for jobs/tasks, one for collecting results
  • Celery and RQ for task queues.
  • Redis, RabbitMQ, Amazon SQS, for message queues.

Resources

• Seven Concurrency Models in Seven Weeks