Sidekiq for your ROR application

I - Background jobs

Before understanding Sidekiq and its necessity, we must know what background jobs is first.

Background jobs are jobs in the system that run in the background without affecting the normal operations in the system.
These jobs are used to reduce the manual effort and to automate the process.
They can run in the background without any user input and can be scheduled to run when the system load is low.

E.g: Sending email, creating CSV files, charging credit cards, ...

Please take a look this illustration for creating a CSV file and notify when this action finishes without using background jobs.

Can you see, user has to wait all of task finish sequentially for this action after 3.1s. But, when using background jobs, this flow can be optimized as follow:

II - Sidekiq

• Sidekiq is a gem that provides a background jobs framework for Ruby.
• This gem can help you improve performance and response wait time.
• Sidekiq works closely with Redis 4.0+ to organize job queues and store statistics
• Multi-threading

III - Setup Sidekiq for your application

1. Add Sidekiq to your Gemfile:

# Gemfile

gem 'sidekiq'

After that, run bundle install to install your new gem

2. Setup Redis and config Sidekid to work with Redis

You can follow this link: https://redis.io/topics/quickstart

In case you're using docker-compose, just need to add this block to docker-compose.yml

services:
  redis:
    image: "redis:alpine"
    command: redis-server
    ports:
      - 6379:6379
    volumes:
      - $PWD/redis-data:/var/lib/redis
      - $PWD/redis.conf:/usr/local/etc/redis/redis.conf
    environment:
      - ALLOW_EMPTY_PASSWORD=yes

Connect to Redis by config/initializers/sidekiq.rb :

# config/initializers/sidekiq.rb
# frozen_string_literal: true
sidekiq_config = { url: ENV['REDIS_URL'] }

Sidekiq.configure_server do |config|
  config.redis = sidekiq_config
end

Sidekiq.configure_client do |config|
  config.redis = sidekiq_config
end

Create .env file like this in your repository's root:

REDIS_URL=redis://redis:6379/0

3. Define worker classes:

# app/workers/test_worker.rb
# frozen_string_literal: true

class TestWorker
  include Sidekiq::Worker
  sidekiq_options retry: false

  def perform(how_hard = 'super hard', how_long = 1)
    sleep how_long
    logger = Logger.new($stdout)
    logger.info "Working' #{how_hard}"
  end
end

4. Running Sidekiq as a separate process:

You can use this command line:

bundle exec sidekiq

Or place this block into your docker-compose.yml below services section:

sidekiq:
    container_name: side_kiq
    depends_on:
      - redis
    build:
      context: .
      dockerfile: docker/backend/Dockerfile
    command: bundle exec sidekiq
    volumes:
      - .:/app
    env_file:
      - .env

5. Start scheduling your workers, just place this line where you want your job run.

# This task is to be executed in 5sec 

::TestWorker.perform_async('hard', 5)

There are other functions (perform_in, perform_at, ...), queues / retries management that you can find on Sidekiq's wiki.

But from now, your application can work with background jobs, right ;) ?

Happy coding!