Image Upload is a very basic requirement in any Web/Mobile application. This service, if not written properly can prove to be very problematic to scale very soon.

We had an image upload API exposed which took in multipart images as input and returned a json response of the created object. Picture something like below:

image = Image.create!(image_params)
render json: image

I had used carrierwave  to handle the image upload.

Problems?

We had several versions of the image, like small, logo, medium, low etc for rendering in various places. What carrierwave does, it processes the images for these versions during the time of upload only. And that takes time. A simple request for a 2.2 MB image used to take somewhere around 3-4 seconds which wasn't acceptable.

There was no ready made solution that worked good. There was carrierwave backgrounder there, but it wasn't under active maintenance.

I decided to employ sidekiq for the purpose. I am a great fan of this gem and I try to use it wherever possible.

I added a column temp_image in my db table, leaving image column as such.

# == Schema Information
#
# Table name: images
#
#  id               :bigint           not null, primary key
#  height           :decimal(, )
#  image            :string
#  meta             :string           default([]), is an Array
#  rekognition_meta :jsonb            is an Array
#  temp_image       :string           not null
#  width            :decimal(, )
#  created_at       :datetime         not null
#  updated_at       :datetime         not null

and started uploading the images to temp_image field, created another uploader for the same, but with no version, and triggered a sidekiq worker after the db commit.

My Image class looked something like below:

# Image ORM
class Image < ApplicationRecord

  mount_uploader :temp_image, TempImageUploader
  mount_uploader :image, ImageUploader

  after_commit :create_versions, on: :create

  validate :temp_image_presence
  validate :image_dimensions, on: :create

  attr_accessor :creation_token

  def url
    image&.url || temp_image.url
  end

  alias original_image_url url
  alias upload_id id

  %w[small logo fixed high medium low].each do |version_type|
    define_method "#{version_type}_version_url" do
      return temp_image.url unless image&.send(version_type)&.url

      image.send(version_type).url
    end
  end

  def create_versions
    return if Rails.env.test? || Rails.env.development?

    ::ImageProcessingWorker.perform_async(id)
  end

  def temp_image_presence
    # didn't use validates :temp_image, presence: true because wanted to add error to image, not temp image
    errors.add(:image, "can't be blank") unless temp_image.present?
  end

  def image_dimensions
    return if width.to_i > 120 && height.to_i > 120

    errors.add(:image, 'Dimensions of uploaded image should be not less than 120x120 pixels.')
  end
end

TempImageUploader was left plain and it looked like:

class TempImageUploader < BaseUploader
  def extension_whitelist
    %w[jpg jpeg gif png webp]
  end

  def content_type_whitelist
    %r{image/}
  end

  before :cache, :capture_size_before_cache

  def capture_size_before_cache(file)
    if model.width.nil? || model.height.nil?
      model.width, model.height = `identify -format "%wx %h" #{file.path}`.split(/x/).map { |dim| dim.to_i }
    end
  rescue StandardError => _e
    ::Rails.logger.info 'Image reading failed!'
  end
end

While the ImageUploader looked like:

class ImageUploader < BaseUploader
  # Include RMagick or MiniMagick support:
  # include CarrierWave::RMagick
  include CarrierWave::MiniMagick

  def extension_whitelist
    %w[jpg jpeg gif png webp]
  end

  def content_type_whitelist
    %r{image/}
  end

  # Process files as they are uploaded:
  # process resize_to_fit: [200, 300]
  # process resize_to_fit: [800, 800]

  # Create different versions of your uploaded files:
  version :small do
    process resize_to_fit: [300, 300]
  end

  version :logo do
    process resize_to_fit: [150, 150]
  end

  version :fixed do
    process resize_and_pad: [400, 400]
  end

  version :high do
    process quality: 90
  end

  version :medium do
    process quality: 80
  end

  version :low do
    process quality: 40
  end
end

The ImageProcessingWorker looked like:

class ImageProcessingWorker
  include Sidekiq::Worker
  sidekiq_options queue: 'carrierwave', retry: 10

  def perform(upload_id)
    object = Image.find(upload_id)
    object.image = object.temp_image
    object.save

    raise ActiveRecord::RecordInvalid unless object.image.present?
  end
end

And voila, now the response time improved drastically to below 100ms.

I have many more interesting use cases of sidekiq|background processing that I'll write up shortly.