Delayed Job Gem for CSV

It is essential to have a fast server as the delay in processing can literally kill a business’ revenue stream and frustrate the users. The speed of a website is the actual time it takes for a web page to load on the screen after being requested by the browser. Now consider a scenario where you need to use huge database and send emails to a list longer than the Wall of China. Exaggeration or not, sending emails to a long list of people will take time, increasing the processing time which will slow the server and annoy the users.

The requirement to perform asynchronous tasks on web applications away from the HTTP request handling cycle is a necessity bane for any organization handling heavy data. If you have tasks which are data heavy and need to be handled in the background, one of the best ways to combat it is through using Delayed Job. As the name itself suggests, the ‘Delayed Job’ is a ruby gem which can be applied to execute certain tasks in background process in the Rails environment. Scheduling tasks with a timetable against running all of them together, the former is preferred due to conservative usage of resources.

Delayed Job is especially more important when it comes to handling CSV files where you have records which are dynamic in nature and size of records are high. It takes a lot of RAM space and doesn’t accept other requests during the preparation of CSV file. To overcome this, Delayed Job gem is used. Check out the codes to find out more on how we can add this gem:

The Usual Way

def download require “csv” csv_string = CSV.generate do |csv| csv << [“ID”,….] @search.each do |site| csv << [site.id, ……..] end end send_data csv_string, :type => ‘text/csv’, :filename => ‘_sites_list.csv’, :disposition => ‘attachment’ end

In above scenario, there are large number of columns with 50k+ records when requested for all records (sites) it holds up the system with heavy load.

The Delayed Job Gem Method

def download

ExportCsv.new(@search.to_a.map(&:id), current_user.id).delay.perform

end

The link/button to download method should be an AJAX call.

The custom class for delayed job operations in lib folder.

export_csv.rb class ExportCsv < Struct.new(:site_ids, :user_id) def perform require “csv” User.find(user_id).update_attributes(xls_download: false) # pre-run, making sure. sites = Site.where(id: site_ids) CSV.open(“tmp/sites_xls/#{user_id}.csv”, “w+”) do |csv| csv << [“ID”, …..] sites.each do |site| csv << [site.id, …..] end end end def after(job) User.find(user_id).update_attributes(xls_download: true) end handle_asynchronously :perform end

As we cannot use session objects, a new attribute xls_download was create for users table. Attribute will be set to true in after method provided by the delayed job, once process of perform method is complete.

Once the csv preparation is complete, the file seems to be in the application path but how does it inform you it’s ready and has completed the process of writing to CSV?

download.js.erb alert(‘Preparing file to download, you will be notified once its complete…’); timeout(‘<%= SOME_DELAY %>’);

So what lies inside timeout function, how does system notify its ready to serve the file?

function timeout(time) { setTimeout(function() { $.ajax({ url: “/controller/check_if_ready?”, type : ‘GET’, dataType : ‘json’, success: function(response) { if(response.value === “success”) { var value = confirm(“File ready to download ?”); if (value == true) { window.location=”/controller/download_csv.csv”; } else if(value == false) { $.ajax({ url: “/controller/remove_file”}) } } else { timeout(‘4000’) // wait 4 seconds } }, }); }, time); } The AJAX call. check_if_ready? def check_if_ready? if current_user.set_xls_download? && File.exist?(@file_path) render json: { value: “success” } else render json: { value: nil } end end

Navigation to different sources.

It seems everything’s fine but what if the user navigates different pages.

Call the function in whichever layout necessary.

%script – if session[:download] timeout(‘#{DELAY}’);

Use a flag attribute to make poll calls, once the download is complete you can toggle back to nil/false.

Final few points

• Delay value can be configured based on count of records to reduce number of Ajax calls. But on navigation to different pages you might also want to have smaller delay, as user may make another request within that delay value.
• Delete the file and reset values after the download.
• What if there is a request of multiple files downloads? You can process it by allowing user to make only one download request at a time.
• On Capistrano deployment process, you may have to restart the delayed job process and make sure no download is in progress. If not handled – The Ajax call may get into loop and request may be endless, counter value can be used to fix this and kill the request.

Want to know more on how this and other gems function? With ingenious solutions, progressive business values, and proven track record, RailsCarma is best suited to help you with all your development needs. Our development process is established heavily on unrelenting attention to details, including quality, design and excellence. We convert your ideas into reality. If you have any queries, you may contact us.

• Implementing TinyMCE in Ruby on Rails
• Understanding Asset Pipeline Plugin
• A Simple Way To Increase The Performance Of Your Rails App
• Scaling Applications with Multiple Database Connection