Streaming CSV Download from Rails 3.2 app

Question

I am desperate to get a streaming CSV download working in my Rails 3.2.2 app.

I have tried the 'csv_builder' gem (https://github.com/dasil003/csv_builder), which advertises support for this feature, however it seems that there were some changes in Rails 3.2 that are keeping it from working (it produces an 'uninitialized constant ActionView::Template::Handler' error on app startup).

Any other ideas or solutions out there? Thanks!

EDIT: To clarify, I am needing to export all entries of a model as a CSV file. There are so many rows, that it is timing out... therefore the need for streaming. I have used the Comma gem (https://github.com/crafterm/comma) for this in the past, but it doesn't support streaming either, at the moment.

Answer 1

OK, after a bit more research I hacked together the following in my controller. It'll stream if response_body is given something enumeratable (is that a word?). Also, the server needs to be able to stream (I am using Unicorn on Heroku). I'd like very much to not have all this stuff in the controller, so my next step is to extract it out somehow.

  format.csv {
    @entries = Entry.all
    @columns = ["First Name", "Last Name"].to_csv
    @filename = "entries-#{Date.today.to_s(:db)}"

    self.response.headers["Content-Type"] ||= 'text/csv'
    self.response.headers["Content-Disposition"] = "attachment; filename=#{@filename}"
    self.response.headers["Content-Transfer-Encoding"] = "binary"

    self.response_body = Enumerator.new do |y|
      @entries.each_with_index do |entry, i|
        if i == 0
          y << @columns
        end
        y << [entry.first_name, entry.last_name].to_csv
      end
    end
  }

Answer 2

The approach that I took with a Rails 2.3.8 app was to spawn a new thread to handle the csv parsing and then use an AJAX call to check the server to see if the file was ready (I relied on File.mtime).

Just ripped it out of the app to post here so I've removed alot of the csv parsing code, and haven't included all of the views

sorry end of the day rush :D

controllers/exports_controller.rb

    class ExportsController < ApplicationController
            require 'fastercsv'
            require 'generic_agent'
            require 'generic_record'

        def listing

          @this_filepath = "../html/whatever/" << Time.now.strftime("%I:%M:%S_%d:%m:%y") << ".csv"

          @spawn_id = spawn(:nice => 1) do

            FasterCSV.open(@this_filepath, "w") do |csv|

              csv << [ "outbreak_id"]
            end

          end

          render :update do |page|
            page.replace_html 'export_status', :partial => 'export_status_partial'
          end

        end

  def send_export

    @this_filepath = params[:with]
    csv_file = File.open(@this_filepath.to_s, 'r')

    csv_string = ""
    csv_file.each_line do |line|
      csv_string << line
    end

   send_data csv_string, :filename => "export.csv",
                :type => 'text/csv; charset=iso-8859-1; header=present',
                :disposition => "attachment; filename=export.csv"
                #send_file @this_filepath.to_s, :stream => false, :type=>"text/csv", :x_sendfile=>true

                #send_data csv_string, :filename => export.csv

                #File.delete(@this_filepath.to_s)
  end

  def export_checker
    filename_array = params['filename'].split(/\//)
                @file_found = 0
                @file_ready = 0

                @file_size = File.size(params['filename'])
                @this_filepath = params['filename']

                if File.exists?(params['filename'])
                release_time = Time.now - 5.seconds
                if File.mtime(params['filename']).utc < release_time.utc

                @file_found = 1
                @file_ready = 1
                @file_access_time = File.mtime(params['filename'])
                @file_release_time = release_time
                @file_size = File.size(params['filename'])

                else
                @file_found = 1
                @file_ready = 0
                @file_size = File.size(params['filename'])

                end

                else

                @file_found = 0
                @file_ready = 0
                @file_size = File.size(params['filename'])

                end

    render :action => "export_checker"
  end
end

views/exports/export_checker.rjs

if @file_found == 1 && @file_ready == 1 && @file_size > 0


page.replace_html 'link_to_file', :partial => "export_ready"
if @file_release_time
page.replace_html 'notice', "<div>Completed #{@file_release_time.strftime("%I:%M:%S %A %d %B %Y")} :: file size #{@file_size.to_s}</div>"
end

page.visual_effect :highlight, 'link_to_file', :endcolor => '#D3EDAB'

elsif @file_found == 1
page.replace_html 'link_to_file', "<div> File found, but still being constructed.</div><div>#{@this_filepath.to_s}</div>"
page.visual_effect :highlight, 'link_to_file', :endcolor => '#FF9900'
else
page.replace_html 'link_to_file', "<div> File not found @file_found #{@file_found.to_s} @file_ready #{@file_ready.to_s}</div>"
page.visual_effect :highlight, 'link_to_file', :endcolor => '#FF0000'
end