RubyFlow : The Ruby Community Blog

Home   Submit   Sign Up   Log In   leaders   Twitter   RSS Feed  
 

How to import millions CSV rows via ActiveRecord in minutes (not hours)?

Rails is know to suffer GC performance issue in importing millions records via AR. Traditionally, people opt to use SQL INSERT(s) however this method does not respect AR validations and callbacks which could lead to poor data integrity. The best way is to use AR but make it run through multiple processes to boost up runtime. Details can be found on http://ruby-journal.com/how-to-import-millions-records-via-activerecord-within-minutes-not-hours/

Comments

Post a Comment

Comment abilities for non registered users are currently deactivated, pending time to add a proper CAPTCHA to solve the escalating spam problem. Sorry!