HireFire - The Heroku Worker Manager
Just pushed out a new gem called HireFire (GitHub link)! HireFire automatically “hires” and “fires” (aka “scales”) Delayed Job (and soon Resque!) workers on Heroku. When there are no queued jobs, HireFire will fire (shut down) all workers. If there are queued jobs, then it’ll hire (spin up) workers. The amount of workers that get hired depends on the amount of queued jobs (the ratio can be configured by you).
HireFire is great for high, mid and low traffic applications. It can save you money by only hiring workers when there are pending jobs, and then firing them again once all the jobs have been processed. It’s also capable to dramatically reducing processing time by automatically hiring more workers when the queue size increases.
Comments
Wow, very nice idea. This will save me a lot of bucks. Thank you so much.
This looks great. I look forward to digging into the code to see if it is suitable for my site. I actually implemented something similar to this for a while but due to race conditions it was not reliable enough. I couldn’t justify the time to solve the race conditions so I just removed the code and paid the full working price. But hopefully you will be saving me $36/month.
Awesome work!. I just implemented my custom solution for this – in “just” i mean yesterday!
@eric Great! Let me know how it goes. If it isn’t suitable, please let me know why, might be able to improve it in certain ways so that it’ll cover more use-cases if needed. Trying to create a “standard library” because there are a few “hacked-up in a night” versions lying around, but nothing really solid, maintained, documented from what I can see.
@jondot Thanks!
how can i verify if this is actually running on heroku. i’ve checked the logs and i dont see anything related to hirefire. using dj plugin and hirefire gem. thx
Post a comment