This article describes setting up dynamic robots.txt for your Rails app. You can use this for your staging environment or even for a multi-site app. https://solidfoundationwebdev.com/blog/posts/how-to-create-a-dynamic-robots-txt-in-rails