There's a gem for that: https://github.com/Shopify/maintenance_tasks