A/B testing small variables like this matters, but far less than most people think, and you can probably scale back. It's easy to get trapped in a cycle of testing minor subject line tweaks or button colors, seeking a few points of lift on a single campaign. While data is good, this is often a low-leverage activity with diminishing returns. The truth is, a better offer will always beat a better subject line. Furthermore, most modern email and SMS platforms are already quite good at handling send-time optimization, so your manual tests are unlikely to produce a significant breakthrough.
The place where testing is incredibly valuable is on big, strategic questions. Instead of testing one subject line against another, use testing to answer fundamental questions about your business and your Email Marketing Strategy. For example, Chase Dimond on Send It! talked about running a fascinating test comparing sending two emails a week versus five. That's a test with huge strategic implications for revenue and customer engagement. As Clint Smith mentioned on Ecommerce Conversations, the goal is to move beyond basic blasts. Test a 15% discount versus a free shipping offer. Test a long-form educational email against a short, direct-to-product one. This kind of testing provides real insight that can change your entire approach, which is a much better use of your time than finding a fractional winner on a single automation email.