Big Tests vs Incremental Improvements on eCommerce Stores

What are D2C Brands Missing Out on Today?

Kausambi: Yeah. So it's not just John, the expert, but John the author, and we'll be putting in the link to the book, in our live stream, as well as in our posts later on. Um, so guys watch out for that. (Link to book: https://thegood.com/books/)

So John, before we jump into all these details about CRO and what's right, what's wrong, I read in one of your blogs - there was this example of improving 10% of your site, leading to 90% increase in performance. Right. And that's really caught my eye. Are B2C brands and e-commerce teams missing out something on hygiene's today?

Jon: You know, it's interesting. Most people believe that you have to completely overhaul your entire site to improve conversions. And we see it all the time when brands come to us, where they have just burned down their old site and built a brand new one, and they thought that was going to solve all of their problems.

But in reality, what you really need to be focusing on is just improving the main path to purchase at the beginning. That is a great place to start and you can find so many improvements to be done. And you know what the, the blog posts you're referring to, um, actually references something called Zipf's law - Zipf was a professor, of actually linguistics of writing.

And what he found was that it's, especially in the English language, that there are 10% of the words that we use all the time. And then there's 90% of words that are very rarely ever used and he could catalog what those 10% of words were. And  he found that most people believed that writing was better if you took those 10% of words and improved them. Basically changing them into that remainder 90%. So taking simple words and not making them a complicated, but making them more interesting than people believe that the writing was more readable, more engaging. And you can do the same thing with your website.

What I mean by this is focus on. The main entry points. So the, the, you know, your home page, maybe your landing pages and review the experience there, and then focus on that path to purchase from landing page all the way through checkout. Most brands focus on the other 90% of the site. Were those of course are important to them, but the other 90% is that they focus on all the other branded pages.

They're focusing on their blog. They're focusing on pages that aren't in that path to purchase about us. Uh, you know, all these things that consumers aren't as interested in understanding, but brands really want to communicate. If you focus on the main 10% and make that more engaging, more interesting and easier to convert the rest of the site will see improvements and you will reach your goals without having to start over and redo 100% of your site, which is really where most brands seem to start.

Uh, the reality is conversion rate optimization (CRO) is all about incremental improvements.

CRO is all about incremental improvements. - Jon MacDonald

It's all about small changes made over time, where you keep learning and then reapplying those learnings. So you learn, make those changes, run some more tests and try to learn some more, make some more changes. And they start to stack over time where you start to see large and large improvements, but it all starts with making small improvements.

So I always tell folks, focus on that 10%. Make 10% improvements. Don't worry about doing everything at once and you will see success.

Make 10% improvements. Don't worry about doing everything at once, and you will see success.

Kausambi: That's interesting that you're talking about and it actually hits very close to home. When you say don't worry about the, about MES or, you know, whatever else that's there, but your baseline, your bread and butter flow is essentially what you can we get a lot of ROI out of, so even there is there some way to look at it, with the perspective of, do I go top down? Which means do I go home page and then collection and PDP and checkout? Or do I go bottom up? How, how is there some famous to think about it too?

Taking a Top-Down Approach

Jon: Well, the, for the vast majority of brands you're going to want to go top down and that's because.

You want to focus on that path to purchase and less people are going to get to the checkout. So for instance, if you have problems in that flow where maybe on your product detail page, a lot of people are dropping and you see a huge drop off rate there. Then it doesn't make sense to really focus on the cart yet because nobody's getting there.

Very few people are getting to the cart. Now there's also the issue of math that comes up. And what do I mean by that? Well, if you are doing A/B testing, you need, in theory, you would want statistical significance, which means you'd want enough traffic to see and interact with that test to be able to, with reasonable certainty say that test is, uh, performing well.

And that test is a winner or N shows that it needs to be rerun and with a different hypothesis, whatever it might be. But the reality here is always focused top down because that's where the majority of your traffic is. Now you're going to have the, you know, if you look at the funnel, right, you're going to have the most amount of traffic at the top of the funnel.

So really you're going to have the biggest impact there. And most brands don't have enough traffic to focus on checkout right out of the gate. A lot of brands come to us and say, we want to. Improve our cart, abandonment, our cart abandonment is super high. And our first response is usually not to look at the cart.

You know, we may take a glance at it, but if they're using Shopify or any of the other large platforms, often there isn't much, you can change in the checkout or. it's um, only available on the enterprise level plans like Shopify plus, right? If you're just on normal Shopify, you can't really do a whole lot with the checkout.

So it's not worth the effort spent there to where, you know, if you are looking at your home page and into your category and product detail pages, we're able to find where most people are dropping off and understand why. And then usually what we find through things like user testing, where we actually are sending people to the site who match an ideal customer profile, and we're asking them to complete some tasks while we record their screen and their audio.

So we're able to really hear from consumers what their expectations are, what do they wish was happening? What questions do they have that aren't being answered and that type of data will really quickly tell us what expectations are not being set in the cart.

So when we look at cart abandonment, for instance, it's usually because some expectation, uh, was not set before the cart. Very rarely do people abandoned a cart because they just decide they're not going to buy. Usually what happens is they encounter something like an, a very large shipping cost. Or maybe the product won't be able to ship for three months and they're saying, you know, I don't want to wait.

I'm just not going to order it today. But it's new information that comes up in the cart that should have been communicated higher up in the funnel on the product detail page, someplace else, along that journey. And so usually most issues in the cart are not because of the cart, they are because of something that came before the cart that was not properly communicated.

Kausambi: That's interesting because I remember when Oliver was on last week, we were talking, he also mentioned something around doing some cart abandonment experiment, just, putting up the shipping timelines up front, got a lot of people to drop off.

And then when they start removing it in a way, because it wasn't a published book. And then that had to be pre-ordered to ship. When people realized that it was like three weeks away, they would just drop off! So it connects to what he mentioned.

Jon: Yeah!

Setting expectations is a large part of the challenge and so many brands who are afraid to communicate things like extra long timelines, because they're afraid people aren't going to convert, but they're really just pushing that challenge to the cart and then they blame the cart.

And so you're better off communicating often and early.

Does Redesigning Help Improve Conversions?

Kausambi: Yeah. That's fair. Audience, if you have any question, quick heads up, please drop it in the chat.

So back to John, so when does it make sense to actually redesign and when does it make sense to really not look at redesign at all?

Jon: So typically when it makes sense to redesign is. The most often we see that can be successful is when brands are rebranding. What do I mean by that? Well, maybe they are completely changing the look and feel of the brand. And at that point, you need to redesign the website to match that new look and feel.

So in that point, it may make sense to say, Hey, you know what? We're not going to give the site just a fresh coat of paint. We're not going to update the colors, the font, whatever. We're also going to look at the user experience because at that point you're already spending good money to redesign the website.

You might as well look at the user experience at that point. You know, the other thing is. There's generally a shelf life with websites. And what I mean by that is they only typically last certain amount of time, because more often than not the technology. Right. So if you think about it, most platforms aren't going to last more than three, four years. So it's not uncommon to have to replatform your site. You know, Shopify right now is going into a 2.0 theme and a lot of brands are moving over to that and it's causing them to rethink, what templates they're using. Right? And maybe the consumer experience, or if you're moving from, let's say big commerce to Shopify.

That's a great time because you already have to rebuild your site for, to meet Shopify template. So at that point, maybe it's a good time to look at the at the user experience as a whole. It's a good opportunity.

When is it a bad time to redesign is just because. What I mean by that is so many brands bring on a new e-commerce manager or a new CMO, and they say I want to put my fingerprints on this.

So I'm going to redesign the site. And the challenge with that is usually when a new team member comes in, they don't have any context or history for why things have been done the way they are. So all those learnings just go out of the window. So that's often when brands come to us is when, you know, somebody has been brought in to improve the site and improve the revenue from their online store.

And the first thing they do is redesign and it doesn't work like they were hoping. So then they, we, they end up coming to us and saying, Something's not right here. And then we start asking questions and they're usually why questions? Well, why did you make that decision? Why did you redesign, why do you have this functionality or this page?

And nine times out of 10, the answer is we thought it would be better. So essentially they were guessing. It wasn't based on data. I often say this, but it's really hard to read the label from inside the jar. So, you know, brands are so close to their site. They're so close to their products. They know everything about them, right.

Brands are so close to their site and products... they miss providing the learning and education required for a new customer. They think they know best, but they haven't talked to their customers yet. - Jon

They know what sizes their products come in, what colors, what categories, what they're good for, but a new consumers, someone who just clicked on an ad and arrives at that, they don't they are missing all of that learning and education and context. So you really need to provide that. And that's where it becomes more and more of an issue because brands, typically, think they know best, but they haven't talked to their consumers yet.

And that's what we really advocate for is bringing that consumer voice, helping you understand what the consumer's looking for. That is when you get a much better. Yeah.

How to Set Up Optimizations for Your Store

Kausambi: So then if you need to bring in the consumer experience and there definitely need to, what would be the kind of like a framework in a way of setting up these optimizations, which gives you the learning to eventually take a call on redesign or not?

Jon: Right. So first thing I would do is start talking to consumers. So there's two types of consumers. You should talk with pretty obvious. Talk to those who just perfect. Pretty easy. Right? You get reviews anyways. Well, pick up the phone, give them a call or send them an email. Hey, can you spend five minutes on the phone with me?

I just have a couple of questions, no pressure and all, you know, give them a gift card or a coupon code or something. Right. It's the, thank you.

The reality there with that subset of users is you understand - Why did you purchase, did you have any questions that weren't answered along the journey? And what were some of the concerns you had that didn't get addressed, et cetera.

So all of those things, um, are great questions to understand why consumers bought.

On the flip side, retargeting with, most people think retargeting ads should be used to get people to come back to the site. I prefer an a great research hack is to run a retargeting ad for someone who did not purchase and, and say in the ad, I want to know why you didn't buy and then click here and take our survey, and we'll give you some type of free gift or with purchase or something that to entice them to actually complete the purchase, but not a huge discount.

Discounts are not sustainable. - Jon

Typically I'm not a discount fan. I think that's not sustainable if we've got a whole chapter in the book about that, but the reality is, you really want to help understand why they dropped off what expectation was not met, what questions weren't answered, all of that will really help you to understand the consumers' challenges. And once you start talking to the consumers, which just so many brands just don't do and it's, oh, it's really sounds simple, but it is. It's just talk to the consumers.

And once you do that, you will find very quickly the reasons why consumers aren't buying and then you can go plug those holes and you can decide at that point, oh, you know what? This is a technology problem. I need to just replatform. And at that point, maybe it's a good time to redesign, or you might find that it's a content issue.

They're having questions that aren't answered or they're confused about something. And then you can make small tweaks to the site that will really over time add up to some large improvements without needing a redesign.

Kausambi: Yeah, that's a great framework to start. And I think it applies to SAAS businesses too. It's not just a e-commerce.

I think everybody should definitely pick up the calls, pick up the phones and get those calls going.

I want to quickly dive into a couple of questions from the audience. Uh, I do have a few more, but making sure that we answered the questions that our audience. This is from our YouTube livestream.

What to Prioritize When Running Tests

Question: When you're running a small test, how do you know which test to prioritize first? Is there a framework for it and how frequently should we be running these experiments? (Joe)

Jon: Great question, Joe. So is there a framework, there are tons of frameworks out there to help you prioritize tests.

Um, it just Google it you'll find that there's a bunch out there. Every agency kind of like the good has their own preferences on that. We have a formula as well, and really the key driver on that is return on investment. What is the return on investment we are going to get from running a test. So what does that mean?

Which test is going to help improve revenue the most, or whatever metric we're set out to improve.

But if you're running really small tests, generally they're not going to move the needle, you know, a very large amount unless you have a lot of traffic, right? So most brands are too small to be running small tests.

So. The first thing Joe, I would say is if you don't have about a hundred thousand sessions per month happening on your site, probably shouldn't be running small tests because it's just going to take too long to prove those out.

You're not going to learn as much. You're going to want to stop the test before it reaches statistical significance.

And then you're probably just going to have more informed guesses happening. So. My real feedback here is to run large tests, focus on full page redesigns or, you know, in terms of changing content, around moving content on the page, maybe rewriting all the copy on a page. Those are things that are larger tests.

And when I think of a smaller test, I think of, you know, uh, maybe changing the call to action and that's it on the. Right. That's a little too small because you're just not going to know if that really had an impact unless you have enough traffic. So it, you know, I like where the question is going, Joe, and yes, there are lots of frameworks and I suggest you do a little bit of research on those and you can find some probably not.

It would take me into the other 10 minutes. We have left today to walk you through them. So I'm not going to deep dive into that, but I will say. There are some great frameworks out there. There's a company called Wider Funnel has a really great framework. I know CXL has them, we have our own. But the reality here is, um, unless you have a lot of traffic focus on bigger tests and you'll see better results then.

Kausambi: I hope that answers your questions. Um, so this is from Katherine.

User Testing to Reduce Churn

Question: Shoppers drop off at the slightest inconvenience. Is there a metric to know what's causing friction or what's holding them up?

Jon: Yeah. The short answer is user testing is going to be your best bet. So that's what I was talking about earlier.

User testing is really going to help you understand their perspective and get inside their head. And that's going to be important because, running an A/B test will tell you the outcome, but it won't tell you the Why.

An A/B test will you the outcome, but it won't tell you the Why

And so you really need that more qualitative to understand the why, uh, the quantitative data like analytics, AB testing, et cetera will tell you what but it won't tell you the reasoning behind it.

So, Catherine, I really recommend you start interviewing customers. If you're a smaller site, start interviewing them right away. You can just even just go to a coffee shop, take your laptop and offer to buy somebody a coffee.

And while they're waiting for that coffee to be made, just to ask them if they'll navigate through your website and talk out loud about the experience they're having, what questions they have and give them simple tasks to do say - 'Hey, you're buying this product. Go find the best variant for you, add it to the cart and watch them go through that process.

And you'll learn so much that way.

Kausambi: Yeah. Their reactions are gold! This, this question is interesting.

What Kind of Experiments Should You Run?

Question: So what are the key differences in experimentation that a younger brand needs to be thinking about versus the larger, and also what kind of tools do you suggest for A/B for someone on a headless deck?

Jon: So Barry, two parts here.

So for the first, what are the key differences and experiment? The younger brands. It's almost always going to be a traffic challenge for younger brands. But let's just say you have the same amount of traffic as some larger brand. Maybe you have a large amount of cash that you're just spending to drive traffic.

The biggest issues for younger (ecommerce) brands is twofold - trust and social proof. - Jon

The biggest issue for a younger brand is going to be twofold. It's usually going to be 'cause, they don't know who you are and it's going to be social proof because you have not had time to collect all of this social proof. So typically younger brands have a trust issue that needs to be overcome and a lack of a brand awareness where, larger stores that are been around for longer generally - maybe you've heard of them, maybe you know friends who have the products, and you already trust the products and they've, you know, you were referred by a friend. So there's less of those barriers to overcome. The good news is it gets easier the longer you're in business, because you have more and more of that trust built up more and more natural ambassadors out there, people who have bought your products and are sharing it or wearing it for instance, and that's, uh, um, social proof in of itself.

So that's generally the biggest difference. What kind of tools do you suggest for A/B and someone on the headless stack?

Well, this has actually gotten a lot easier over the years.

It used to be near impossible, just because most of the brands, um, or most of the tool sets didn't really work extremely well. What we have done at the good, um, is actually build our own framework, um, that we are using, uh, Google Optimize. And then we're not using the, what you see is what you get editor.

We're actually running the test, um, with, through our own code base, using Optimize to keep track of all the data, and actually, launch the test and segment the audience. So we've really separated the logic from the segmentation, if you will. So Google Optimize helps us.

Choose the audience that sees the test based on their data and, keeps track of the data of the test because we're feeding the info back up into Google Optimize. So, um, I know Barry, that's probably not as technical of an answer as maybe we're looking for, but, unfortunately the only other option that I've seen work extremely well is Optimizely and Optimizely is very expensive!

It's an enterprise level tool, but, they ha they solve problems like this on a daily basis with their tool. So it's much easier to use. The problem is 99% of brands either can't afford or won't spend the money that's needed for Optimizely.

So that's not what we've chosen to build around for that.

Kausambi: I have one final question before we want to get into a little bit about John's book.

How to Improve Purchase Frequency on Your Store

Question: What do you think would be some of the effective ways to improve purchase frequency through repeat customers? (Ankit)

Jon: Great question. You know, there's really  about as far as we take it at the good cause we mainly focus on that initial  purchase.

We do some optimization of recurring purchase. So helping optimize customer portals, things of that sort for subscription customers.

But at the end of that first purchase, there's really five emails that should go out. Five communications that should be sent. Most brands aren't doing all five of these.

So if you just do all five of these, you're going to be. Really well off and on a great path for getting repeat purchases.

The first is order confirmation. Obviously you want it to reassure somebody that the order was placed correctly, that they're going to get the right product. They were charged appropriately, et cetera.

The second is a shipping update. You really just need to tell them when the product ships, so you're setting expectations around timing, when they can expect that, how they can track it.

The third is what I like to call education email. So if you have a product that requires any bit of education, um, even I bought a bodywash brand not too long ago, that did a great job by sending me something and saying, Hey, here's how you use our product. Yeah. It's bodywash! But like, you know, they included some moisturizer and some other stuff in there and they say, Hey, like, this is how you use this product. We want you to know.

And that type of educational content does two things - it bridges the gap in that shipping time where people are just waiting for the product and they might have a little bit of maybe regret that they just spent money buying a product that they can't use yet.

Or they're just tired of waiting. Right. So now you've given them something to kind of keep the brand fresh.

The fourth one there that I always like is really a follow up email about a week after the product arrives. And most people just immediately send a review email and say, leave a review.

What I would recommend here is instead of that review email, just a simple founder email, (all text) that says - Hey, I really would appreciate any feedback on your experience. You be shocked what people tell you. And then if somebody says - Hey, I had a really great experience, five stars all around say - Great! Would you click here, and just to say that on, on Google shopping or a review platform? Most people will do that.

The fifth email is just put them on your marketing list to keep in touch with them and segment them appropriately. So if you're using something like Klaviyo, et cetera, make sure you segment them as a prior customer so that now the messaging can be tailored according to that.

Because I can tell you nobody likes when they purchased. And then all of a sudden you run a sale and you send an email, but you say it's for first-time customers only. And I just spent more money than what I could have spent. And everyone's getting a better deal. So you want to make sure you're segmenting out your audience there.

Kausambi: All right. We have a hard stop, but we have gone really beyond the time!

But before we end, my favorite part - a little bit about the book and where can folks get the private.

Jon: Yeah. So if you just go to thegood.com/books, um, it will take you to our books. (Link to book: https://thegood.com/books/)

And if you use code - MA S O N you can get a free copy of the ebook. So a free PDF copy of the book. So it's thegood.com/books. Highly recommend you check them out and, um, also feel free to email me if you have any questions. Email: jon@thegood.com.

Kausambi: And thank you so much, John. This was amazing folks. If you have more questions, you know where to get John and definitely grab a hold of the book, maybe most of them will be answered.

And that's a wrap for today. Thank you so much everybody. And thank you, John.

Jon: Of course. Thanks for having me. I look forward to doing this again sometime!