In this guide to smoke testing, we cover the different uses of the term, and how you can use this strategy to validate your SaaS product before you build it.
New SaaS entrepreneurs are tasked with an important project: make sure people actually want to buy their software before they build it. Building a SaaS MVP can range from $50,000 to $90,000 when developing a true MVP.
Unfortunately, SaaS entrepreneurs often get roped into working with agencies that specialize in building apps for established organizations.
In this case, these new startups end up sending $200,000 to $1,000,000 to build a launchable product, because the agency has convinced them to launch with more features than needed. Or the agency hasn’t talked the entrepreneur off the ledge of feature overwhelm.
In order to save money, SaaS entrepreneurs do best when taking some of the effort into their own hands. They can smoke test their product idea, or work with a company who offers smoke testing as a service before committing to building the entire product.
What is smoke testing?
In software, smoke testing can refer to two different things. For QA, it means an initial test of critical functionality prior to in-depth QA. And in product development, it refers to building a landing page for a product concept to validate it. This is done to test the ease of acquiring waitlist sign ups before actually developing the app.
In either case, smoke testing refers to an initial test before moving onto more time-consuming and costly efforts.
What is smoke testing for software quality assurance?
The term smoke testing is commonly used in the quality assurance world. Smoke testing means QAing the most critical functions of a software build right after it is built in order to determine if the build should move on to other forms of testing, such as functional testing, exploratory testing, regression testing, automated testing, etc.
Think of a smoke test as a smoke signal. Does the app immediately reveal major bugs? Or do the critical features work? If they do, then it’s time to hand the build over to a manual tester so they can dive deep and try to break it.
Smoke testing is never an adequate method of quality assurance all on it’s own. Rather, it’s the first state gate on the path to in-depth testing that companies need to execute on in order to ship quality products and updates to customers.
What is smoke testing for product validation?
When it comes to product validation, smoke testing is used to check market demand before building a product.
You might be wondering how this differs from a design sprint.
Here’s the main difference between smoke testing and a design sprint:
Smoke testing - Marketing your product concept before it is built to verify market demand. This is done at the end of a design sprint process in order to further validate the concept and ease of marketing it before investing money into the product and its go to market strategy.
Design sprint - One or two week process to come up with a winning product concept to a problem. This process includes building prototypes, conducting user testing, and reiterating on your prototype. At the end, smoke testing may be done to check the market demand and ease of acquiring waitlist sign ups.
As you can see, smoke testing is not the same thing as a design sprint.
While a design sprint validates the product and its target audience, smoke testing accepts that validation and moves on to validate the product’s marketing and messaging strategy.
How to do smoke testing for SaaS product validation
While smoke testing is very different from a design sprint, you can’t hop straight into smoke testing.
You shouldn’t attempt to validate the marketing of a product before you’ve validated the product itself. That’s why we’re including some elements from a design sprint in this step-by-step process as well.
1. Understand the problem you’re trying to solve
The first step is to deeply understand the problem that you’re trying to solve. This is often easier if you have experienced the problem first hand. If not, you’ll need target users to interview and ideally shadow on the job.
During our design sprints with clients, we like to map things out with sticky notes. We cover the target users, what they’re currently doing, the direct and indirect competitors, and other elements that address the entire landscape of the problem.
2. Come up with a solution
Next, you need to come up with a solution. Usually, this looks like coming up with 2 to 5 different solutions, each with a different (and possibly overlapping) feature set.
As a team, you need to work together to pick the best solution. Which one addresses customer needs in the most lightweight, efficient, and affordable way?
3. Make a low-fidelity prototype
With the right product concept, it’s time to make a low-fidelity prototype. This might be nothing more than a drawing on a piece of paper, a digital mockup, a PDF, or a slideshow.
When we did our first demos with prospective customers to validate the product, it wasn’t even built yet. In fact, our “product” was a PowerPoint deck with a UI and hyperlinks that jumped from one slide to the next. - Garrett Moon, founder of CoSchedule
Your low-fidelity prototype shouldn’t be fancy, but it should clearly communicate what features would be included in the MVP, and what jobs or tasks those features would achieve. You can use the Jobs to Be Done framework to help communicate these.
4. Conduct user testing to get initial feedback
Next up, you’ll want to conduct user testing to get feedback. Show your low-fidelity prototype to real target users and listen to what they have to say.
They might say that your initial concept is missing an important feature, or that what it solves isn’t as important as a much bigger problem they are dealing with.
Come in with an open mind and no assumptions so you can address the most profitable problem possible.
5. Reiterate and create a high-fidelity prototype
With feedback to guide your way, you need to reiterate on your prototype. Make a high-fidelity version that looks and feels like your final product will. It should have the core UX of the product, even if clicking on buttons doesn’t trigger any server calls.
The point is to have something that users can play around with and that you can screenshot for your smoke test.
At this stage, you’ll want to validate this high-fidelity prototype with further user testing.
6. Make a landing page for your SaaS product
Next, it’s time to make a landing page for your product. This is where you’ll send traffic using various marketing strategies.
Here’s an example of a waitlist landing page we built for our new SaaS product RaiseIQ. Make sure to include images of your high-fidelity prototype, and also add quality, engaging copy that hits at the heart of the problem.
7. Get waitlist signups (market before you build)
Here are some ways you can drive traffic to this landing page:
SEO blogging
LinkedIn posts
LinkedIn articles
LinkedIn ads
LinkedIn outreach
Facebook ads
YouTube ads
Cold email
Cold calling
Direct mail
You’ll have to pick the right strategies for your target audience and price point.
We started blogging as a company before we even had a product...as a way of validating early on if the problem we were trying to solve for marketers actually existed - Jennifer Pepper, Marketing Manager at Unbounce.
8. Evaluate the your marketing, demand, and commitment level
Make sure to set up Google Analytics tracking before you begin marketing your SaaS so you can see which channels are referring the most traffic to your website. Direct traffic will likely be coming from your cold outreach campaigns.
You should check which marketing campaigns worked the best. Also, your conversion rate should be at least 25% since they are only signing up for your waitlist.
Not only do you need to validate marketing campaigns and demand, but you also need to take an honest look at your own commitment level.
Do you have what it takes to commit to this business? Is the problem you’re solving important enough? Are you passionate about it?
The great thing about smoke testing is that it helps you validate yourself as a founder of this particular business, not just your new product.
Should you work with a smoke testing company?
You can hire DevSquad to partner with you on:
Design sprint - Our 10-day design sprint process is designed to validate a true MVP using real target users.
Smoke testing - We’ll build out your landing page for you featuring screenshots of your validated high-fidelity prototype.
MVP development - We build a real MVP (as small as possible) in 3 to 9 months, using our pay-as-you-go monthly development model. We can also offer UX design alone, if you have backend developers on hand.
Smoke testing carries your product validation a step further. Without great marketing, a product can’t succeed. Just because people say your product would help them with their problem, doesn’t mean they would actually pay for it.
Use smoke testing to get as close to product-market fit as possible before you actually invest money in building a software product.
DevSquad excels at building brand new SaaS products. Learn more about DevSquad.