The Truth About Website Speed Tests: Do You Really Need a Perfect Score?

When it comes to optimizing your website's performance, there are few things more frustrating than going after that perfect 100 score in tools like Google PageSpeed Insights.

But here's the thing: Getting a perfect score isn't what matters the most. Instead, you should take a broader look at the real-world load times for your actual human visitors.

While speed scores can still be useful for optimizing your site, these real-world load times are the only thing that really matters for your site (and for your users).

In this post, we'll dig deeper into this topic to help you understand everything you need to know about speed scores.

Table of Contents

• What is a speed test?
• What are website speed testers?
• Do you need a perfect speed score?
• Are your speed-testing tools legit?

What is a speed test?

A speed test helps you understand your website's performance. Typically, it provides you with information on how quickly your website loads as well as some type of "performance score" that aims to summarize your site's performance in one metric.

You can use this information for two main purposes.

1. You can understand how quickly your website loads for visitors and what type of experience you're creating for them.
2. You can discover areas where you could improve your site's performance.

Most speed tests are "synthetic" tests.

Website speed tests typically use a "synthetic testing" or "lab testing" approach. This means that the tool is simulating a real human visitor, rather than looking at data from actual human visitors.

Understanding the synthetic nature of speed tests is important.

Why? Because how the tool sets up its synthetic testing environment will affect the results that it gets.

Here are the three most common testing environment configurations that can affect results.

Connection Speed

The connection speed of the test will affect the results. For example, whether the test is simulated using an ultra-fast wired connection or a slow 3G connection.

Physical Location

The physical distance between the test site and your website's server will affect the results. Let’s say your site is hosted in a U.S. data center. A speed test run from California will have faster results than a speed test run from Japan.

Device

The computing power of the testing device can affect the results, especially for sites that use a lot of JavaScript. A high-powered desktop computer will process/execute JavaScript much faster than a low-end smartphone. This means that the site will load faster in the desktop environment.

All of these configuration choices can affect the results of a speed test, and therefore your site's performance score.

What are website speed testers?

Website speed testers are tools that help you run speed tests. Typically, you plug in your site's URL. The speed tester runs its tests and provides you with results.

Image Source

Here are some of the most popular website speed testers.

• Google PageSpeed Insights
• Pingdom Tools
• GTmetrix
• WebPageTest

• IsitWP Website Speed Test

Many speed tester tools will give you one overall "performance" score. However, most tools also give you more detailed performance metrics for that page, though you might need to browse around to find them.

For example, let's look at PageSpeed Insights, one of the most popular website speed testers.

When you plug in a page on your site, PageSpeed Insights will give you a single performance score for desktop and mobile (a separate score for each).

However, in addition to that score, you'll also see more real-world metrics such as Largest Contentful Paint.

Image Source

Typically, the performance score is based on a set of testing criteria, whereas real-world metrics give you a better idea of how your site performs for actual human visitors.

How does the website speed tester collect data?

Each speed test tool will have its own way of collecting data.

However, nowadays many tools use Google's Lighthouse engine as part or all of their underlying testing engine. This means that different tools can use the same performance scoring method, even if it might not be obvious at first.

Lighthouse, which is what the popular PageSpeed Insights tool is based on, is an open-source tool for testing performance (along with other areas such as accessibility and SEO).

Lighthouse runs a number of performance-based tests against the page and then comes back with an overall score. It also provides tips for how a page's performance can be improved.

If a tool uses Lighthouse, you'll typically see four separate scores:

1. Performance.
2. Accessibility.
3. Best Practices.
4. SEO.

However, some tools will only display the performance score.

Beyond Lighthouse, some tools will use their own test system. They often only display the actual load time metrics or use their own scoring system.

Do you need a perfect speed test score?

Have you ever visited a website and said to yourself "wow, I bet that this website has a really good performance score?”

Of course not! And neither have your visitors.

That's because your visitors care about how quickly they're able to achieve their goals on your site. Those "goals" could be reading the content in a blog post, purchasing a product, performing a search, and so on.

The best way to see whether or not your site is providing a good experience to your visitors is the actual load time metrics of your site — not the abstract performance score.

Here are some actual load metrics that will have a direct effect on your visitors' user experiences.

Time to First Byte (TTFB)

This measures how long it takes for your hosting server to respond with the first byte of data. Basically, this is how long until a user starts seeing content loading from your site.

Largest Contentful Paint (LCP)

This measures how long it takes your "main" content to load. It's a good user experience metric because this is usually when visitors can start getting value from your site.

It's also part of Google's Core Web Vitals metrics, which are an SEO ranking factor since the Page Experience update.

Fully Loaded Time

This measures how long it takes your site to completely load. It's not quite as important as a more user experience-focused metric like LCP, but it's still helpful to understand how your content is loading for visitors.

Depending on your site's configuration and hosting, it's possible to do very well in these real load metrics without getting a perfect score in your speed test tool.

And that's ok! It's better to have very fast performance metrics and low-performance scores than the opposite.

Case Study: Piece of Yum

For example, consider Pinch of Yum, a popular food blog.

Image Source

In the Google Chrome User Experience Report (which is based on real users' load times), Pinch of Yum passes Core Web Vitals with flying colors.

Based on real user data, the average Largest Contentful Paint time is just 1.6 seconds, which is quite good.

Basically, Pinch of Yum is a fast website — or at least it's fast enough to provide a good experience to its human visitors.

But if you look at the mobile performance score, it's an atrocious 11 out of 100.

Image Source

Does that mean Pinch of Yum has poor performance? No, the site loads quite fast, and the low performance score doesn't change that.

But because of the quirks of how the performance score is calculated, Pinch of Yum seems to not be optimized. More on what these quirks are in a second…

Does that mean speed test scores are meaningless?

While real load times matter more than scores, it's important not to swing the pendulum too far in the other direction.

Speed test scores are not totally meaningless. They just need perspective.

If you see that a page has a low score, that does probably mean there are tactics and best practices that you can implement to improve your site's performance.

For example, optimizing your site's code delivery, shrinking the size of your images, improving server response time, and so on.

Implementing those tactics will typically improve your performance score and your real-world performance times.

The problem only arises when abstract performance scores are given more weight than real-world load times. Remember:

• Fast real-world load times without perfect performance scores are still fine.
• Perfect performance scores but without fast real-world load times are still a problem.

Additionally, small changes in your speed testing tool's configuration can have big effects on your performance score, which is why it's important to choose the right tool to test your site's performance.

Are your speed testing tools legit?

If you want to collect meaningful data about your site's performance, you need the right tool.

As we mentioned earlier, many speed testing tools use the same basic Lighthouse engine to collect their results, so there can be a lot of similarities between different tools.

However, some tools also go above and beyond when it comes to letting you configure your tests to collect meaningful real-world data.

Quality speed test tools let you adjust your test.

If you want to collect meaningful information with a synthetic speed test tool, it's important that your tool lets you adjust the testing environment to match your site's audience.

That is, you want your speed testing tool's testing environment to mimic your real-world audience.

Here are some of the most important considerations to help you do that.

Test from different geographic areas.

The first important consideration is the physical location of your test, as physical distance still plays a role in your site's load times.

If your audience is in a single geographic area, e.g. a single country, you'll want to run your tests exclusively from that country.

If your audience is spread out across different countries, you'll want to run multiple tests from different locations to get a better understanding of your site's global performance.

If you're not sure where your audience is coming from, you can use the Geo → Locations report in Google Analytics.

Image Source

Many speed test tools don't make it easy to account for location — most notably Google PageSpeed Insights.

Google PageSpeed Insights tests from the server that's closest to the physical location from which you're running the test and doesn't give you any option to change this (short of using a VPN).

For example, if your website targets visitors in the U.S. but you're running the speed test while traveling in Vietnam, PageSpeed Insights will run its speed test from the nearest server in Asia.

If your site's visitors are from the U.S., testing the load times from Asia isn't really helpful because it's not representative of your actual user experience. Instead, it will just make your site seem like it loads slower than it does for your actual audience.

In fact, this is one of the reasons why Pinch of Yum's performance scores were so low in the screenshot above, despite the Pinch of Yum website actually having quite good performance.

If you want to see where PageSpeed Insights is testing from, you need to hover over the small Slow 4g Throttling option below the test results.

Image Source

Test with different connection speeds.

Another important testing variable is the connection speed of the testing tool.

If your tool is configured to have a very fast connection speed, your site will appear to load very quickly.

Conversely, if the tool is configured to have a slow connection speed, your site will appear to load slower than it does in the real world.

Here, Google PageSpeed Insights is limited again, most notably when it comes to the mobile test. When testing mobile performance, PageSpeed Insights uses an abnormally slow 4G connection, which can often lead to poor mobile performance results.

In fact, this is why it's common for sites to have good desktop performance scores, but very low mobile speeds.

However, this connection speed isn't really representative of your site's audience, so it's painting a very pessimistic picture. It's more of a "worst-case scenario" than a realistic scenario for what your mobile visitors will experience.

If you want to get more meaningful data, it's important to choose a speed-testing tool that lets you mimic the real connection speeds of your target audience.

Test from different devices.

Finally, you'll also want to consider the device that you're testing from. Not only the difference between desktop and mobile devices but also the differences in device computing power.

For the third time, PageSpeed Insights can be overly pessimistic in its results, especially when it comes to mobile performance. This can lead to low performance scores despite good real-world performance.

PageSpeed Insights collects its mobile performance data using an older Moto G4 device, which has less processing power than the devices many of your users will have. Again, it's more of a "worst case" scenario than a realistic test for the average visitor.

Having less processing power can especially affect sites that use a lot of JavaScript because it will take longer for these low-powered devices to process the code.

To collect more meaningful data, you'll want to test from the same types of devices that visitors are using to access your site (which you can see in the Devices report in Google Analytics).

Image Source

WebPageTest gives you more control over these metrics.

If you want to collect more meaningful information about your site's performance, a tool like WebPageTest offers a much better option than PageSpeed Insights.

This is because WebPageTest lets you configure all of the factors above, rather than locking you into preset configurations.

When setting up a test in WebPageTest, you can fully customize all of the following information:

• Test location.
• Connection speed.
• Test device.

Image Source

What's more, WebPageTest lets you run up to nine tests at the same time, which helps you eliminate single-test variability.

WebPageTest also skips arbitrary performance scores and instead focuses on real-world data. Here's an example of the results page for a WebPageTest analysis.

Image Source

Want even more data? Look at real user load times.

If you really want to understand the experiences of your users when it comes to your site's performance, you can implement a testing strategy called real user monitoring.

With this approach, you'll add a tracking script to your site, much like you add the script for Google Analytics.

Then, the testing tool will monitor the load time metrics for each individual visitor.

This lets you see what your site's performance is for your real audience. You'll be able to factor in your audience's diverse locations, devices, connection speeds, and so on.

The only downside is that these real user performance monitoring tools typically cost money. For example, Pingdom's Real User Monitoring service starts at $10 per month, though you can test it out with a 30-day free trial.

Scores Don't Always Tell the True Story

If you want to understand your website's performance, website speed tester tools are incredibly useful. However, it's important not to get overly caught up in abstract performance scores.

Instead, you'll want to focus on real performance metrics, such as Largest Contentful Paint (LCP), time to first byte (TTFB), and so on.

Performance scores can have some basic value when it comes to quickly assessing how optimized your site is. But they should never take priority over your site's actual load time metrics.

To collect real load time metrics that mimic the real world, it's important to configure your test with realistic test variables when it comes to the location, connection speed, and device of the test.

Start focusing on the website performance metrics that actually matter. You can then ensure you're delivering a great experience for your visitors, while not stressing over arbitrary performance scores.