How to Do a Competitor Analysis With Public Web Data

Published:

16 minute read

Acar Diveroli
Written by: Acar Diveroli
Dimmed rows of a comparison sheet behind a board with four bar rows, YOU, A, B and C; the YOU row is blue

An online shop that sells home espresso machines notices that its grinder sales in the UK have fallen for three months in a row. Its prices have not changed and its ads still run. The cause probably sits outside the company: a rival dropped its delivery fee, a new shop started ranking for "burr grinder", or a supermarket chain began selling a cheaper model. Finding out which one, and deciding what to do about it, is the job of a competitor analysis.

This post explains what a competitor analysis is, who counts as a competitor, a step-by-step method built on public sources, what to compare and where each piece of data comes from, a template you can copy, where SWOT fits, why the picture changes by country and city, and the legal limits of studying rivals.

What is a competitor analysis?

A competitor analysis is the process of identifying the businesses that compete for your customers and comparing them with you on what makes customers choose: product, price, convenience, message and reputation. Its output is not a portrait of each rival but a short list of decisions: where to match them, where to be different, and what to keep watching.

The terms overlap. "Competitive analysis" usually means the same exercise, sometimes widened to the whole market. "Competitive intelligence" is the ongoing function behind it: a person or team that follows competitors all year and reports on them, instead of one document written for a business plan.

What is the difference between direct and indirect competitors?

Direct competitors sell the same kind of product to the same customers; indirect competitors meet the same need with a different product. For the espresso shop, another online store selling similar machines and grinders is a direct competitor. A coffee chain selling capsule machines, a subscription that ships ground beans every month and a supermarket with a cheap filter machine are indirect ones: a customer who buys any of them may never buy a grinder.

To find both kinds, search your main product terms as a customer would, ask your sales and support teams which names come up, and read your own reviews for "I switched from" lines. A company that wins your customers without ever appearing in search results is still a competitor.

How do you do a competitor analysis, step by step?

A competitor analysis takes seven steps, from one clear question to a routine that repeats:

  1. Write the question. "Why are grinder sales falling in the UK?" produces a better analysis than "analyse the market", because it decides which competitors and which data matter.
  2. Pick the competitors. Three to five direct and one or two indirect, each named in the sheet with the countries where you meet them.
  3. Choose what to compare. Keep to the areas that can answer the question, usually five to eight: range, price and delivery, message, search visibility, ads, reviews, hiring, company facts.
  4. Collect from public sources. Visit their websites, search results, ad libraries, review pages and filings, and write down the source and the date for every entry.
  5. Put it in one sheet. One row per area and one column per company, your own included, with the same units, date and country throughout.
  6. Read it and decide. Mark where you lead, where you trail and where nobody competes yet, and turn each finding into an action with an owner.
  7. Repeat on a schedule. Prices and ads change weekly, messages and ranges every few months, so refresh each area at its own pace.

What should you compare, and where do you find it?

Compare what customers weigh when they choose, and take each item from the source that shows it most directly. The usual areas:

AreaWhat to recordPublic sourceRefresh
RangeModels, sizes, bundlesProduct and category pagesMonthly
Price and deliveryPrice, shipping fee, returnsProduct and delivery pagesWeekly
MessageHeadline, main promiseHome and landing pagesQuarterly
Search visibilityWho ranks for key queriesSearch results, licensed dataMonthly
AdsOffers, formats, countriesPublic ad librariesWeekly
ReviewsRatings, repeated complaintsReview sites, marketplacesMonthly
HiringOpen roles, new locationsCareers page, job boardsMonthly
Company factsSize, accounts, ownersRegisters, annual reportsYearly

Range, price and delivery come from the competitor's own pages. Record the full price a customer pays, meaning the product price, the delivery fee and any minimum order, because a lower shelf price with a high shipping charge is not a lower price. Checking hundreds of prices every week is a process of its own; competitor price tracking covers product matching and check frequency.

The message is on the home and landing pages. Note the headline and the main promise (price, speed, quality or service); together they show the competitor's positioning, the place it wants to hold in the customer's mind.

Reviews show what customers praise and what they complain about. Three- and four-star reviews are often the most specific. Record themes such as "slow delivery to Ireland", not the names of reviewers.

Hiring is an early signal. Job postings for a warehouse in a new country, a team for a new product line or a wave of support roles often appear months before a launch.

Company facts come from official registers and, for listed companies, annual reports. In the UK, for example, Companies House offers its public company data free of charge, including filed annual accounts that show a private rival's size and growth.

How do you analyse a competitor's website?

Read a competitor's website first as a customer and then as an analyst. As a customer, go from the home page to a product and on to the delivery and returns pages, noting the costs and conditions you would face before paying. As an analyst, list the categories the site pushes hardest and sum up in one plain sentence what it says about the company.

The careers and press pages show plans, and the Internet Archive's Wayback Machine keeps old copies of many public pages, so you can see when a competitor changed its headline, its prices or its range.

How do you compare search visibility with competitors?

You compare search visibility by listing the queries your customers type and recording who appears for each, in the country that matters. Start with your own data: the Performance report in Google Search Console shows the clicks, impressions and average position your own site gets, broken down by query, page, country and device. It shows nothing about other sites.

For competitors, two routes are legitimate. Licensed SERP data services sell search-result data, collected under their own agreements and often available down to city level. For a few important queries, a person can also search by hand in a clean, logged-out browser and note who ranks; automated rank tracking explains how to combine both with Search Console.

A script that queries Google and collects the results is not a legitimate route. Google's spam policies describe automated queries, including scraping results for rank-checking without express permission, as machine-generated traffic that violates its policies and terms of service.

For each query, record who holds the top results and which of the competitor's pages ranks. A rival whose buying guides rank competes for an earlier moment in the purchase than one whose product pages rank.

How do you see what competitors advertise?

You see competitors' ads in the public ad libraries that the large platforms keep, searched by the competitor's name and website. Meta's Ad Library and the Google Ads Transparency Center show the ads an advertiser runs, with dates and, for ads shown in the EU, extra detail on reach and targeting. Spend and results stay hidden for ordinary commercial ads. How to find competitor ads walks through each library.

For the sheet, record the offer each competitor leads with (a discount, free delivery, a bundle) and how long its main ads have been running. An ad that has run for months is one the advertiser keeps paying for.

What does a competitor analysis template look like?

A usable template is a grid: areas in rows, companies in columns, and a source and date behind every cell. Keep each cell to a few words so the whole sheet fits on one screen. The espresso shop's sheet for the UK might look like this:

AreaYouCompetitor ACompetitor B
Grinder range6 models11 models4 models
DeliveryFree over a minimumAlways freePaid
Main promiseExpert adviceLow pricesFast delivery
"burr grinder" searchPage 2Top 3Not ranking
Ad offerNone running10% off first orderFree delivery
Review themeSlow repliesDamaged parcelsFew reviews
HiringNoneUK warehouse jobsNone
Checked on17 Nov 202617 Nov 202617 Nov 2026

Read across each row to find the gap. Here, Competitor A's free delivery and warehouse job ads suggest it is building local stock in the UK, which explains the lost grinder sales better than price alone. Keep the sources in a second tab or as cell comments.

The same grid is often called a competitor matrix; a feature matrix is the variant that marks which company offers each product feature.

Where does a SWOT analysis fit?

A SWOT analysis is a summary of the competitor sheet, not a replacement for it. It lists your strengths and weaknesses, which are internal, and the opportunities and threats around you, which are external. Each line should point to evidence in the sheet: "Threat: Competitor A is building stock in the UK (warehouse job ads, free delivery)" helps a decision, while "Threat: strong competition" does not.

Porter's five forces, another common framework, looks wider: it adds suppliers, buyers, new entrants and substitutes to rivals, and suits a market-entry decision better than a monthly review.

Why does a competitor look different from another country or city?

A competitor looks different from another location because many websites change what they show by where the visitor is. A shop can show different prices, delivery fees and stock to visitors from different countries, search results for "espresso machine" differ between cities where local shops rank, and ads are bought for specific regions. An analysis done only from your office records your office's view of the market.

The fix is to look from the market you are analysing. A Residential Proxy routes your browser through a home internet connection in the country and city you choose, so a public page loads as it would for a local visitor. Use it at human scale, and keep to each site's terms.

Search results are where location matters most. Our SEO proxy page describes how teams view search result pages from a chosen country or city, alongside Search Console and licensed data rather than instead of them.

Competitor analysis is legal when it stays with public information and your own observation; it crosses the line when it reaches for information a competitor protects or turns into coordination. In the EU, the Trade Secrets Directive (EU) 2016/943 counts independent discovery and the observation, study, disassembly or testing of a product made available to the public as lawful ways to obtain information. Unauthorised access to documents, files or other material that contain a trade secret is unlawful.

The practical rules follow from that:

  • Public pages only. No logging in where you have no right to, no guessing addresses of unpublished pages, no leaked documents.
  • No fake identities. No fake customer accounts, no posing as a buyer to draw price lists out of sales staff, no asking former employees for confidential details.
  • Buying the product is fine. Testing a competitor's product bought as an ordinary customer is the kind of observation the directive allows; some software terms restrict trials by competitors, so read them first.
  • Respect site terms and load. Read a site's terms and robots.txt before you automate anything, and keep request rates low.
  • Aggregate personal data. Reviews and social posts carry people's names; record themes and counts, not individuals, as the EU's GDPR and similar laws such as Türkiye's KVKK expect.

Competition law sets the second limit. Watching rivals' public prices on your own is normal: the European Commission's guidelines on horizontal co-operation note that companies can independently use algorithms to monitor competitors' prices and inform their own pricing. Exchanging information with competitors is different, because even an exchange of public information can reduce the uncertainty that keeps prices competitive. Collect alone, keep your sheet to yourself, and never discuss future prices with a rival.

Where is competitor analysis used?

  • Business plans: showing investors who else serves the market and why you can win.
  • Pricing reviews: full delivered prices compared before a price change.
  • Product planning: gaps in rivals' ranges that customers ask about.
  • Marketing: a promise nobody else in the market makes.
  • Entering a new country: who leads there, at what price, with which delivery terms.
  • Sales teams: short comparison sheets, often called battlecards.
  • SEO and content: queries where rivals rank and you do not.

Common mistakes

  • Analysing everyone. Fifteen competitors produce a sheet nobody reads; five produce decisions.
  • Ignoring indirect competitors. The capsule machine at the supermarket may take more sales than the shop you watch.
  • No source or date. By next month, a price without a date is a rumour.
  • Comparing shelf prices only. Delivery fees and minimum orders change the real price.
  • Looking from one location. Prices, search results and ads differ by country and city.
  • Copying instead of learning. Reusing a rival's texts or images infringes copyright, and copying its strategy keeps you one step behind.
  • Doing it once. Last year's analysis describes a market that has since moved.

Decision guide

NeedRecommendation
A business plan sectionFive competitors, company facts, SWOT
Why sales fell in one marketOne question, that market's view
Your price positionFull delivered price, checked weekly
Search visibilitySearch Console plus licensed SERP data
Rivals' campaignsPublic ad libraries, by advertiser
The view from another cityLocal connection, human scale
Many products in many marketsPolite automated collection or a service
Anything behind a loginLeave it out

Frequently asked questions

What should a competitor analysis include?

A competitor analysis should include the list of competitors, direct and indirect, the areas you compare, the source and date of each finding, and the conclusions with an owner for each action. The usual areas are range, price and delivery, message, search visibility, ads, reviews, hiring and company facts. A SWOT summary is optional; the sheet with its sources is not.

How many competitors should I analyse?

Three to five direct competitors and one or two indirect ones are enough for most businesses. Add another only when a new name keeps appearing in search results, reviews or sales conversations.

How often should a competitor analysis be updated?

Update each area at the pace it changes, from weekly for prices and ads to yearly for company facts. Review the whole sheet at least once a quarter, and again whenever a competitor launches something or enters your market.

What is the difference between competitor analysis and market research?

Market research studies customers and demand: who buys, how much, why and at what price. Competitor analysis studies the companies that compete for those customers. Most projects need both, because a gap in competitors' offers matters only if customers want what is missing.

Can AI do a competitor analysis?

AI assistants can speed up parts of it, such as summarising a competitor's public pages, grouping review themes or drafting a SWOT from your sheet. They can also be out of date or simply wrong, so check every claim against the source page and record the date you checked it. Do not paste confidential company data into a tool your company has not approved.

Yes, reading public pages and taking notes is legal. The limits are unauthorised access to protected information, automated collection against a site's terms, reusing a rival's texts and images, and sharing price plans with competitors.

Summary

A competitor analysis compares your business with the companies your customers could choose instead. Start with one question, pick a handful of direct and indirect competitors, collect each area from its public source with a date, keep everything in one sheet and refresh it at the pace each area changes. Look at every market from where its customers are and stay with public information; when the work grows to many products and countries, our market research proxies are built for that kind of collection.

Ask ChatGPTAsk Claude