Monitoring shop search: when search finds nothing at all

Why an empty result list goes unnoticed for so long

From the server's point of view, a page with zero results is a success. It is delivered in full, carries header, navigation and footer, and usually answers with status 200. An uptime check asks exactly that: it checks the HTTP response of the home page (XICTRAQ® documentation). That the search behind it no longer finds any products lies outside what such a check can see. The team notices late as well, because products are maintained through the administration, and customers who find nothing usually switch shops without a word.

The figures on online retail show how much depends on it. In the digital association's representative survey, 96 percent of respondents aged 16 and over had shopped online in the past twelve months (Bitkom). On the retail side, 97 percent of the surveyed retail companies with at least ten employees run their own web shop (Bitkom). For many visitors, the search in that shop is the shortest route to the product, especially in large assortments.

Around the sales days, this becomes more pressing. Of the 2025 Black Friday and Cyber Week shoppers, 69 percent wanted to take advantage of the deals online (Bitkom), and the planned budget averaged 312 euros (Bitkom). According to the study report's classification, 48 percent fall into the category of rather planned purchases (Bitkom). People who buy with a plan usually already know what they are looking for, and they type it in. Black Friday falls on November 27 in 2026; from today, that leaves just under four weeks.

Even a well-designed no-results page helps little in that situation. It is meant for individual terms for which the catalog has no products, and it can then correct spelling, offer categories or suggest related terms. If search fails entirely, every query ends on that page, including the one for the bestseller.

How the search index breaks after an import, update or index run

A shop's search usually does not read directly from the product table but from an index built from the product data. Shopware 6, for example, maintains a dedicated sub-index that holds the search keywords of all products (Shopware documentation). Larger shops can also move search to a search server such as OpenSearch or Elasticsearch. When data and index drift apart, search shows an old state, part of the products or nothing at all.

Typical triggers are tied to a change that ran shortly before:

• Import without an index run: an import writes products, but the index is not updated or only partially, for example because indexing was switched off for the import.
• Stalled queue: if indexing runs through the message queue, it can take several hours according to Shopware (Shopware documentation). If no worker processes the queue, the index stays at the old state.
• Search server unreachable or index empty: the search service did not restart after an update, or a rebuild aborted after the alias already pointed to the new, still empty index.
• Refresh suspended: for bulk imports, index refresh can be paused to write faster. If the import aborts before the old value is restored, new and changed products stop being searchable.
• Theme or plugin update: the result list template changes, an instant search script throws an error, or the search form sends the term under a different parameter.
• Visibility and sales channel: imported products exist but are not assigned to a sales channel or not set to visible. Search works correctly and still finds nothing.

How quickly new data reaches the search server is governed by the refresh interval. By default, OpenSearch refreshes an index every second (OpenSearch documentation). The value -1 disables refreshing (OpenSearch documentation), and for pure indexing phases the documentation names exactly that as a way to speed up indexing (OpenSearch documentation). What saves time during the import becomes a silent error when nobody restores the normal value afterwards.

Not every outage ends in an empty list. In Shopware, an environment variable determines what happens when the search server fails: with the value 1, the default, the exception is thrown (Shopware documentation), and the search page shows an error. With the value 0, the error is only logged and search falls back to the database (Shopware documentation). The shop keeps answering, with different matching logic and possibly a different response time; this case shows up in step timings rather than in a red status.

What uptime, API and browser checks each see

Three types of check are candidates for search, and they cover different layers. Uptime monitoring checks availability. API monitoring checks individual JSON endpoints with assertions (XICTRAQ® documentation). The browser check drives a real Chromium browser through a multi-step flow (XICTRAQ® documentation) and therefore sees what customers see.

The table shows why the browser check plays the lead role and the API check complements it. The article Browser monitoring: when uptime is green but checkout is broken summarizes the basics of synthetic flows.

Setting up search as a browser check

In XICTRAQ® browser monitoring, a scenario consists of steps such as opening a page, filling in a field, clicking, waiting for an element and checking text or URL. A scenario can have up to 40 steps, and the interval ranges from 5 minutes to 24 hours (XICTRAQ® documentation). Seven are enough for search:

1. Open the home page (goto): if this already fails, the check reports an error status instead of an empty search.
2. Enter the term (fill) in the search box in the header.
3. Submit the search (click) via the search form button, just as customers do.
4. Wait for the result list (wait_for) using the selector of the first product card. With zero results it does not appear, and the step runs into its timeout.
5. Check the URL (assert_url): the results page carries the term in its parameter. If it is missing, the form sent something else.
6. Check the result (assert_text) against the product name on the first card.
7. Screenshot as evidence of the green run.

{
  "name": "Shop search: winter jacket returns results",
  "viewport": {
    "width": 1280,
    "height": 800
  },
  "timeout_ms": 15000,
  "step_retry": 1,
  "steps": [
    {
      "type": "goto",
      "url": "https://shop.example.invalid/"
    },
    {
      "type": "fill",
      "selector": "header input[name=search]",
      "value": "winter jacket"
    },
    {
      "type": "click",
      "selector": "header form.search-form button[type=submit]"
    },
    {
      "type": "wait_for",
      "selector": ".search-results .product-box",
      "timeout_ms": 10000
    },
    {
      "type": "assert_url",
      "contains": "search=winter"
    },
    {
      "type": "assert_text",
      "selector": ".search-results .product-box .product-name",
      "contains": "Winter Jacket"
    },
    {
      "type": "screenshot"
    }
  ]
}

The selectors come from your own theme and can be read in the browser's developer tools. Three details decide whether the check measures reliably. First, wait_for waits until the selector is visible in the DOM, with a configurable timeout (XICTRAQ® documentation); an empty result list therefore leads to a clear failure instead of a green run. Second, assert_text is case-sensitive when checking that text is contained (XICTRAQ® documentation): the expected text belongs in the check exactly as it appears in the product name. Third, the step checks the first matching result. If the product you search for does not reliably rank first, the result list as a whole is the better target.

For the result list, a dedicated timeout of about ten seconds makes sense: a search that takes longer is hardly better for customers than an empty one. Technically, more would be possible, because a step may take up to 60 seconds, and the entire run is aborted after 90 seconds (XICTRAQ® documentation). The retry counter, which can be set from 0 to 3 per step (XICTRAQ® documentation), is best kept low when waiting for the result list: every extra attempt lengthens the run and can mask a search that only answers on the second try. If you prefer recording flows to writing them, the article Record browser flows and monitor them automatically shows the way; the full list of step types is in the browser monitoring documentation.

A second monitor that calls the results URL directly, without the form, is worth it: if only the run through the form fails, the error lies in the search box; if both fail, in the index or search server.

Choosing a search term that proves something

A search check is only as meaningful as its term. It must return results as long as the shop is healthy and must not depend on season, stock or a promotion. Several small monitors have proven better than one large one, because the alert then says in its name what has stopped working:

• Steady seller from the core assortment: a product that has been in the catalog for months and will stay there, with a name customers actually type.
• Item number: checks exact search. Some search configurations keep numbers in a separate index field that is easily forgotten during a migration.
• Term with an umlaut or made of two words: a German term such as "Wärmflasche" or a query such as "women's jacket" checks character handling and word splitting in the index.
• Service term: "returns" or "shipping costs". Experience shows that visitors search not only for products but also for returns, shipping or payment methods. If the shop routes such terms to a service page, this check tests whether that still holds after a migration.
• Not suitable: promotional items, clearance stock and products that disappear from search when out of stock. They turn the check red without search being broken.

A positive run needs a counter-check. A term that certainly does not exist in the shop, such as "zz-test-term-empty", must trigger the no-results page; the check verifies its message with assert_text. That sounds redundant, but it catches a tricky error: if an update loses the search parameter, the shop shows the same unfiltered product list for every term. The positive run stays green because the winter jacket appears in it. Only the counter-check turns red, because the nonsense term also returns results.

The result count itself is a weak target because it fluctuates with the assortment; a check for exact equality fires with every new product. The message about the result count still has a meaning of its own. If the result list loads without a page change, short messages such as "18 results returned" or "No results returned" count as status messages under WCAG success criterion 4.1.3 (W3C), which sits at level AA (W3C). If you address that message with a selector anyway, you can also check that it is marked up as a status for screen readers.

Step timings: when search slows down before it fails

Not every outage comes suddenly. Search can get slower over days: the search server lacks memory, a queue backs up, or after a change search runs through the database. For this, the run history records the total duration, the duration per step and success or failure for every run (XICTRAQ®). This makes it possible to spot creeping degradation in a single step long before the whole flow breaks (XICTRAQ®).

For search, the wait_for step on the result list is the interesting line: it measures the time from submitting to the first visible product card, so search, transfer and rendering together. The overview shows the last 50 runs per monitor (XICTRAQ® documentation); with a 15-minute interval, that is a good twelve hours of history. Typical patterns:

• Jump after a deploy: the duration rises from one run to the next and stays high. First question: what was shipped, and does search still run through the search server?
• Slow rise over days: a sign of growing queues, memory pressure or an index that is no longer being cleaned up.
• Outliers at fixed times: if the slow runs coincide with an import or index run, both compete for the same resources.
• Clearly different duration without a deploy: then it is worth checking the shop's error log to see whether search is falling back to an alternative path.

Checking search suggestions and instant search too

Many shops show suggestions with product images as soon as you start typing. This list usually comes through a separate request and a separate script; it can fail while full search works, and vice versa. For such components there is the type step: it works like fill but types character by character, so fields that react to every keystroke respond (XICTRAQ® documentation).

{
  "name": "Search suggestions: winter ja shows Winter Jacket",
  "timeout_ms": 10000,
  "steps": [
    {
      "type": "goto",
      "url": "https://shop.example.invalid/"
    },
    {
      "type": "type",
      "selector": "header input[name=search]",
      "value": "winter ja"
    },
    {
      "type": "wait_for",
      "selector": ".search-suggest .search-suggest-product",
      "timeout_ms": 5000
    },
    {
      "type": "assert_text",
      "selector": ".search-suggest",
      "contains": "Winter Jacket"
    },
    {
      "type": "screenshot"
    }
  ]
}

The suggestion check belongs in a monitor of its own. If only this one fails, the alert says exactly that, and nobody looks for the error in the index. A short timeout is appropriate here, because suggestions that appear only after several seconds go largely unnoticed.

In addition: checking the search endpoint with an API check

If the shop delivers its search results through an API, for example for a headless storefront or instant search, this endpoint can also be checked directly. API monitoring knows four types of assertion: status code, response time, values in the JSON body via JSON path, and text content (XICTRAQ®). For search, this combination is a good fit:

• Status 200 as the basic condition.
• Response time below a threshold derived from your own history.
• Result count in the JSON body greater than zero, using the gt operator (XICTRAQ®).
• A known product name in the response text via a text check.

Before setting it up, a manual spot check shows which field carries the result count. URL and field name here are examples; the real path is in the shop system's API documentation.

The advantage of the API check is its frequency: endpoint checks run from every 30 seconds (Agency) to every 5 minutes (Starter) depending on the plan (XICTRAQ®), and the history shows the response time of the last 100 runs (XICTRAQ®). The drawback is its view: the API check sees neither template nor script nor search form. It reports that the index answers, not that customers get to see the results.

Search check red: reading the screenshot, narrowing down the cause

If a step fails, XICTRAQ® takes a screenshot at the moment of failure and stores it in the run history (XICTRAQ® documentation). It also records the index of the failed step and the error message (XICTRAQ® documentation). The failure creates an alert with severity critical (XICTRAQ® documentation), which goes out through the configured alert channels; deduplication groups repeats, with 10 minutes as the default (XICTRAQ® documentation). When the run turns green again, XICTRAQ® closes the alert automatically (XICTRAQ® documentation).

For search, the screenshot is usually half the diagnosis:

The alert should reach the person who is allowed to trigger an index run or roll back a deploy, not a shared inbox. Whether the chain also holds at night can be checked beforehand in an alerting drill.

Check immediately after imports, index runs and deploys

Search outages typically follow a change. That is why the search check should be triggered right after every change, not only on its interval. "Run now" marks a monitor as due, and the next cron tick, which runs every minute, executes it without waiting for the interval (XICTRAQ® documentation). After a deploy, this usually shows within minutes whether search still finds products; how this works together with a rollback is described in the article on deploys as a cause of outages.

For nightly imports, a second measurement makes sense: a heartbeat at the end of the import script reports whether the import ran through at all. The browser check afterwards shows whether the result has reached search. If the heartbeat is missing, the error lies with the import; if it is there and search is empty, with the index.

For the sales weeks, this sequence works well:

1. Early November: set up monitors for the steady seller, item number, service term, counter-check and suggestions, and let them run green several times.
2. Before the promotion starts: do not schedule large imports and price updates for the last night; "Run now" after every import.
3. During Black Week: set a tight interval and route alerts to the on-call person who may trigger an index run.
4. For promotional items: add a check on the name of the promotional product, only for the promotion period. How to check the redemption of the promo code alongside it is shown in the article on discount codes at checkout.
5. After the promotion: pause the promotion checks and reset the interval of the permanent checks.

What the search check does not replace

A browser check proves that a handful of fixed terms return results. It does not measure whether the ranking of the results is good, whether synonyms work, or which terms from real customers lead nowhere. That is what the analysis of search terms is for, as described in the article on internal site search linked above. The check verifies function, the analysis verifies quality.

Your own figures stay clean in the process: the runner blocks the XICTRAQ® tracker so that synthetic runs do not fill analytics with test page views (XICTRAQ® documentation). The shop's own search statistics, on the other hand, may count the test term. Every run sends the user agent XICTRAQ-BrowserMonitor/1.0 (XICTRAQ® documentation), which can be used to filter out these requests in the shop.

After a major overhaul of search server, theme or shop version, search should also be tested by hand with a longer list of terms before going live. More checkpoints for Shopware shops are collected on the Shopware monitoring page; the same type of check also monitors the label of the order button.

Checklist: monitoring shop search before Black Week

• Define three to five search terms: steady seller, item number, term with an umlaut, service term
• For each term, create a browser check with wait_for on the first product card and assert_text on the product name
• Set up a counter-check with a nonsense term that expects the no-results message
• Check search suggestions in a separate monitor with the type step
• Watch the step timings of the result list for a week and note them as the normal value
• If there is a search API, also create an API check with a result count greater than zero
• "Run now" after every import, index run and deploy
• Add a heartbeat at the end of the import script
• Route alerts to the person who may trigger an index run

Search belongs among the functions you measure

A shop search that answers is not yet a search that finds. Between the two lie index, queue, search server and template, and each of these layers can fail silently after a change. A browser check with a fixed search term, a counter-check and an eye on step timings turns this into a measurement with a timestamp, a screenshot and an alert. Browser monitoring is included from the Business plan (XICTRAQ® documentation); the plans are listed on the pricing page. If you set up the monitors in browser monitoring now, you will have a series of green runs by November 27 and a normal value for step timings against which any deviation can be read.