SKIP TO FINDINGS
SAMPLE REPORT · EXAMPLE DATA

This is a sample report for Fjellsport Demo, a made-up chain. It describes no real store. Back to SendTilly Request a pilot audit

SendTilly SAMPLE REPORT

Back to SendTilly

SENDTILLY · LOCAL STOCK CHECK

Fjellsport Demo

STORE CHAIN
Fjellsport Demo, 5 stores
CHECKED
to
AGENTS
Anthropic and OpenAI
QUESTION ASKED
Is this product in stock at this store, and can I collect it today?

What we checked

4products
  • Ridge Shell Jacket · size M
  • Merino Base Layer · size L
  • Trail Hiking Boot · size 43
  • Frost Beanie
5stores
  • Oslo
  • Bergen
  • Trondheim
  • Stavanger
  • Tromsø
20missions in each mode
  • Agent mode
  • Fetch mode
MISSION
one product at one store, asked one question
AGENT MODE
the AI clicks through the store's site, like a shopper
FETCH MODE
the AI takes one quick look at the page to answer in a chat, and clicks nothing

THE QUESTION, FILLED IN FOR ONE MISSION

Is Ridge Shell Jacket, size M in stock at Fjellsport Demo Bergen, and can I collect it today?

  1. 01ASK

    The question goes to two models from different vendors, Anthropic and OpenAI, with the product and the store filled in.

  2. 02ANSWER

    In agent mode the AI clicks through the store's site, like a shopper. In fetch mode it takes one quick look at the page, the way a chat answer does.

  3. 03CHECK

    A fixed recipe reads the store's own page, with no model, at the start and at the end of each check.

  4. 04GRADE

    Each answer gets a verdict and an owner: the store's site, the access layer or the agent. A finding needs agents from two vendors to agree, or an access check that saw the same refusal.

Results at a glance

20 MISSIONS IN EACH MODE

Agent mode

The AI clicks through the store's site, like a shopper.

14 of 18 missions passed

  • 13 PASS
  • 1 PASS (stock only)
  • 3 WRONG ANSWER
  • 1 PARTIAL
  • 1 AGENT ERROR
  • 1 NOT GRADED
  • 4 had a store-facing failure.
  • 1 could not be graded.
  • 1 ended in an agent error, which is not counted against the store.

Fetch mode

The AI takes one quick look at the page to answer in a chat, and clicks nothing.

5 of 19 missions passed

  • 5 PASS
  • 12 FAIL
  • 2 BLOCKED
  • 1 NOT GRADED
  • 14 had a store-facing failure.
  • 1 could not be graded.

Who the failures belong to

BOTH MODES

THE STORE'S SITE

16

failures in 3 findings

THE ACCESS LAYER

2

failures in 1 finding

THE AGENT

1

agent error, not counted against the store

Results by product and store

ONE CELL PER MISSION

AGENT MODE · 14 OF 18

The AI clicks through the store's site, like a shopper. 14 of 18 graded missions passed. 1 could not be graded.

PRODUCT Oslo Bergen Trondheim Stavanger Tromsø PASSED
Ridge Shell Jacketsize M Oslo PASS Bergen WRONG ANSWER Trondheim WRONG ANSWER Stavanger PASS Tromsø NOT GRADEDStock changed during the check. 2 of 4
Merino Base Layersize L Oslo PASS Bergen PASS Trondheim PASS (stock only) Stavanger PASS Tromsø PASS 5 of 5
Trail Hiking Bootsize 43 Oslo PASS Bergen AGENT ERROR Trondheim WRONG ANSWER Stavanger PARTIAL Tromsø PASS 2 of 4
Frost Beanie Oslo PASS Bergen PASS Trondheim PASS Stavanger PASS Tromsø PASS 5 of 5

FETCH MODE · 5 OF 19

The AI takes one quick look at the page to answer in a chat, and clicks nothing. 5 of 19 graded missions passed. 1 could not be graded.

PRODUCT Oslo Bergen Trondheim Stavanger Tromsø PASSED
Ridge Shell Jacketsize M Oslo FAIL Bergen FAIL Trondheim FAIL Stavanger FAIL Tromsø NOT GRADEDStock changed during the check. 0 of 4
Merino Base Layersize L Oslo FAIL Bergen FAIL Trondheim FAIL Stavanger BLOCKED Tromsø BLOCKED 0 of 5
Trail Hiking Bootsize 43 Oslo FAIL Bergen FAIL Trondheim FAIL Stavanger FAIL Tromsø FAIL 0 of 5
Frost Beanie Oslo PASS Bergen PASS Trondheim PASS Stavanger PASS Tromsø PASS 5 of 5

Select a coloured cell to go to its finding.

What the labels mean

PASS
The answer matched the store's own page.
PASS (stock only)
The stock answer matched. The collect-today claim could not be checked.
WRONG ANSWER
Agents from two vendors gave the same answer, and the store's own page shows something different.
FAIL
Agents from two vendors could not reach an answer on this site.
PARTIAL
The page did not hold all the data needed to answer.
BLOCKED
The site refused or challenged the request, or a robots.txt rule stopped the agent.
AGENT ERROR
The agent failed in a way that is not the store's fault. It is not a finding.
NOT GRADED
Could not be compared with the store's page. The reason is shown in the cell.

Findings

4 · MOST MISSIONS AFFECTED FIRST
01FINDING FAIL Fetch mode Attributed to THE STORE'S SITE
AFFECTS 12 of 19 Fetch-mode missions

Store stock only shows after someone picks a store

AI assistants from Anthropic and OpenAI could not reach an answer on this site.

Same failure on both first attempts in 12 missions.

AFFECTED PRODUCTS AND STORES

Ridge Shell Jacket size M
  • Oslo
  • Bergen
  • Trondheim
  • Stavanger
Merino Base Layer size L
  • Oslo
  • Bergen
  • Trondheim
Trail Hiking Boot size 43
  • Oslo
  • Bergen
  • Trondheim
  • Stavanger
  • Tromsø

WHAT HAPPENED

The product page shows no store stock until someone picks a store. An AI that takes one quick look at the page clicks nothing, so it never sees a stock answer.

WHY IT MATTERS TO A SHOPPER

When a shopper asks an AI assistant in a chat, it cannot say whether the store has the product, so it sends the shopper elsewhere or guesses.

WHAT TO CHECK

Put each store's stock line in the page HTML itself, or publish store stock in the page data, so it is there before any script runs.

ANSWER VS PAGE

WHAT THE PAGE RETURNEDFetch mode, nothing clicked
STOCK
not on the page
COLLECT TODAY
not on the page
WHAT THE STORE'S OWN PAGE SHOWSReference reading
STOCK
In stock
COLLECT TODAY
Yes
What the page returnedChoose a store to see stock

SCREENSHOT

SCREENSHOT · BEFORE09:14:07 UTC

Drawn for this sample. A real report shows the stored screenshot.

REPRODUCE

PRODUCT PAGE ADDRESSfjellsport-demo.test/product/merino-base-layer?size=L
  1. Open fjellsport-demo.test/product/merino-base-layer?size=L.
  2. Without choosing a store, read the stock line. It says "Choose a store to see stock".
  3. Choose "Fjellsport Demo Oslo" under "Choose a store".
  4. The stock line now reads "In stock at this store".
02FINDING WRONG ANSWER Agent mode Attributed to THE STORE'S SITE
AFFECTS 3 of 18 Agent-mode missions

Page data says in stock for every store

AI assistants from Anthropic and OpenAI gave the same wrong answer.

Same failure on both first attempts in 2 missions; 2 of 3 after a third attempt in 1 mission.

AFFECTED PRODUCTS AND STORES

Ridge Shell Jacket size M
  • Bergen
  • Trondheim
Trail Hiking Boot size 43
  • Trondheim

WHAT HAPPENED

The page data marks the product InStock with no store attached. Agents from both vendors read it as stock at the store they were asked about. The store's own page says the size is not in stock there.

WHY IT MATTERS TO A SHOPPER

A shopper who asks an AI assistant is told to come in and collect a size that is not on the shelf.

WHAT TO CHECK

Give each store its own offer in the page data, with that store’s availability, and keep the online offer separate.

ANSWER VS PAGE

WHAT THE AGENT ANSWEREDAgents from Anthropic and OpenAI
STOCK
In stock
COLLECT TODAY
Yes
WHAT THE STORE'S OWN PAGE SHOWSReference reading
STOCK
Out of stock
COLLECT TODAY
No
What the agent answeredYes, the Ridge Shell Jacket in size M is in stock at Fjellsport Demo Bergen and you can collect it today.

AGENT SESSION

Anthropic's agent
PRODUCT PAGE ADDRESSfjellsport-demo.test/product/ridge-shell-jacket?size=M
  1. goto /product/ridge-shell-jacket
  2. read page data: Offer availability InStock
  3. read button "Collect in store"
  4. scroll to "Stores", store picker not opened
  5. read store list: Bergen open until 20:00
  6. Gave the final answer.WRONG ANSWER

SCREENSHOT

SCREENSHOT · BEFORE09:14:07 UTC
SCREENSHOT · AFTER09:21:40 UTC

Drawn for this sample. A real report shows the stored screenshot.

03FINDING BLOCKED Fetch mode Attributed to THE ACCESS LAYER
AFFECTS 2 of 19 Fetch-mode missions

Links to a size get a security check instead of the page

Confirmed by one attempt and the run's access check in 2 missions.

AFFECTED PRODUCTS AND STORES

Merino Base Layer size L
  • Stavanger
  • Tromsø

WHAT HAPPENED

Two links that point to a specific size showed a security check page instead of the product. The same pages without the size opened normally.

WHY IT MATTERS TO A SHOPPER

An AI assistant that follows a link to a size sees no product at all, so it cannot answer for that size.

WHAT TO CHECK

Look for a bot-protection rule that matches the size parameter on product pages, and allow declared, signed visitors through it.

REQUEST AND RESPONSE

REQUESTSendTillyBot, signed
ADDRESS
/product/merino-base-layer?size=L
USER AGENT
SendTillyBot/1.0
SIGNED
yes
RESPONSEThe store's server
STATUS
403
WHAT CAME BACK
security check
BASELINE VISIT
opened normally · 200

The response is as the run's access check recorded it.

REPRODUCE

PRODUCT PAGE ADDRESSfjellsport-demo.test/product/merino-base-layer?size=L
  1. Request /product/merino-base-layer?size=L without a browser.
  2. The answer is HTTP 403 with a security check page.
  3. Request /product/merino-base-layer. The answer is 200.
04FINDING PARTIAL Agent mode Attributed to THE STORE'S SITE
AFFECTS 1 of 18 Agent-mode missions

Pickup time is missing for the boots at Stavanger

AI assistants from Anthropic and OpenAI found the same data missing.

Same failure on both first attempts in 1 mission.

AFFECTED PRODUCTS AND STORES

Trail Hiking Boot size 43
  • Stavanger

WHAT HAPPENED

The page shows the boots in stock at Stavanger but gives no pickup time for that store, so the question about collecting today has no answer on the page.

WHY IT MATTERS TO A SHOPPER

A shopper is told the boots are there but not whether they can collect them today.

WHAT TO CHECK

Show the pickup time for every store that has the product in stock, as the other stores already do.

ANSWER VS PAGE

WHAT THE AGENT ANSWEREDAgents from Anthropic and OpenAI
STOCK
In stock
COLLECT TODAY
Not stated
WHAT THE STORE'S OWN PAGE SHOWSReference reading
STOCK
In stock
COLLECT TODAY
not on the page
What the agent answeredThe Trail Hiking Boot in size 43 is in stock at Fjellsport Demo Stavanger.

SCREENSHOT

SCREENSHOT · BEFORE09:27:30 UTC

Drawn for this sample. A real report shows the stored screenshot.

Access

OBSERVATIONS FROM ONE CHECK

robots.txt by crawler and assistant

HTTP 200
TOKENPRODUCT PAGESITE ROOTMATCHED
SendTillyBotSendTilly, this check allowed allowed wildcard group, line 2
GPTBotOpenAI allowed allowed wildcard group, line 2
OAI-SearchBotOpenAI allowed allowed wildcard group, line 2
ChatGPT-UserOpenAI allowed allowed wildcard group, line 2
ClaudeBotAnthropic allowed allowed wildcard group, line 2
Claude-UserAnthropic allowed allowed wildcard group, line 2
PerplexityBotPerplexity allowed allowed wildcard group, line 2
CCBotCommon Crawl RULE PRESENT RULE PRESENT own group, line 9
*any allowed allowed wildcard group, line 2
Google-ExtendedGoogle TRAINING OPT-OUT TRAINING OPT-OUT own group, line 12

A rule being present does not show whether any assistant is blocked. A training opt-out is a control for model training, not an access rule.

How the site treated a declared visit

SendTillyBot, signed requests
DECLARED VISIT
security check · 403
BASELINE VISIT
opened normally · 200
COMPARED
The site treated the declared visit differently from the baseline visit.

Checks ran as SendTillyBot/1.0, egress 203.0.113.7.

Method and limits

CHECKED

Run to . Report built .

MODES

The AI clicks through the store's site, like a shopper. The AI takes one quick look at the page to answer in a chat, and clicks nothing.

Agent mode uses an agent-style model driving a browser. Fetch mode reads the page without JavaScript, the way a simple crawler does.

MODELS

Models are called through their APIs inside SendTilly's own browser loop. They are agent-style shoppers, not the consumer assistant apps.

Agent, agent-style (Anthropic, claude-sonnet-5-5, example-id-a1)

Agent, agent-style (OpenAI, gpt-5.6-luna, example-id-b1)

Fetch, agent-style (Anthropic, claude-haiku-5-5, example-id-a2)

Fetch, agent-style (OpenAI, gpt-6-luna, example-id-b2)

Third attempts, agent-style (Anthropic, claude-opus-5-5, example-id-a3)

Third attempts, agent-style (OpenAI, gpt-5.6-sol, example-id-b3)

MARKETS

Norway

REFERENCE READING

Store page read by recipe “Local stock reading”, version 3, at the start and end of each check.

ENGINE

Rules v1.8, stored cause key ck1, cross-page grouping fk1, wording templates version 2.

HOW FINDINGS ARE COUNTED

Affects is the number of distinct missions (a product at a store, or one task) with this failure, out of the missions in that mode that had at least one graded attempt. Agreement shows how many of those failed the same way on both first attempts, how many were settled 2 of 3 after a third attempt, and how many were confirmed by one attempt and the run's access check. Findings are ordered by affected missions, then agreement. Nothing is combined into a single number.

AGENT ERRORS

1 of 38 graded attempts ended in an agent error. These are the agent's own failures. They are not findings and are not counted against the store.

NOT GRADED

Stock changed during the check: 2.

BLOCK RATE

0 of 18 graded agent missions ended BLOCKED.

2 of 19 graded fetch missions ended BLOCKED.

COLLECT-TODAY CLAIMS

1 PASS (stock only) result: the stock answer matched, and the collect-today claim could not be checked.

LIMITS
  • These results describe what the checked pages and agents did on the dates shown. Stores change, so a later check can differ.
  • Findings are confirmed by agents from two different vendors, or by an access check that observed a challenge or a refusal. An agent's own mistake is counted as an agent error and is never a finding.
  • The reference for each answer is the store's own page, read by a fixed recipe at the start and end of the check. When stock changed between the two readings, the result is marked not graded.
  • Checks ran as SendTillyBot, a declared automated visitor. A store may treat a declared bot differently from an ordinary shopper.
  • Agent-style models are not the consumer assistant apps, and these results do not predict what those apps would do.
  • Payment is not tested. Agents stop before any payment step and use a synthetic identity. Nothing is bought.

Want a report like this for your stores?

Request a pilot audit with the form below.

No install. We only visit your public site.

We use your email only to reply to this request, and we delete it after 12 months. Privacy notice

Prefer email? Write to hello@sendtilly.com.