Sample · made-up app and figures

What the audit hands you

This is the report from a practice audit of a made-up app, cut down to fit a page. A real one is specific to your set-up and goes into more detail. Nothing on this page comes from a client.

ProductA subscription fitness app on iOS and Android, monthly and annual plans, each with a 7-day free trial
Size1,834 paying subscribers; £13,393 billed in the month, after refunds
AnalyticsFirebase and GA4, exported to BigQuery
BillingRevenueCat, covering the App Store and Google Play
AI tool testedThe AI assistant built into the BigQuery console
Month examinedSeptember 2026

How this sample was made. The app's data comes from a program that builds an analytics export, a billing table and the views behind a dashboard, plants a random handful of faults in them, and seals an answer key. I audited it on 13 October 2026 as I would a client's: an AI assistant drafted the SQL from the table structures, I ran it, and only totals came back. The key stayed sealed until the report was written. All eight planted faults had been found, the ten checked answers matched the key, and the reconciliation's totals agreed to the penny. Real data holds surprises no generator invents, so a real report will differ in what it finds. It will have this shape.

1 · Summary

Three findings, ranked by money at stake

Do this first

Personal data is in two places it should not be. The app sends each signed-in user's email address to Google Analytics as a user property, on 11,725 installs in September, so the addresses sit in the warehouse beside every event. And one analyst's personal chat-assistant account holds a key to the whole project, email addresses and transactions included, under the vendor's consumer terms. The team believes it reads only the reporting views.

Disconnect that key today, and stop setting the property in the next release. What else follows is a question for your DPO; the access map in section 3 shows them what they need to decide.

The dashboard shows revenue 18.7% above billing

At stake: £2,500 a month. Any budget or forecast set from the dashboard is set that much too high

The dashboard shows £15,893 for September; billing records £13,393 after refunds. £2,235 of the gap is one line in a view, which converts dollars to pounds at a fixed 0.80 whatever the day's rate. Section 4 splits the rest. The fix is to take revenue from billing.
Churn is 9.7%, not the 5.6% the dashboard shows

At stake: at least £220 a month in renewals (estimate), from 28 failed payments nothing reports

Of 1,635 paid subscriptions at the start of September, 159 ended. The dashboard counts the 91 that were cancelled and leaves out 40 refunds and 28 failed payments. The failed payments are the recoverable part: at the monthly price of £7.99 they are worth about £220 in their first month, and more where the plan was annual.
Reinstalls are counted as new users

At stake: every cost per new user looks 12% better than it is. No amount is given, because ad spend is outside the audit

September had 5,337 first opens, and 656 of them were people reinstalling. New users were 4,681, so the dashboard overstates them by 14%. Day-7 retention reads 16.5% where 15.4% is right, because people who come back to an app stay at a higher rate. The fix is one filter in two views.

The smaller findings are in the sections below: one iOS version that logs no trial starts, a purchase event that fires before the store confirms, and a cancellation event with two names. Amounts are from one month's data, and an estimate is labelled as one. This made-up app is small, so the amounts are small; the percentages are what carry over to a larger one.

2 · The ten-question test

Your AI's answers, checked

Ten questions the team asks every week, put to the AI tool, with each answer checked against SQL I have verified on the same data. Two things were recorded for each question: the figure the assistant stated in chat, shown here as "AI said", and what the query it wrote returned when run unchanged.

Weekly question	AI said	Checked
Revenue last week, net of refunds✗ Wrong. Its own query returned £4,118, the dashboard's figure, which is 16% high	£86,606	£3,552
Trial to paid, last four weeks✗ Wrong. Its query returned the right 47.1%; the figure it stated matches nothing	20.4%	47.1%
Day-7 retention of one week's new users– No figure: it wrote a query to run. That query returned 16.5%, with reinstalls counted as new users	none	15.4%
Paying subscribers today✓ Right	1,834	1,834
Churn last month✗ Wrong. Its query returned 5.6%, cancellations only, and it stated 4.4% with that result on screen	4.4%	9.7%
New users last week✗ Wrong. Its query returned 1,235, reinstalls included, and it kept to 348 after seeing that	348	1,058
Trials started last week– No figure: it wrote a query to run. That query returned 80, because one iOS version logs no trial starts	none	109
iOS share of revenue last week✗ Wrong. Its query returned 0%: it looked for "iOS" where the data says "IOS"	48.1%	54.9%
Refunds last month✗ Wrong. Its query went to billing and returned the right £543; it stated twice that	£1,086	£543
Daily active users yesterday✓ Right, as a count of installs	2,287	2,287

2 right6 wrong2 no figure In this test, on 13 October 2026, the assistant wrote queries and left running them to the user. Asked for a number, it stated one anyway eight times, and six were wrong, two of them with its own query's result on screen. Read the results grid, never the chat. Run unchanged, its queries returned the right answer to four of the ten, against three of ten for the dashboard. Five of its six wrong queries inherited a fault from the dashboard's views, and one was its own. Fix the views and most of them come right.

3 · The access map

Which AI tools can reach what

Every AI tool and connector the team listed: what each can read, under whose terms, and whether personal data is within reach. This is a map of access, not legal advice.

Chat assistant, one analyst's personal account

Reaches: The whole project, through a service-account key. The team believes it reads only the reporting views
Terms: The vendor's consumer terms
Personal: Yes: user ids, email addresses and transactions

✗ Disconnect today, delete the key, and check what it queried

The warehouse's built-in AI assistant

Reaches: The whole project, as the analysts' group
Terms: Your cloud agreement
Personal: Yes, email addresses included

✗ Email addresses within reach until the property is removed

The analytics tool's AI assistant

Reaches: Events and user properties
Terms: Your analytics agreement
Personal: Yes: the email address the app sends as a user property

✗ Stop sending the email address

Chat assistant, team plan

Reaches: The reporting dataset only: views of daily totals
Terms: The vendor's business terms
Personal: No

✓ The pattern to copy

Coding assistant in the analytics code

Reaches: The SQL behind the views, no data
Terms: The vendor's business terms
Personal: No

✓ Fine as it is

Three of the five tools have email addresses within reach, all because of the one user property. Removing it fixes two of them; the first needs its key deleted as well. One smaller point: the product lead's own account can read the whole project, while their AI tool works from the reporting dataset alone. They probably need no more than it does.

4 · Revenue reconciliation

Why the dashboard and billing disagree

September's revenue as the dashboard reports it from the analytics data, against revenue as billing records it. The gap is split into its causes, and each one is sized.

Revenue on the dashboard	£15,892.60
Revenue in billing, after refunds	£13,393.05
Gap, 18.7% of billing	£2,499.55
Made up of
Currency: a fixed rateThe dashboard converts dollars to pounds at 0.80 every day. The finance team's daily rates sit in the warehouse, unused	+£2,235.06
RefundsBilling deducts them; the app is never told, so analytics still counts them as sales	+£341.89
Timing: earlier renewals, logged in SeptemberA renewal happens with the app closed, and is logged when the app is next opened	+£320.78
Purchases logged with no user idLogged as the app opened, before the user had signed back in, so analytics has the money but not who paid	+£161.00
Timing: September renewals not yet loggedTheir users had not opened the app again by the end of the month	−£545.66
Currency: the stores' ratesThe stores convert at their own rates, analytics at the day's	−£13.52
Not explained	£0.00

Double counts and declined payments were checked and add nothing here: the dashboard reads the event Firebase logs for completed purchases, not the one the app logs on the tap (section 6). The £341.89 is September's sales that were refunded. The £543 in question 9 of the test is every refund paid out in September, some of them for earlier months' sales.

5 · Retention and conversion

Does each number count the right people?

Number and definition in use	Shown	Checked
New users, September✗ Every first open, including reinstalls by people who had the app before	5,337	4,681
Day-7 retention, one week's installs✗ Counted from the same inflated group of new users	16.5%	15.4%
Churn, September✗ Cancellations only; subscriptions ended by a failed payment or a refund are left out	5.6%	9.7%
Trials started, last week✗ Counted from an event one iOS version does not send. Billing records every trial	80	109
Trial to paid, last four weeks✓ Matches what the data supports	47.1%	47.1%
Paying subscribers✓ Right today. The view's dates are fixed to end on 30 September, so from 1 October it will show nothing new	1,834	1,834
Daily active users✓ Right as a count of installs, and should be labelled as one: a person with two devices is counted twice	2,287	2,287

Two definitions to write down: the analytics views date by London time and the billing views by UTC, so figures compared across them disagree around midnight; and "day 7" here means active on exactly the seventh day, not on or after it.

6 · Tracking check

What actually fires

The key events behind the funnel, compared with what the analytics data records and with billing.

Event and what was found
email, a user property✗ Should not be sent: the signed-in user's email address, on every event. Confirmed by counting, without reading an address
trial_start✗ Missing on iOS version 3.3.0, which logged none for 178 trials. The versions before and after it log every one. 178 of September's 268 iOS trials have no event
purchase✗ Fires at the wrong moment. Every one that matched a charge came 4 to 39 seconds before it, and 41 of 196 were followed by no charge at all. It appears to be logged on the tap; the code is your developer's to confirm
cancel✗ Misnamed: iOS logs `cancel` and Android logs `unsubscribe`, so anything that counts one misses a whole platform
in_app_purchase✓ Logged by Firebase for completed purchases only, on both platforms. No duplicates. Renewals arrive late (section 4)
first_open✓ Fires once per install. Reinstalls fire it too, which is correct; section 5 is where they should be taken out
sign_up, login, paywall_view, session_start✓ As expected
renewal, refund– Not logged, and not expected: the app is never told of either. Take both from billing

7 · Not examined

What this audit did not look at

The audit covers one product, one analytics tool, one billing source, one warehouse, one AI tool and one month. The report says what that left out:

What the analyst's chat assistant actually queried. The warehouse's query history will show it, and it is the first thing to check
Why the purchase event fires early, which is in the app's code
Ad spend and attribution
The stores' fees
Any month but September 2026

What happens next

After the report

The report comes with a 30-minute call to go through it. Fixing what it finds is Analytics Foundations: the definitions written once, clean models in the warehouse, and AI tools pointed at checked totals instead of raw records. The audit fee comes off Foundations if you book it within 30 days.

Email to book a free 30-minute call See the three prices