Security

What the SDK sends us

LLMJury stores what your SDK sends and nothing more. There is no hidden collection — but “what your SDK sends” depends on which of the two integration paths you take, so the third row below is the one to read closely.

Exposure events

That a user identifier you supplied was assigned to a variant, and when. The identifier is whatever string your code passes — we never ask for and cannot derive a real-world identity from it.

Operational measurements

Response latency, token counts, and computed cost for the model calls the SDK wraps.

Prompt and response text — if you use wrap()

The wrap() / intercept_model_call path attaches the prompt your application sent and the text your model returned to each model_call event, and we store both raw for your plan’s retention window. That happens whether or not LLM-as-judge grading is switched on: grading reads the text, it is not what causes it to be sent. If you do not want us to hold model text, use the key-only path below instead.

Business events

Whatever you explicitly send via client.track(…) — typically a conversion flag or a revenue amount.

Two integration paths, two answers

Whether we ever see a prompt is a decision you make in your own code, not a plan setting. Both paths below are supported in all three SDKs and documented in full.

Key-only — no model text leaves your process

get_prompt / get_variables / assign resolve a user to their variant from client memory, and track(…) sends the business outcome. Nothing in that path carries a prompt or a response, so we never receive either. You keep prompt versioning, deterministic assignment, and business metrics; you give up judge-scored quality metrics, which need the text to score.

Interception — model text reaches us

wrap(provider_client, experiment) observes your model calls and records the prompt, the response, latency, tokens, model, and errors with no call-site code. This is the path that unlocks LLM-as-judge quality scoring, and the one that puts your prompt and response text in our storage.

Hash-only storage and per-org redaction of model text are a known gap, not a feature we have. They are on the roadmap and deliberately out of scope for this version — named here rather than left for you to discover. Until they ship, the key-only path is the answer for text you cannot send us. The per-SDK detail is in the documentation.

And what it does not

• —Nothing on assignment. Variant assignment is a local MurmurHash3 computation inside the SDK — no network call is made on your request path, so an identifier only reaches us attached to an event you chose to send.
• —No fingerprinting, no device or browser telemetry, and no third-party tracker inside the SDKs. They are open source on PyPI, npm, and Maven Central, so that is a claim you can verify rather than accept.
• —No model calls of your own are proxied through us. Your provider client talks to your provider directly; the wrapper observes it.

How much personal data reaches us is your decision. We cannot see inside your application, so we cannot strip personal data out of a prompt you send. Pass a pseudonymous user identifier rather than an email address, and redact what you would not want graded.

How it is protected

Sub-processors

The vendors that touch customer data today. This is the same list the privacy policy publishes — one source, so the two cannot disagree.

Amazon Web Services

Application hosting, databases, backups, and outbound email (SES), in the United States.

Cloudflare

DNS, CDN, and static hosting for this website and the documentation site.

Clerk

Authentication and organization membership — holds your name, email, and org role.

Stripe

Subscription billing. Card details go to Stripe directly; LLMJury never receives or stores them.

Anthropic

The LLM-as-judge model. Sampled outputs are sent for grading under an API agreement that excludes training on submitted data.

Calendly

The demo scheduler embedded on the /demo page only. Loading that page loads Calendly’s script; no other page does.

Getting your data out, or gone

Email support@llmjury.com and say what you want — export, correction, or deletion. No form and no legal basis required. We do it within 30 days and confirm when it is done. The full set of rights under the GDPR, UK GDPR, and CCPA/CPRA is in the privacy policy.