From 2e31aac9bfb90f3d93c139dc3763021bacec4d32 Mon Sep 17 00:00:00 2001 From: Greg Kilmer Date: Tue, 7 Jul 2026 15:32:56 -0400 Subject: [PATCH] Form Update --- .env.example | 24 +++-- docs/DEPLOYMENT.md | 61 ++++++++++- public/robots.txt | 94 +++++++++++++++++ src/components/ObfuscatedEmail.astro | 73 ++++++++++++++ src/components/professional/ContactForm.tsx | 106 +++++++++++++++++++- src/layouts/BaseLayout.astro | 5 + src/pages/api/contact.ts | 47 ++++++++- src/pages/contact.astro | 48 +++++++-- src/pages/professional/index.astro | 8 +- 9 files changed, 440 insertions(+), 26 deletions(-) create mode 100644 public/robots.txt create mode 100644 src/components/ObfuscatedEmail.astro diff --git a/.env.example b/.env.example index 78700e3..ab930f8 100644 --- a/.env.example +++ b/.env.example @@ -19,20 +19,30 @@ SITE_URL=https://gregkilmer.com # neither is set, the API returns 503 so you can wire delivery later. # SMTP (nodemailer). CONTACT_TO is the inbox that receives submissions. -# SMTP_HOST= -# SMTP_PORT=587 -# SMTP_USER= -# SMTP_PASS= -# CONTACT_TO=greg@gregkilmer.com -# SMTP_FROM="Greg's Site " +# Point this at your homelab SMTP relay (Postfix/Mailcow/etc.). The relay must +# have SPF/DKIM/DMARC configured or mail will land in spam. Leave SMTP_HOST blank +# to disable delivery (the API returns 503 until this is set). +SMTP_HOST= +SMTP_PORT=587 +SMTP_USER= +SMTP_PASS= +CONTACT_TO=contact@gregkilmer.com +SMTP_FROM="Greg's Site " -# Optional webhook fallback (JSON POST). +# Optional webhook fallback (JSON POST). Only used if SMTP_HOST is unset. # CONTACT_WEBHOOK_URL= # Honeypot / basic abuse control. Maximum submissions per IP per window. CONTACT_RATE_LIMIT=5 CONTACT_RATE_WINDOW_MIN=15 +# --- Cloudflare Turnstile (bot protection) ---------------------------------- +# Get free keys at: https://dash.cloudflare.com/?to=/:account/turnstile +# SITE_KEY is public (ships to the client); SECRET_KEY is server-only. +# Leave both blank to skip Turnstile entirely (e.g. local dev). +TURNSTILE_SITE_KEY= +TURNSTILE_SECRET_KEY= + # --- Gaming box-art tool ---------------------------------------------------- # Used by scripts/fetch-box-art.mjs to download box/banner art for Gaming Hall # of Fame entries via the RAWG API. Get a free key at: diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index 22d9a61..a5cbb58 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -25,9 +25,10 @@ delivery, set **either** SMTP vars **or** `CONTACT_WEBHOOK_URL` (see | `SITE_URL` | Canonical URL for metadata | `https://gregkilmer.com` | | `SMTP_HOST` / `SMTP_PORT` / `SMTP_USER` / `SMTP_PASS` | SMTP relay | unset | | `SMTP_FROM` | From address for outgoing mail | derived from `SITE_URL` | -| `CONTACT_TO` | Inbox that receives submissions | `greg@gregkilmer.com` | +| `CONTACT_TO` | Inbox that receives submissions | `contact@gregkilmer.com` | | `CONTACT_WEBHOOK_URL` | JSON webhook fallback (Discord/Slack/n8n) | unset | | `CONTACT_RATE_LIMIT` / `CONTACT_RATE_WINDOW_MIN` | Per-IP rate limit | `5` / `15` | +| `TURNSTILE_SITE_KEY` / `TURNSTILE_SECRET_KEY` | Cloudflare Turnstile bot protection | unset (skipped) | --- @@ -102,14 +103,72 @@ path: 3. **Neither configured**: returns `503` with a friendly "email me directly" message so the form never silently swallows input. +### SMTP relay DNS requirements (homelab) + +When pointing the form at a homelab SMTP relay (Postfix, Mailcow, etc.), the +relay itself must have proper DNS records or mail **will** land in recipients' +spam folders — this is the relay's responsibility, not the site's. Ensure the +sending domain (the host part of `SMTP_FROM`, or the relay's HELO name) has: + +- **SPF** (`TXT` on the domain): lists the relay's IP as an authorized sender. +- **DKIM** (`TXT` on `selector._domainkey`): signs outgoing messages so the + relay's identity can be verified. +- **DMARC** (`TXT` on `_dmarc`): tells receivers how to handle SPF/DKIM + failures (start at `p=none` to monitor, tighten to `quarantine` later). +- **rDNS / PTR** on the relay's public IP: must match the relay's HELO name. + +`SMTP_FROM` should use a domain you control (e.g. `no-reply@gregkilmer.com`), +not `gmail.com` — SPF on a domain you don't own will reject or sink the mail. + +Test delivery + spam score with [mail-tester.com](https://www.mail-tester.com/) +(send a form submission to the unique address it gives you). + +### Cloudflare Turnstile (bot protection) + +The form includes an invisible Cloudflare Turnstile challenge. To enable it: + +1. Create a widget at + [Cloudflare Turnstile](https://dash.cloudflare.com/?to=/:account/turnstile). + Use the **Managed** challenge mode (invisible, no puzzle). +2. Put the **site key** in `TURNSTILE_SITE_KEY` and the **secret key** in + `TURNSTILE_SECRET_KEY` (both in `.env`). +3. The site key is public and ships to the browser; the secret stays + server-side. The API verifies each token against Cloudflare's `siteverify` + endpoint before sending mail. + +If neither key is set, Turnstile is skipped entirely (useful for local dev — +the form works without a challenge). The honeypot + per-IP rate limit still +apply regardless. + Test it: ```bash +# With Turnstile configured, this is rejected (no/bad token) with 400: +curl -X POST http://localhost:4321/api/contact \ + -H "Content-Type: application/json" \ + -d '{"name":"Test","email":"test@example.com","message":"Hello there."}' + +# Without Turnstile (keys unset), this succeeds and delivers mail: curl -X POST http://localhost:4321/api/contact \ -H "Content-Type: application/json" \ -d '{"name":"Test","email":"test@example.com","message":"Hello there."}' ``` +### Anti-scraping + +The site applies a layered anti-scraping policy: + +- **`public/robots.txt`** allows legitimate search engines (Google, Bing) but + blocks known AI training bots (GPTBot, ClaudeBot, CCBot, Bytespider, etc.) + and disallows `/api/` and `/contact` for everyone. Note: robots.txt is + voluntary; determined scrapers ignore it. +- **`noindex` on `/contact`**: the contact page emits + `` so it won't be indexed + even if a bot reaches it. +- **Email obfuscation**: the Gmail address is never in the HTML source as + plaintext; it's base64-encoded and decoded to a `mailto:` client-side on + click. Dumb harvesters that grep for `@` or `mailto:` find nothing. + --- ## 5. Updating the site diff --git a/public/robots.txt b/public/robots.txt new file mode 100644 index 0000000..5cbe00c --- /dev/null +++ b/public/robots.txt @@ -0,0 +1,94 @@ +# ============================================================================= +# robots.txt — gregkilmer.com +# ============================================================================= +# Strict anti-scraping policy. +# +# - Google/Bing/etc. (legitimate search) are ALLOWED for SEO. +# - /api/ and /contact are disallowed for everyone (the form + endpoints +# should not be indexed or crawled). +# - Known AI training bots are blocked by User-Agent below. Note that +# robots.txt is voluntary; determined scrapers will ignore it. This is a +# first layer (paired with noindex on /contact and email obfuscation). +# ============================================================================= + +# --- Default policy: allow crawling of public pages, block the API + contact +User-agent: * +Allow: / +Disallow: /api/ +Disallow: /contact + +# --- Block known AI training / scraping bots explicitly --------------------- +# OpenAI +User-agent: GPTBot +Disallow: / + +User-agent: ChatGPT-User +Disallow: / + +User-agent: OAI-SearchBot +Disallow: / + +# Anthropic +User-agent: ClaudeBot +Disallow: / + +User-agent: Claude-Web +Disallow: / + +User-agent: anthropic-ai +Disallow: / + +# Google's AI/Vertex crawlers (separate from regular Googlebot, which is allowed) +User-agent: Google-Extended +Disallow: / + +# Common Crawl (feeds many AI training datasets) +User-agent: CCBot +Disallow: / + +# ByteDance / TikTok +User-agent: Bytespider +Disallow: / + +# Meta +User-agent: meta-externalagent +Disallow: / + +User-agent: Meta-ExternalAgent +Disallow: / + +User-agent: FacebookBot +Disallow: / + +# Amazon / Alexa +User-agent: Amazonbot +Disallow: / + +# Apple's AI/Applebot-Extended (regular Applebot remains allowed for Siri/Suggestions) +User-agent: Applebot-Extended +Disallow: / + +# Perplexity +User-agent: PerplexityBot +Disallow: / + +# Cohere +User-agent: cohere-ai +Disallow: / + +# Diffbot +User-agent: Diffbot +Disallow: / + +# Imagesift / OC-72 / other image scrapers +User-agent: ImagesiftBot +Disallow: / + +User-agent: omgili +Disallow: / + +User-agent: Omgilibot +Disallow: / + +# Sitemap (uncomment once a sitemap.xml is generated) +# Sitemap: https://gregkilmer.com/sitemap.xml diff --git a/src/components/ObfuscatedEmail.astro b/src/components/ObfuscatedEmail.astro new file mode 100644 index 0000000..efe51e2 --- /dev/null +++ b/src/components/ObfuscatedEmail.astro @@ -0,0 +1,73 @@ +--- +/** + * ObfuscatedEmail.astro + * --------------------------------------------------------------------------- + * Renders an email address as a clickable link without ever putting the + * plaintext address into the HTML source. The address is base64-encoded at + * render time and stored in a data attribute; a tiny inline script decodes it + * on click and builds the mailto: href at that moment. Scrapers that simply + * grep the HTML for `@` or `mailto:` see nothing useful. + * + * This is not cryptographically secure obfuscation (base64 is trivially + * reversible), but it defeats the vast majority of dumb harvesters that don't + * run JS. Combined with robots.txt + noindex on /contact, it raises the cost + * of harvesting enough to be worthwhile. + * + *