{{ site.title }}

{{ theme.darkModeSwitchLabel || 'Appearance' }}

PMTiles Migration Design ​

Summary ​

Migrate Marandoo tile overlay from 138K scattered PNG files (1.7GB) on R2 to a single .pmtiles file with WebP compression. Cloudflare Worker handles PMTiles unpacking — front-end unchanged.

Architecture ​

Google Maps ImageMapType
  → GET https://map-tiles.fudong.workers.dev/{site}/{z}/{x}/{y}.png
  → Cloudflare Worker (pmtiles JS library)
  → R2: {site}.pmtiles (HTTP Range Request)
  → Response: WebP tile data

R2 Storage ​

# Before
map-tiles/marandoo/14/8683/4566.png  (138K files, 1.7GB)

# After
map-tiles/marandoo.pmtiles           (single file, ~200-300MB)

Multi-site Support ​

Worker maps {site} in URL to {site}.pmtiles in R2 automatically. Adding a new site = upload one .pmtiles file.

Local Conversion Pipeline ​

PNG tiles → WebP tiles (cwebp -q 80) → MBTiles (mb-util) → PMTiles (pmtiles convert)

Tools Required ​

  • cwebp — brew install webp
  • mb-util — pip install mbutil
  • pmtiles — npm install -g pmtiles

Conversion Script ​

scripts/convert-tiles-to-pmtiles.sh — reusable for future sites.

Parameters: tiles directory path, site name.

Steps:

  1. Batch convert PNG → WebP (quality 80) preserving directory structure
  2. Pack into MBTiles via mb-util (TMS scheme, --image_format=webp)
  3. Convert MBTiles → PMTiles via pmtiles convert

Cloudflare Worker Changes ​

File: workers/map-tiles/src/index.ts

Key Changes ​

  1. Add pmtiles npm dependency
  2. Implement custom R2Source class (maps getRange(offset, length) to R2 get() with range option)
  3. Parse URL /{site}/{z}/{x}/{y}.png → lookup tile in {site}.pmtiles
  4. Cache PMTiles header/directory via Cloudflare Cache API
  5. Cache tile responses with Cache-Control: max-age=31536000, immutable

CORS ​

typescript
const ALLOWED_ORIGINS = [
  'https://dashboard.dustac.com.au',
  'http://localhost:3000',
];

Check Origin header, return matching origin or no CORS header.

Backward Compatibility ​

If {site}.pmtiles not found in R2, fall back to scattered file path {site}/{z}/{x}/{y}.png. Allows incremental migration per site.

Front-end ​

No changes. overlayConfig.ts keeps existing tileBasePath and URL format.

Update maxZoom from 18 to 20 to use all available zoom levels (tiles exist for 14–20).

Deployment ​

  1. Run conversion script locally → marandoo.pmtiles
  2. Upload: wrangler r2 object put map-tiles/marandoo.pmtiles --file=marandoo.pmtiles
  3. Deploy Worker: cd workers/map-tiles && wrangler deploy
  4. Verify with curl
  5. (Optional) Clean up scattered files from R2

Rollback ​

Delete marandoo.pmtiles from R2 → Worker falls back to scattered files. Or rollback Worker version via Cloudflare dashboard.

Adding Future Sites ​

  1. Prepare tiles directory
  2. Run ./scripts/convert-tiles-to-pmtiles.sh ./tiles {site-name}
  3. Upload {site-name}.pmtiles to R2
  4. Add entry in overlayConfig.ts