How to use headless Chrome in serverless functions with a 50MB limit

I collect online web development tools on Tiny Helpers, and some parts of the site are automatic tool screenshots.

The site is deployed on Vercel, and screenshots are taken by a serverless function endpoint that spins up headless Chrome to take a picture. Here's an example screenshot URL: https://tiny-helpers.dev/api/screenshot/?url=https://css.land/lch/&ratio=1.

And this setup worked great until Vercel announced Node 14 end of life. And with Node 16 or 18 my initial setup with puppeteer and chrome-aws-lambda broke. Google searches weren't very helpful, so let me share how I got it to run again.

After switching to Node 16 and later version 18, I was hitting the Vercel function bundle limit. I don't know why the exact same setup grew in size but apparently, bundled serverless functions on Vercel can't exceed 50MB. If you want to include and ship a headless browser, this isn't a lot of data!

Replacing chrome-aws-lambda with @sparticuz/chromium

First, I had to recognize that chrome-aws-lambda received its last update two years ago. Even though I wasn't planning on doing more than taking a few screenshots, that's a lot of missed Chrome releases.

The @sparticuz/chromium package is the new and maintained alternative.

Bundling puppeteer/core and @sparticuz/chromium still exceeded 50MB

I followed the example docs, and guess what? It all worked fine locally until I deployed it again to Vercel, and the function bundle was 57MB. 57MB — sooooo close!

So what now?

Self-hosting Chromium to keep the function size low

Then I discovered that Kyle McNally (@Sparticuz) also provides a chromium-min package that doesn't include Chromium. The downside of it is that you then have to provide Chromium yourself.

const chromium = require('@sparticuz/chromium-min');
const puppeteer = require('puppeteer-core');

puppeteer.launch({
  args: [...chromium.args, '--hide-scrollbars', '--disable-web-security'],
  defaultViewport: chromium.defaultViewport,
  // you have to point to a Chromium tar file here 👇
  executablePath: await chromium.executablePath(
    `https://your-uploaded-chromium-pack.tar`
  ),
  headless: chromium.headless,
  ignoreHTTPSErrors: true,
});

Where do you get a self-hostable Chromium, though? Every new @sparticuz/chromium package release comes with a downloadable chromium-[version]-pack.tar. Thank you, Kyle!

Download the tar file, make it publicly available, and you're ready.

In my first attempt, I checked the tar file into the GitHub repository, and served it via Vercel's CDN. And this worked great, but it was burning through my entire monthly Vercel bandwidth limits in just a few days. I wasn't planning on upgrading to a "real-money" Vercel account for this. 😅

Another Chromium hosting solution had to be found. I evaluated Cloudflare, S3 and even smaller file storage solution, but it turned out that serving Chromium to my tiny serverless function consumed a massive bandwidth regardless the hosting provider.

So what was the solution at the end? Now I'm fetching the Chromium file directly from the GitHub CDN. And this setup works without any cost!

const chromium = require('@sparticuz/chromium-min');
const puppeteer = require('puppeteer-core');

async function getBrowser() {
  return puppeteer.launch({
    args: [...chromium.args, '--hide-scrollbars', '--disable-web-security'],
    defaultViewport: chromium.defaultViewport,
    executablePath: await chromium.executablePath(
      `https://github.com/Sparticuz/chromium/releases/download/v116.0.0/chromium-v116.0.0-pack.tar`
    ),
    headless: chromium.headless,
    ignoreHTTPSErrors: true,
  });
}

// then you can do something with the returned browser 
// in your serverless function 👇
// module.exports = async (req, res) => {
//   const browser = await getBrowser();
//   const page = await browser.newPage();
//   await page.goto(/* ... */);
//   ...
// }

The GitHub-hosted headless Chrome now runs in Vercel's serverless functions on Node.js 18. That's kind of wild setup and this journey was way too complicated, but yay! Here's the complete serverless function to take screenshots using headless Chromium. There are some issues with running it via the Vercel CLI, but for now, I'm happy that it works again on Tiny Helpers.

Let's see how long it'll work until it breaks again!