Skip to main content
In this guide, you’ll learn how to scrape Hacker News using scripts and powerful AI agents to automate with Hyperbrowser’s cloud based browsers.

Prerequisites

Get Your API Key

Sign up and get your API key from the Hyperbrowser Dashboard.
export HYPERBROWSER_API_KEY=<your-api-key>

Install Dependencies

  • Playwright
  • Puppeteer
npm install @hyperbrowser/sdk playwright-core dotenv

Part 1: Automate with Playwright or Puppeteer Scripts

Let’s start by scraping the top story from Hacker News using Playwright or Puppeteer. This shows how you can launch Hyperbrowser sessions and connect to them with automation libraries like Playwright or Puppeteer.
  • Playwright
  • Puppeteer
import { Hyperbrowser } from "@hyperbrowser/sdk";
import { chromium } from "playwright-core";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

async function getTopStory() {
  // Create a cloud browser session
  const session = await client.sessions.create();
  console.log("Session created:", session.id);

  try {
    // Connect Playwright to the cloud browser
    const browser = await chromium.connectOverCDP(session.wsEndpoint);
    const context = browser.contexts()[0];
    const page = context.pages()[0];

    // Navigate to Hacker News
    await page.goto("https://news.ycombinator.com/");

    // Extract the first story title
    const topStory = await page.evaluate(() => {
      const titleElement = document.querySelector(".titleline > a");
      return titleElement ? titleElement.textContent : null;
    });

    console.log("Top Story:", topStory);
    return topStory;
  } catch (err) {
    console.error(`Encountered error: ${err}`);
  } finally {
    // Clean up the session
    await client.sessions.stop(session.id);
  }
}

getTopStory().catch(console.error);

Part 2: AI-Powered Automation

Now let’s solve the same problem using HyperAgent, our AI-powered browser automation tool. Instead of writing selectors and navigation logic, you simply describe what you want in natural language.
import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

async function getTopStoryWithAgent() {
  const result = await client.agents.hyperAgent.startAndWait({
    task: "Go to Hacker News and get the title of the first post",
  });

  console.log("Top Story:", result.data?.finalResult);
  return result.data?.finalResult;
}

getTopStoryWithAgent().catch(console.error);

The Difference

With HyperAgent, you don’t need to:
  • Write CSS selectors
  • Handle page navigation
  • Manage browser state
  • Debug DOM changes
Just describe what you want, and the AI handles the rest. It’s perfect for complex workflows, dynamic sites, and rapid prototyping.
Hyperbrowser also integrates with other popular AI agent frameworks including BrowserUse, Claude Computer Use, Gemini Computer Use, and OpenAI CUA. All agents benefit from Hyperbrowser’s cloud infrastructure, stealth capabilities, and proxy support.

Next Steps

Learn more about managing browsers sessions and other capabilities:

Additional Resources

Browser Integrations

SDKs

AI Agent Integrations