Unflare

Unflare is a Node.js API service designed to bypass Cloudflare protection. It uses puppeteer-real-browser to automatically solve Cloudflare challenges.

Instead of your script interacting with the challenge page directly, you send a request to Unflare. It handles the challenge and returns the necessary cf_clearance token, session cookies, and browser headers. You can then use these returned values in your own subsequent requests to access the target website without being blocked by Cloudflare.

The preferred way to run Unflare is via Docker, as the image includes all necessary dependencies (including those required for browser automation).

Build and Run manually

Use these commands to build the image locally and run it on port 5002:

docker build -t unflare .
docker run -p 5002:5002 unflare

Use Docker Compose

You can also use the pre-built image from GitHub Container Registry:

services:
  unflare:
    image: ghcr.io/iamyegor/unflare
    ports:
      - "5002:5002"

The ServiceResponse<T> class defines a standardized API response format used throughout the service. Every response contains a success flag, a descriptive message, the actual data payload (if any), and an HTTP status code.

Fields

• success: A boolean indicating if the operation was successful.
• message: A string providing context or error details.
• responseObject: A generic payload of type T. This is typically present in successful responses and may be optional or null in failure scenarios.
• statusCode: The HTTP status code associated with the response (e.g., 200 for OK, 400 for Bad Request).

You can deploy the Unflare service using Docker Compose. By default, the service uses the image ghcr.io/iamyegor/unflare and exposes port 5002 on the host machine.

services:
  unflare:
    image: ghcr.io/iamyegor/unflare
    ports:
      - "5002:5002"

You can integrate Unflare into your JavaScript applications using the fetch API. This example demonstrates how to request cookies and headers for a target URL, including proxy configuration.

const res = await fetch('http://localhost:5002/scrape', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    url: 'https://example.com',
    timeout: 60000,
    proxy: {
      host: "proxy.example.com",
      port: 8080,
      username: "user",
      password: "pass"
    }
  })
});

const { cookies, headers } = await res.json();
// Use cookies/headers for next requests

Unflare uses environment variables for configuration, loaded via dotenv based on the NODE_ENV value. The application expects a .env.${NODE_ENV} file. The configuration is validated using envalid and is exported as the env object.

If running in a non-test environment, the following variables must be provided or will use their respective defaults:

# Example .env.development file
NODE_ENV=development
HOST=0.0.0.0
PORT=8080
CORS_ORIGIN=http://localhost:8080
COMMON_RATE_LIMIT_MAX_REQUESTS=500
COMMON_RATE_LIMIT_WINDOW_MS=60000

You can instantiate standardized responses using the static success and failure methods.

• ServiceResponse.success(message, responseObject, statusCode?): Creates a response with success: true. The statusCode defaults to StatusCodes.OK (200).
• ServiceResponse.failure(message, responseObject, statusCode?): Creates a response with success: false. The statusCode defaults to StatusCodes.BAD_REQUEST (400).

import { ServiceResponse } from './path-to-serviceResponse';
import { StatusCodes } from 'http-status-codes';

// Successful response
const okResponse = ServiceResponse.success('Operation completed', { id: 123 });

// Failure response with custom status
const errorResponse = ServiceResponse.failure('Invalid input', null, StatusCodes.UNPROCESSABLE_ENTITY);

To validate the structure of a ServiceResponse (for example, when testing or consuming the API), use the ServiceResponseSchema helper. This function takes a Zod schema representing the expected responseObject and returns a Zod object schema for the entire response.

Note that responseObject is marked as .optional() in the resulting schema.

import { z } from 'zod';
import { ServiceResponseSchema } from './path-to-serviceResponse';

const UserSchema = z.object({
  id: z.string(),
  name: z.string(),
});

// Creates a schema that validates a ServiceResponse containing a User
const ResponseValidator = ServiceResponseSchema(UserSchema);

const result = ResponseValidator.parse({
  success: true,
  message: 'User found',
  responseObject: { id: '1', name: 'John Doe' },
  statusCode: 200,
});

The scraperRouter is an Express router that exposes endpoints for scraping tasks. Currently, it provides a single POST endpoint /scrape which is handled by the scrapeClearance controller. This router is intended to be mounted within an Express application to enable scraping capabilities.

import express from 'express';
import { scraperRouter } from './src/scraperRouter';

const app = express();
app.use(express.json());

// Mount the scraper router
app.use('/api', scraperRouter);

app.listen(3000, () => console.log('Scraper API running on port 3000'));

To use the scrapeClearance endpoint, you must provide a request body that conforms to the ScrapeClearanceRequestSchema. This schema defines how the scraper navigates to a URL and handles proxies and request methods.

Request Fields:

• url (string, required): The target URL to navigate to.
• timeout (number, optional): The timeout in milliseconds for the operation. Defaults to 60000 if not specified.
• proxy (object, optional): A proxy configuration object.
• method (enum, optional): The HTTP method to use. Supported values are "GET" and "POST". Defaults to "GET".
• data (object, optional): A record of unknown values used for POST requests.

{
  "url": "https://example.com",
  "timeout": 30000,
  "proxy": {
    "host": "proxy.example.com",
    "port": 8080,
    "username": "user",
    "password": "pass"
  },
  "method": "GET"
}