israeli-bank-scrapers

To use the scrapers in a standard Node.js environment, install the default package via npm. This version includes puppeteer, which automatically manages the installation of a local Chromium instance.

npm install israeli-bank-scrapers --save

The israeli-bank-scrapers-core package is designed for environments like Electron where you want to manage Chromium manually using puppeteer-core.

Steps to use core:

1. Install the core package: npm install israeli-bank-scrapers-core --save.
2. Determine the required Chromium revision using getPuppeteerConfig().chromiumRevision.
3. Provide the absolute path to your local Chromium instance using the executablePath option in createScraper.

import { getPuppeteerConfig, createScraper } from 'israeli-bank-scrapers-core';

// 1. Get required revision
const chromiumVersion = getPuppeteerConfig().chromiumRevision;

// 2. Use in scraper (assuming you have downloaded chromium at this path)
const options = {
  companyId: CompanyTypes.leumi,
  executablePath: '/path/to/your/chromium',
  // ... other options
};
const scraper = createScraper(options);

To avoid sharing cookies between parallel scraper runs (e.g., different users), provide a Puppeteer browserContext in the options instead of a full browser instance.

import puppeteer from 'puppeteer';
import { CompanyTypes, createScraper } from 'israeli-bank-scrapers';

const browser = await puppeteer.launch();
const browserContext = await browser.createBrowserContext();
const options = {
  companyId: CompanyTypes.leumi,
  startDate: new Date('2020-05-01'),
  browserContext
};
const scraper = createScraper(options);
const scrapeResult = await scraper.scrape({ username: 'vr29485', password: 'sometingsomething' });
await browser.close();

To control the browser lifecycle yourself (e.g., using an existing Puppeteer instance), pass a browser object in the options. Use skipCloseBrowser: true to prevent the library from automatically closing the browser when finished.

import puppeteer from 'puppeteer';
import { CompanyTypes, createScraper } from 'israeli-bank-scrapers';

const browser = await puppeteer.launch();
const options = {
  companyId: CompanyTypes.leumi,
  startDate: new Date('2020-05-01'),
  browser,
  skipCloseBrowser: true,
};
const scraper = createScraper(options);
const scrapeResult = await scraper.scrape({ username: 'vr29485', password: 'sometingsomething' });
await browser.close();

The possibleResults property in LoginOptions allows you to map the resulting URL (or other state) to a LoginResults type. This is critical for the scraper to know if it should proceed or report an error like InvalidPassword or ChangePassword.

Supported condition types for matching:

1. String: Case-insensitive exact match against the URL.
2. RegExp: A regular expression tested against the URL.
3. Function: An async function (options: { page: Page, value: string }) => Promise<boolean> that allows for complex logic using the Puppeteer Page instance.

possibleResults: {
  'SUCCESS': ['https://bank.com/home'],
  'InvalidPassword': [/login\?error=wrong_pass/],
  'ChangePassword': async ({ page }) => {
    const url = page.url();
    return url.includes('reset-password');
  }
}

The Max scraper maps various internal plan names to standard TransactionTypes. Common mappings include:

• Normal: Includes Normal, ImmediateCharge, MonthlyCharge, InternetShopping, MonthlyCardFee, etc.
• Installments: Includes Installments, Credit, and CreditOutsideTheLimit.

If a transaction type cannot be mapped via the plan name, the scraper falls back to checking the planTypeId (where IDs 2 and 3 are treated as Installments).

The BaseScraperWithBrowser class manages a Puppeteer browser instance. You can control how the browser is launched via the scraper's options:

• Existing Browser: Provide browser or browserContext in the options to reuse an existing instance.
• Headless Mode: Controlled by showBrowser. If showBrowser is false, the browser runs in headless mode.
• Customization: Use prepareBrowser to run logic on the browser instance upon launch, and preparePage to run logic on the specific page instance after it is created.
• Cleanup: The scraper automatically handles closing the page and the browser via the terminate method. If terminate is called with success: false, you can configure storeFailureScreenShotPath in the options to save a screenshot of the failure state.

When initializing a MaxScraper, you can pass several options to control the data extraction behavior:

• startDate (Date): The starting point for transaction scraping. Defaults to 1 year ago.
• combineInstallments (boolean): If true, installments are grouped/fixed. If false, they are treated as individual transactions.
• includeRawTransaction (boolean): If true, the resulting Transaction object will include the rawTransaction field containing the original unmapped data.
• futureMonthsToScrape (number): How many months into the future to attempt scraping (defaults to 1).
• outputData.enableTransactionsFilterByDate (boolean): Controls whether transactions older than the startDate are filtered out (defaults to true).