google-play-scraper

repository·main·Indexed 25 days ago

https://github.com/facundoolano/google-play-scraper

A Node.js library for scraping data from the Google Play Store. It provides methods to retrieve application details, reviews, search results, developer information, permissions, and data safety details. The library includes features for result memoization to reduce network requests and throttling to avoid rate limits.

Tokens
4.6K
Snippets
15
Records
20
Agent score
84%

What's inside google-play-scraper

  1. Overview of google-play-scraper

    main

    The google-play-scraper is a Node.js module designed to scrape application data from the Google Play store.

    Note: The maintainer is no longer actively maintaining this project except for reviewing community PRs. The parser may break if Google Play's layout changes.

  2. Cache results using memoization

    main

    To avoid redundant network requests to Google Play, you can use the memoized function. This returns a store object that caches results for each method.

    By default, each method caches up to 1000 values with a 5-minute expiration. You can customize the cache behavior using options supported by the memoizee module (e.g., maxAge).

    import {memoized as m} from "google-play-scraper"; // cache with default options
    const memoized = m(); // cache with customized options
    const memoizedCustom = m({ maxAge: 1000 * 60 }); // cache with customized options
    
    // first call will hit google play and cache the results
    memoized.developer({devId: "DxCo Games"}).then(console.log);
    
    // second call will return cached results
    memoized.developer({devId: "DxCo Games"}).then(console.log);
  3. Avoid rate limits with throttling

    main

    Making too many requests in a short period (especially when using fullDetail) can trigger Google Play's throttling limits, resulting in 503 errors or IP bans.

    To prevent this, you can pass a throttle property to any scraper method. This property defines the maximum number of requests allowed per second. Once the limit is reached, subsequent requests are held until the next second begins.

    import gplay from "google-play-scraper";
    
    // the following method will perform batches of 10 requests per second
    gplay.search({term: 'panda', throttle: 10}).then(console.log);
  4. Search for applications with search()

    main

    Use the search() method to find applications based on a search term.

    Options:

    • term (required): The search query.
    • num (optional): Number of apps to retrieve. Defaults to 20, max is 250.
    • lang (optional): Two-letter language code. Defaults to 'en'.
    • country (optional): Two-letter country code. Defaults to 'us'.
    • fullDetail (optional): If true, fetches full details for every result. Defaults to false.
    • price (optional): Filter by price. Values: 'all', 'free', or 'paid'. Defaults to 'all'.
    import gplay from "google-play-scraper";
    
    gplay.search({
        term: "panda",
        num: 2
      }).then(console.log, console.log);
  5. Retrieve application reviews with reviews()

    main

    Use the reviews() method to fetch a page of reviews for a specific application.

    Note on Volume: To get all reviews at once, set num to a high value (e.g., 5000). For paginated results (150 per page), set paginate: true.

    Options:

    • appId (required): Unique application ID.
    • lang (optional): Two-letter language code. Defaults to 'en'.
    • country (optional): Two-letter country code. Defaults to 'us'.
    • sort (optional): Sorting method. Use gplay.sort.NEWEST, gplay.sort.RATING, or gplay.sort.HELPFULNESS. Defaults to gplay.sort.NEWEST.
    • num (optional): Quantity of reviews. Defaults to 100. If paginate is true, this parameter is ignored.
    • paginate (optional): If true, returns a paginated response. Defaults to false.
    • nextPaginationToken (optional): The token used to fetch the next page. Defaults to null.
    import gplay from "google-play-scraper";
    
    // Single call for many reviews
    gplay.reviews({
      appId: 'com.dxco.pandavszombies',
      sort: gplay.sort.RATING,
      num: 3000
    }).then(console.log, console.log);
    
    // Paginated request (first page)
    gplay.reviews({
      appId: 'com.dxco.pandavszombies',
      sort: gplay.sort.RATING,
      paginate: true,
      nextPaginationToken: null
    }).then(console.log, console.log);
  6. Retrieve application lists with list()

    main

    Use the list() method to retrieve a collection of applications from Google Play.

    Options:

    • collection (optional): The Google Play collection to retrieve. Defaults to gplay.collection.TOP_FREE.
    • category (optional): Filter by app category. Use gplay.category constants.
    • age (optional): Age range filter (only for FAMILY and subcategories). Options: age.FIVE_UNDER, age.SIX_EIGHT, age.NINE_UP.
    • num (optional): Number of apps to retrieve. Defaults to 500.
    • lang (optional): Two-letter language code. Defaults to 'en'.
    • country (optional): Two-letter country code. Defaults to 'us'.
    • fullDetail (optional): If true, makes an extra request for every app to fetch its full details. Defaults to false.
    import gplay from "google-play-scraper";
    
    gplay.list({
        category: gplay.category.GAME_ACTION,
        collection: gplay.collection.TOP_FREE,
        num: 2
      })
      .then(console.log, console.log);
  7. Get similar applications with similar()

    main

    Use the similar() method to find a list of applications similar to a specified one.

    Options:

    • appId (required): The Google Play ID of the target application.
    • lang (optional): Two-letter language code. Defaults to 'en'.
    • country (optional): Two-letter country code. Defaults to 'us'.
    • fullDetail (optional): If true, fetches full details for every similar app. Defaults to false.
    import gplay from "google-play-scraper";
    
    gplay.similar({appId: "com.dxco.pandavszombies"}).then(console.log);
  8. Get application permissions with permissions()

    main

    Use the permissions() method to retrieve the list of permissions an application has access to.

    Options:

    • appId (required): The Google Play ID of the application.
    • lang (optional): Two-letter language code. Defaults to 'en'.
    • country (optional): Two-letter country code. Defaults to 'us'.
    • short (optional): If true, returns only the permission names as strings instead of permission/description objects. Defaults to false.
    import gplay from "google-play-scraper";
    
    gplay.permissions({appId: "com.dxco.pandavszombies"}).then(console.log);
  9. Get application data safety with datasafety()

    main

    Use the datasafety() method to retrieve an application's data safety information, including data shared, data collected, security practices, and the privacy policy URL.

    Options:

    • appId (required): The Google Play ID of the application.
    • lang (optional): Two-letter language code. Defaults to 'en'.
    import gplay from "google-play-scraper";
    
    gplay.datasafety({appId: "com.dxco.pandavszombies"}).then(console.log);
  10. Retrieve application details with app()

    main

    Use the app() method to fetch full details for a specific Google Play application.

    Options:

    • appId (required): The Google Play ID (e.g., the ?id= parameter from the URL).
    • lang (optional): Two-letter language code. Defaults to 'en'.
    • country (optional): Two-letter country code. Defaults to 'us'. Useful for apps available only in specific regions.
    import gplay from "google-play-scraper";
    
    gplay.app({appId: 'com.google.android.apps.translate'})
      .then(console.log, console.log);