google-search-results-python

repository·master·Indexed 20 days ago

https://github.com/serpapi/google-search-results-python

A Python wrapper for SerpApi that scrapes and parses search results from engines like Google, Bing, Baidu, and others into standardized JSON or dictionary formats. It provides specialized classes such as GoogleSearch and SerpApiClient, supports asynchronous batch searches, pagination via generators, and the Search Archive API. Note: This package is deprecated in favor of serpapi-python.

Tokens
2.7K
Snippets
11
Records
14
Agent score
23%

What's inside google-search-results-python

  1. Configure SerpApi search parameters

    master

    You can control your search using a dictionary of parameters passed to the GoogleSearch class. Common parameters include:

    • q: The search query.
    • location: The location for the search.
    • device: The device type (desktop, mobile, or tablet).
    • hl: Google UI language.
    • gl: Google country.
    • safe: Safe Search flag.
    • num: Number of results.
    • start: Pagination offset.
    • api_key: Your SerpApi key.
    • tbm: Search type (e.g., nws for news, isch for images, shop for shopping).
    • tbs: Custom search criteria.
    • async: Set to true or false to enable/disable asynchronous requests.
    • output: Output format (json or html).

    You can also override parameters after initialization by modifying the search.params_dict dictionary.

    params = {
      "q": "coffee",
      "location": "Location Requested", 
      "device": "desktop|mobile|tablet",
      "hl": "Google UI Language",
      "gl": "Google Country",
      "safe": "Safe Search Flag",
      "num": "Number of Results",
      "start": "Pagination Offset",
      "api_key": "Your SerpApi Key", 
      "tbm": "nws|isch|shop", 
      "tbs": "custom to be search criteria",
      "async": "true|false",
      "output": "json|html"
    }
    
    search = GoogleSearch(params)
    # override an existing parameter
    search.params_dict["location"] = "Portland"
  2. Enable specific Google search types using the tbm parameter

    master

    To switch between different Google services (like Images, News, or Shopping), you must set the tbm (to be matched) parameter in your search request. If no tbm parameter is provided, the library performs a regular Google search.

    Supported tbm values include:

    • isch: Google Images API
    • nws: Google News API
    • shop: Google Shopping API

    Other Google services are generally supported out of the box by passing their respective identifiers to the tbm field.

    # Example conceptual usage
    params = {
        "q": "search query",
        "tbm": "isch"
    }
    # This would trigger a Google Images search
  3. Set the SerpApi key

    master

    You can provide your API key in two ways:

    1. Globally: Set it on the GoogleSearch class to apply to all instances.
    2. Per Search: Pass it directly in the parameters dictionary for a specific query.
    # Global setting
    GoogleSearch.SERP_API_KEY = "Your Private Key"
    
    # Per-search setting
    query = GoogleSearch({"q": "coffee", "serp_api_key": "Your Private Key"})
  4. Quick start with GoogleSearch

    master

    To perform a Google search, import the GoogleSearch class from serpapi. Initialize it with a dictionary containing your search parameters (such as q for query and location) and your api_key. Call .get_dict() to execute the search and retrieve the results as a Python dictionary.

    from serpapi import GoogleSearch
    search = GoogleSearch({"q": "coffee", "location": "Austin,Texas", "api_key": "secretKey"})
    result = search.get_dict()
  5. Perform Batch Asynchronous Searches

    master

    To handle large batches of queries efficiently, use the async: True parameter. This makes the request non-blocking: the client sends the query and immediately moves to the next one without waiting for the results.

    To retrieve the results, you must poll the get_search_archive(search_id) method until the search_metadata['status'] indicates Cached or Success.

    from serpapi import GoogleSearch
    import os
    
    # Initialize with async enabled
    search = GoogleSearch({
        "location": "Austin,Texas",
        "async": True,
        "api_key": os.getenv("API_KEY")
    })
    
    # Execute non-blocking searches
    for company in ['amd', 'nvidia', 'intel']:
        search.params_dict["q"] = company
        result = search.get_dict()
        search_id = result['search_metadata']['id']
        # Store search_id to retrieve later...
    
    # Later, retrieve from archive
    archived_search = search.get_search_archive(search_id)
    if 'Success' in archived_search['search_metadata']['status']:
        print(archived_search.get_dict())
  6. Migration notice: Deprecation of google-search-results

    master

    WARNING

    This package is being deprecated in favor of serpapi-python. It is recommended to migrate to the newer implementation to ensure continued support and access to the latest features.

    Note that current documentation and examples on serpapi.com are written for this legacy package and are not yet compatible with the new library.

  7. Retrieve search results in different formats

    master

    The GoogleSearch object provides several methods to access the results:

    • get_dict(): Returns results as a Python Dictionary.
    • get_json(): Returns results as a JSON string (use the json package to parse).
    • get_object(): Returns results as a dynamic Python object, allowing attribute-style access (e.g., result.organic_results).
    • get_html(): Returns the search results formatted as raw HTML.
    # parse results as python Dictionary
    dict_results = search.get_dict()
    
    # as JSON using json package
    json_results = search.get_json()
    
    # as dynamic Python object
    object_result = search.get_object()
    
    # search format return as raw html
    html_results = search.get_html()
  8. Use the Location API to find locations

    master

    The get_location method allows you to find canonical names and metadata for specific locations. This is useful for building localized searches.

    from serpapi import GoogleSearch
    search = GoogleSearch({})
    # Returns a list of matching locations
    location_list = search.get_location("Austin", 3)
    print(location_list)
  9. Search different engines with SerpApiClient

    master

    The SerpApiClient class provides a generic interface to interact with any search engine supported by SerpApi (e.g., Google, Bing, Baidu, etc.) by specifying the engine parameter.

    from serpapi import SerpApiClient
    
    query = {"q": "Coffee", "location": "Austin,Texas", "engine": "google"}
    search = SerpApiClient(query)
    data = search.get_dict()
  10. Retrieve searches from the Search Archive API

    master

    SerpApi caches search results temporarily. You can retrieve a previous search for free using its search_id found in the search_metadata of the original result.

    from serpapi import GoogleSearch
    
    # 1. Perform a search and get the ID
    search = GoogleSearch({"q": "Coffee", "location": "Austin,Texas"})
    search_result = search.get_dict()
    search_id = search_result.get("search_metadata").get("id")
    
    # 2. Retrieve the archived result using the ID
    archived_search_result = GoogleSearch({}).get_search_archive(search_id, 'json')
    print(archived_search_result.get("search_metadata").get("id"))