facebook-scraper

To download comments from a specific post, use get_posts with the post_urls parameter containing the POST_ID. You must use the options dictionary with the comments key to enable comment extraction.

import facebook_scraper as fs

# POST_ID can be extracted from URLs like:
# https://www.facebook.com/USER/posts/POST_ID
# https://www.facebook.com/groups/GROUP_ID/posts/POST_ID
POST_ID = "pfbid02NsuAiBU9o1ouwBrw1vYAQ7khcVXvz8F8zMvkVat9UJ6uiwdgojgddQRLpXcVBqYbl"
MAX_COMMENTS = 100

# get_posts returns a generator
gen = fs.get_posts(
    post_urls=[POST_ID],
    options={"comments": MAX_COMMENTS, "progress": True}
)

# Get the requested post
post = next(gen)

# Extract comments from 'comments_full'
comments = post['comments_full']

for comment in comments:
    print(comment)
    for reply in comment['replies']:
        print(' ', reply)

The get_posts function allows you to scrape posts from a unique page name, profile name, or ID. It returns a generator that yields post dictionaries.

from facebook_scraper import get_posts

for post in get_posts('nintendo', pages=1):
    print(post['text'][:50])

The get_profile function extracts information from a profile's 'About' section. You can pass an account name or ID. To access sensitive information like Date of Birth or Gender, you must provide cookies.

from facebook_scraper import get_profile

# Basic usage
get_profile("zuck")

# Usage with cookies for more info
get_profile("zuck", cookies="cookies.txt")

Use the get_group_info function to retrieve metadata about a Facebook group, such as its ID, name, member count, type, and admins.

Note: To access the list of admins, you must provide a cookies parameter pointing to a valid cookies file to authenticate your session.

from facebook_scraper import get_group_info
get_group_info("makeupartistsgroup") # or get_group_info("makeupartistsgroup", cookies="cookies.txt")

When using any of the iter_* functions, you can pass additional keyword arguments (**kwargs) to control behavior:

• start_url: The URL to begin scraping from.
• request_url_callback: A function called before each new URL request is made.
• base_url: The base URL used for joining relative links (defaults to FB_MOBILE_BASE_URL).
• options: A dictionary that can include posts_per_page to adjust the number of posts fetched per request.

When using the scraper, you can pass specific keys in the options dictionary to control the depth of data extraction:

• reactions: If set, attempts to extract reaction counts.
• reactors: If set to a number (int/float), limits the number of people reacting to be returned. If set to 'generator', returns a generator instead of a list.
• sharers: If set, attempts to extract people who shared the post. If set to 'generator', returns a generator instead of a list.
• comments: If set, attempts to extract full comments. If set to 'generator', returns a generator instead of a list.
• allow_extra_requests: (Boolean) Controls whether the extractor can make additional requests to fetch high-quality images.
• HQ_images: (Boolean) Controls whether to attempt fetching high-quality images.

You can enrich an existing post object (obtained via get_posts) with additional metadata such as reaction counts (LIKE, ANGER, etc.), sharers, and the w3_fb_url. This method may perform additional HTTP requests per post to retrieve the full data.

Note: This is typically used in conjunction with a post object already retrieved from the scraper.

# Example: enriching posts with reaction data
for post in get_posts('fanpage'):
    more_info_post = fetch_share_and_reactions(post)
    print(more_info_post)

You can run the scraper from the command line. Use --help to see all available options. If you encounter UnicodeEncodeError, use the --encoding utf-8 flag.

$ facebook-scraper --filename nintendo_page_posts.csv --pages 10 nintendo