Exporting comments from a website or application can be useful for analyzing user feedback, conducting sentiment analysis, or migrating data. However, some sites and apps don’t have a built-in export feature. In cases where exporting comments isn’t straightforward, there are a few potential alternatives.
Use a web scraper
One option is to use a web scraper to extract comments. Web scraping involves creating a script that can simulate a human visiting a web page and copying the content. Popular scraping tools like Import.io, ParseHub, and Octoparse make it relatively easy to scrape certain types of data from websites, including comments.
The basic steps for web scraping comments are:
- Identify the web page(s) that contain the comments you want to extract.
- Inspect the page HTML to find the unique selectors for the comment elements.
- Use a web scraper tool or scripting language like Python and Selenium to simulate navigating to the page and extracting the comment text.
- Store the scraped comments in a structured format like CSV or JSON.
Scraping comments works well for public websites, but it won’t be possible on pages that require login or have anti-scraping protections.
Export comments from the database
For websites and applications where you have access to the back end and database, another option is exporting comments directly from the database.
Steps include:
- Identify the database table that contains the comments data.
- Write a script or query to extract the comment records into a file format like CSV or JSON.
- Run and schedule this export script to run on a regular basis to get updated comments.
This gives you direct access to comments without needing to scrape them from the front-end UI. However, it requires access privileges to the database which external parties won’t have.
Access comments via API
For modern web and mobile apps, there may be a public or private API that provides access to comments data.
If an API exists:
- Review the API documentation for the endpoints and methods to retrieve comments.
- Use an API testing tool like Postman to test accessing comments via the API.
- Build a script that calls the API to extract comments on a regular schedule.
The advantage of using the API is that it does not require scraping or direct database access. The downside is that not all sites and apps necessarily have comment APIs available.
Export comments manually
If automated methods won’t work, exporting comments manually is always an option although it will require more effort.
Some steps for manual exports include:
- Browsing to the website or app that contains comments.
- Manually copying and pasting comments from the UI into a document or spreadsheet.
- Cleansing and organizing the comments in Excel or Google Sheets.
- Repeating this process on a regular basis to capture new comments.
Manual export works when automated methods fail, but it is time consuming and labor intensive. Still, it may be the only choice in some situations.
Summary
In summary, here are some potential options for exporting comments when a standard export feature is unavailable:
- Web scraping using tools like Import.io, ParseHub, Octoparse
- Direct database export if you have access
- Leverage a public or private API if available
- Manual export by copying and pasting comments
The best approach depends on the specific site or application, your level of access, and your technical capabilities. For large volumes of comments to export on an ongoing basis, automated options like web scraping and APIs provide the most efficient solution.
Frequently Asked Questions
Is web scraping comments legal?
The legality of web scraping depends on the terms of service of the specific website. Scraping publicly accessible data from sites that don’t expressly prohibit it is generally permissible. However, always review the ToS first before scraping.
What are the risks of web scraping?
Potential risks of web scraping include getting blocked by anti-scraping defenses, exceeding rate limits resulting in ban, and inaccurate or low quality data. Scrapers should implement robust error handling and parsing to minimize these risks.
How can I export Facebook comments?
Facebook does not provide an automated way to export comments. You could scrape comments from public page posts using a tool like Import.io. Or manually copy and paste comments – this is tedious but avoids scraping legal risks.
What format should I export comments to?
CSV, Excel, or JSON formats are ideal for exporting comment data. CSV is a simple text format that can be opened by virtually any spreadsheet app. JSON retains more data structure. Choose the format based on your analysis needs.
How do I access comments data via API?
Review the developer documentation for the website or application API. Look for endpoints and methods that allow retrieving comments. Use an API testing tool like Postman to call the API and ensure you can access comments successfully.
Is it possible to export comments from YouTube?
YouTube does not have a public API for comment data. Comments on public YouTube videos can potentially be scraped, but this comes with challenges around bypassing anti-bot protections. Manual export may be the most practical YouTube option.
Example Export Methods
Here are some examples of how you might implement both automated and manual exports for some popular sites and apps.
WordPress
Automated: Use the built-in WordPress REST API to export comments in JSON format.
Manual: Install a plugin like WP Ultimate CSV Exporter, generate CSV manually.
Automated: Use the PRAW Python API wrapper to extract Reddit comments.
Manual: Copy/paste interesting comments from subreddits into a spreadsheet.
Automated: Scrape public page posts for comments with Import.io or ParseHub.
Manual: Manually copy Facebook comments to Excel or Google Sheets.
Automated: Use Instaloader python library to scrape and download comments.
Manual: Instagram has no export feature, manual copy/paste only.
Disqus
Automated: Use Disqus API to extract comments in JSON format.
Manual: No easy way, try copy/paste if absolutely needed.
Conclusion
Exporting comments when there is no standard tool available requires creativity. Scraping, APIs, direct database access, and manual approaches each have pros and cons. The optimal method depends on the website, access level, and technical expertise. With some diligent effort, extracting comment data is possible in nearly any scenario.