How to scrape twitter comments?

hi

I try to pull out from some tweet in twitter. Is it possible?..can you give some idea?.I’ve lot of doubts in web automation process.

Hello,

Could you please clarify what exactly do you want to do?
Do you want to extract text from some twitter messages? Or you want to find some specific comment in twitter?
Please provide me with more details of what you’re trying to do, that information will help.
Thank you in advance!

I want to extract text from twitter using workfusion

Hi again,

You can extract text from twitter using OCR from actions library.
I have attached an example of how to do it.
TextFromTwitter.zip (185.6 KB)

In order to add this example to your WokFusion studio, follow this guide - https://kb.workfusion.com/display/RPAe/How+to+Use+RPA+Recorder#HowtoUseRPARecorder-Addrecording

Also, you can find more useful information about OCR in our Knowledge Base - https://kb.workfusion.com/display/RPAe/OCR

2 Likes

Using ocr format can scrap the data one by one. But I want to scrap the bulk amount of data from twitter comments.

I am not on twitter but I think you should look into relative Xpath that selects all the twitter comments [but first you will have to see if such an xpath existst] if you want to scrap bulk data.

1 Like

Actually this is my workflow.I have tried to scrap tweets from personal twitter page.I can scrap single tweets at a time.But I want to retrieve all tweets in single shot.

is it possible?

Yes it is certainly possible. I made a quick login to twitter and got an xpath that can be used to select all the comments to scrap the data.

To make your work easier with Xpath and scraping, try looking for a google chrome extension called as “Scraper”. (I hope this doesn’t violate any rules as I am only recommending an easy way to find Xpath)
You can scrap a common xpath with it. For example, there is a tweet with 13 replies


I simply right clicked on one of the reply and clicked on the option provided by the extension (Scrap similar) and it gave an xpath that works for all the comments (as you can see in the screenshot, it shows the result for 13 tweets when you search the xpath in inspect tool)

You can also find an Xpath on your own by reading the html code (but this is time consuming.)

1 Like

I forgot to mention that if you are going to use an xpath that selects all the comments and when them scraped, save the comments to a list variable!

can you send a sample.rpae project file?

Here you go!

Scraping.zip (1.7 KB)

1 Like