Max number of rows that can be stored through "Get Column" action

Hello forum members,

I am trying to automate a web-scraping script that uses IP addresses to look up machine details on an internal website. I am facing a problem where the “Get column” action can only hold 2000ish entries of the 45,000+ line cells (contents are IP addresses) in the excel file when I transfer the contents to a list variable.

  1. Is there an upper limit on how many cells worth of data can be stored in a list variable?
  2. Is there a workaround to pick only 1500 cells at a time for processing using the “For Each” action till all 45K entries are processed since the filter does not currently accept variables?

Thank you,
Akshaya

Hi @akshaybs 45,000+ entries should be saved in a list. Could you send your data file so we can test a script on it?

Also, do you get any error when you play the script?

The script is doing basic scraping from an internal web page. Currently I have 2 ongoing issues with the script:

  1. The script was running fine but now I also get a “Unable to open file” error. I checked the data types on the variables and everything looks OK.

  2. When the script runs, it runs fine but ends abruptly without updating the execution log or the results file with extracted data. I can see variable values being extracted during the execution of the script but see no output entries.

Your help is much appreciated.

Script file: IP Address Lookup_2.7z (6.6 KB)

I have resolved the unable to load problem - there was a problem with the excel dump function.

1 Like

So have all the issues been resolved? Or do you still have a problem with the Get column action?

I still have the problem with the Get column action.

Could you please send your data file so we could test it?

You cans send it via a private message it you don’t want to share publicly.

unfortunately I cannot share the file since the contents are confidential. You can recreate the issue by storing over 2500 string values in each of the 21+ variables and trying to write the values into an excel file. Please let me know if you want me to create a sample script to do that using public sources.

1 Like

@akshaybs sorry, I couldn’t reproduce this issue.
The only thing I can suggest is to tweak your script a little bit:
at the end, instead of opening, the Excel file and setting each cell value in a loop, combine all lists with data in a table, and then open the file once and set the data there with Set Range.

Hope it helps.