Mouse click to accept dynamic coordinates

mouse-actions
#1

Hi,

Mouse click issues. I set them all here as they are much relevant to each other.

I try to use variables as X and Y when I use Mouse Click with coordinates, but it seems that WF only accepts fixed numbers. Is there a way to use dynamic values for that?

Also, is it possible to click outside the captured image?
E.g. I have a logo image and I know that a variable hyperlink is shown 100 pixels below this image. Can I use the logo (standard image) and say that I want to click at an offset of Y+100, where Y is the number for the bottom part of the captured image?

And a last one, can I get the current coordinates of the mouse somehow?

P.S.: All this with the condition that the general problem with the mouse clicking in a wrong place while using coordinates is be solved.

Kind regards,
Dimitrios

#2

@dimitrios_nt
Hi Dimitros,

Here are answers to your questions:

  1. No, you cannot use dynamic values as coordinates, only fixed numbers.
  2. In such case, you can capture the whole image, but only use the logo as the anchor area. RPA Express will not look for any images outside of the anchor area, but will click there according to the settings.
  3. There is no option to get the current coordinates of the mouse.

Alesia

1 Like
#3

Many thanks for the clarifications @ashapkina !

#4

Then how do you use the coordinates ? (find the x & y)

#5

Janus, when you record a mouse click and then change the settings of the action from “Click on image” to “Click on coordinates”, the coordinates are defined automatically.

#6

Hi, the Co-ordinates just end up clicking on other places and not where you set them from the image click. when changing to coordinates it doesnt work

#7

@nontobeko.mlambo We had a bug with coordinates in the previous releases. It has been fixed for RPA Express 2.0, which will be officially released tomorrow.

#8

@nontobeko.mlambo The bug with coordinates has been fixed in RPA Express 2.0.

2 Likes
#9

Hi, could you confirm that dynamic coordinates is still not possible? If I edit the code generated, could I use a dynamic variable?

#10

Hi,

Dynamic coordinates IS possible (see attached script) dynamicMouseClick.txt (453 Bytes)

In general you can do pretty much EVERYTHING with “custom actions” since it can execute java code.
If you want to use a workfusion fonctionnality out of the box, just export your rpae as bot task, and see what is the line of code that executes the desired operation. Then you can use this line in your rpae project in “custom actions”.

Now answering @dimitrios_nt, you can also get current coordinates of the mouse (using java code). For a working example, you can see the code of my post here: Detect Color in Image Detection RPA Express

Best,
Alexandre

3 Likes
Dynamic acess of cordinates
#11

Thanks a ton! Let me try and do that :slight_smile:

1 Like
#12

I seem to be doing something wrong, I am trying to extract the X,Y position of my mouse using a variation of the second script.
I am getting an error for this: General error during canonicalization: Output parameter should be defined as variable name.
I have defined it in the recorder variables though.
Could you help me?
Thank you!

script rec_var

#13

Hi @SaurabNair.
Looking at your script and it seems to me that you missed starting quote before “mouse_x” parameter:
image.
You can compare how it should look on the page with the samples of Custom script: https://kb.workfusion.com/display/RPAe/Code+Samples

3 Likes
#14

Thanks Lera - I made that change, but the error is still there. Could you tell me what I could do further? error.txt (14.6 KB)

#15

Strange. Let’s save your script (make sure that all your variables are equal to script output variables), close Studio and run again.
If this didn’t help, please remove and create back output variables in Studio and save the script without running. Then run it again.

#16

Also, may I suggest you to change

mouse_x = MouseInfo.getPointerInfo().getLocation().x
mouse_y = MouseInfo.getPointerInfo().getLocation().y

to

mouse_x = RNumber.of(MouseInfo.getPointerInfo().getLocation().x)
mouse_y = RNumber.of(MouseInfo.getPointerInfo().getLocation().y)

Since mouse_x and mouse_y are RPA variables

2 Likes
#17

Hi, thank you for trying to help me. I tried both suggestions but it still doesn’t work.
I am attaching the screnshot where you can see the recorder variables are pre-defined.
Do I need some plugin which I am missing? I am also attaching the error message.error.txt (47.9 KB)

#18

@SaurabNair, are you trying to create Bot Task? Or do you use Custom Script in Actions Flow?

#19

I am using custom scripts in action flow. I exported the code to see if something was amiss in the above XML.

#20

Ah, I see what’s the issue! :bulb:
You have 2 output parameters, but there is the only one can be used in Custom Script. It’s also mentioned in this guide: https://kb.workfusion.com/display/RPAe/Script+as+Custom+Action

3 Likes