Help Needed With JSON Use to Move Mouse and Search In Screen Area

Hi All,

Really Sorry if this is a straight forward solve but have been trying myself for a couple of days and can't quite work it out.

Short Summary:

What I am trying to achieve is that the list below of Left,Top,Width,Height is used to set the screen area that is then utilised in a find image action. Ultimately I need to have the ability to search 10 different screen area's as my ObstacleNumber variable changes (from 1-10).

In Order to build the list you see below I have used the Get Area function within a click at found image action to dictate the Left,Top,Width,Height (Let me know if this is incorrect).

To make things simple for myself I added a move mouse function to try and parse out my error and it seems that it wants to move the mouse to X - 33561 rather than the 366 in the JSON variable Current Left (Again possibly my error or misunderstanding on JSON)
Screenshot 2025-04-25 at 17.11.18

Any help and guidance would be much appreciated because I am truly stuck!

Also I am aware that in my image find it says ScreenLeft instead of CurrentLeft I was adding different calculation steps and Variable to Text to try and get it to work it did not work with CurrentLeft :slight_smile:

Area Find & Mouse Move Test.kmmacros (18.0 KB)

Your "Move Mouse" is positioning "relative to the front window's top-left corner" but "Mouse Display" is showing absolute coordinates, which doesn't help.

But the actual reason is that you're trying to use "JSON Value"s in a calculation field. Change each of your "CoOrd Pull" Group's actions to "Set Variable to Text", eg:

image

Do you actually need JSON here? If you list your numbers, each group of 4 on a line, you can access a line using pseudo array notation and your numbers with standard "rectangle" notation:

Area Find & Mouse Move Test (no JSON).kmmacros (16.1 KB)

Image

That worked great thank you!

Can you briefly explain for me two elements please for my future projects:

%Variable%Local_ObstacleAreas[Local_obstacleNumber]\n% what doest the \n do at the end of this to help parse the strings into buckets of 4 numbers?
2.
Does the system automatically understand what left, top,width & Height are without labelling them as it has a string of 4 numbers so just assumes without naming each individually?

Apologies if these are super basic questions but will help me not bug people for answers in the future.

Pseudo array notation is in the form:

%Variable%variableName[index]separator%

"index" you know, but remember that KM's indexes start at 1 (unlike eg JS's 0-based indexing).

"separator" is the string to be used to delimit the elements of the array. We're using \n -- the linefeed character -- to get each line as an element. The default is , so

image

...would display dog.

You can also use a multiple-character string for your separator:

image

...which can be handy when combining then splitting unknown strings.

There's more about this in the "Variable Arrays" part of the manual's "Using Variables" section.

In a Caclulation field

  • A pair of numbers separated by , can be treated as coordinates -- x,y
  • Four numbers separated by , can be treated as an object rectangle -- n,m,p,q
  • And five numbers are an object rectangle with "fuzz" as the fifth element -- n,m,p,q,r

...and you can use "dot notation", not only to access the parts but also as calculation shortcuts (.bottom, .MidX, etc).

See the "Screen Coordinates and Dot Notation" section of the manual for more.

The upside is that it is more concise than your JSON way of doing things, and it's pretty obvious what's happening when you're dealing with a list of screen rectangles. You could also use ,-based array access -- but that isn't so self-documenting ("Local_myWindow[3], remind me, what was the 3rd number for?" vs Local_myWindow.width).

An array of JSON dictionaries requires more futzing about but will make it clear which values you are pulling at each stage, and really comes into its own when you can't guarantee the order of the numbers for some reason.

1 Like