How to Iteratively Click a List?

I'd like to click each entry in a list as shown in the following and send it to a bounded color.

I am not sure how to achieve this in keyboard maestro. Could anybody let me know? Thanks.

I want to help, if I can, but it's not clear to me what you are using in that image. Are you using the thinkorswim desktop client for macOS, or the thinkorswim web platform, and if so, which browser are you using?

Once I understand this, I will start to ask what you mean by "click on each entry in a list." The video showed a spreadsheet, not a list. I think I'm unclear what you mean by your question.

If you can be more specific about what you want to click on, I'm fairly sure someone can help.

I am talking about ThinkorSwim Desktop not ThinkorSwim web. There is nothing related to a web browser.

You should see a screen like this in the video at the time around 1:24, where there are many rows on the left (for call options) and many rows on the right for (put options).

I want to click each call option and put option (for each expiration date) and send the entry to a bounded color. Does it make sense?

Since I have no experience with that app, some of the terms, like "[sending to] a bounded colour" have no meaning to me.

Let me try to rephrase your question using only macOS vocabulary... do you want to right click on a cell, then move the mouse over a menu item, then click on a colour. Is that what you want to do? Can you use words like that, instead of app-specific words like "send it to a bounded colour" which doesn't mean anything to me. If this is correct, how does KM know which cell to start on? Do you want to hover over the starting cell and then press a hotkey to trigger a KM macro?

If my questions are frustrating, we can wait for someone who's familiar with your app to chime in.

Using the screenshot that I showed in the last post as an example.

You can see "10 SEP 21", "12 SEP 21", "24 SEP 21", "1 OCT 21", "8 OCT 21" on the left. These are expiration dates of the options. If you scroll down the bar on the right, you will see more expiration dates that currently can not be seen in the window.

The column named "Strike" are the strike prices of the options. You can see, for "10 SEP 21", there are many strick prices, i.e., 170, 175, ..., 375. You can see them when the options for this expiration date are expanded (indicated by a downward caret or ">" rotated clockwise by 90 degrees).

The strike prices are folded for other expiration dates (indicated by ">" on the left of the expiration dates), so that you don't see them on the sceenshot.

You see "CALLS" and "PUTS" above the column header. They indicate the left columns are call options and the right columns are put options.

For each expiration date, I first click it (so that the options for each strike prices are expanded, but they may not be all shown in the current window as there can be many strike prices to show), then command-click each row on the left and each row on the right for each strick price. Since I can not see all the expiration dates, I will need to scroll the window down so that I can click the rest expiration dates that I can not see.

Once I right clicked each option, you will see something like "Send .MSFT210910P300" as shown in the screenshot. I then click the first one labeled "1 Red".

I will repeat this process for each call and put options that I command-clicked.

Does it make sense now?

That's a lot more good information to a non-economist like me. You're speaking a language that I can (kind of) understand now.

The first thing we have to know is whether this app is AppleScript-enabled. Perhaps the product documentation will tell you that, or perhaps you can find out yourself by opening the app in the macOS App called Script Editor and reading the names of the functions and parameters that appear in there.

  • If the answer is yes, then some of the things you want to do can be reliably performed using KM's AppleScript interface. This could be called Plan A since it's very reliable. However I'm not an AppleScript expert so someone else would have to help you with this.

  • If the answer is no, then the next thing to investigate is whether there are keyboard shortcuts for each of the steps you want to do. If yes, then we have a Plan B. If no, then we may have to resort to Plan C, which is using KM's mouse manipulation to solve this.

Before we pursue Plan B or C, please try to confirm whether Plan A is a possibility. Until we know if Plan A is viable, there's not much point in me talking about Plan B or C solutions.

It is not applescript enabled.

They have shortcuts, but I don't find anything within "Trade" tab -> "All products" subtab.

https://tlc.thinkorswim.com/center/howToTos/thinkManual/Getting-Started/Keyboard-Shortcuts.html

So neither A and B can be done. C is the only way to go.

BTW, there is one thing that I just notice. That is the double-downward-overlapping-triangle at the right end of the "Option Chain" header. Once it is clicked, options of all expiration dates will be expanded. So the task is simplified a little. But I will still need to command-click all the call and put options one by one.

I will accept that Plan C is the way to go, but nearly all apps have some keyboard shortcuts, so it's probably more of a Plan "b+C" (sic.)

I'm re-reading your explanation. I'm having some difficulties. For example (one of many examples) you say "for each expiration date" but I suspect that you mean "for each expiration date in a range that you want to specify" since there's a screenful of expiration dates and I doubt that you want each of them every time. So perhaps the best way for you to resolve these questions is to make a video of an example of what you are trying to do. Do you know how to use the macOS built-in screen recorder to make a video (make sure the mouse pointer is being recorded)? Can you upload the video to Youtube? That should be easier for you than documenting the keystrokes and mouse clicks.

I'm quite handy with using KM's mouse and keyboard manipulation actions (and its Find Image action.) At this point I can't write any code for you because I still can't see the steps your mouse and/or keyboard are taking.

But I can tell you what I'm already thinking. I've done things like the following... I've created macros that tell the user (either by sound or pop-up message) to move the mouse to the "start location" and leave the mouse there for a second. Then the macro tells the user to move the mouse to the end location and leave the mouse there for a second. That gets the start and end locations of the actions to be taken. Then a series of mouse and/or keyboard actions will occur on each "row" from the start location to the end location. I've done this before and it's not hard at all. But I can't write any code for you because I can't see what the macro really needs to do.

I doubt that you want each of them every time.

No. I want to iterate all the expiration dates instead of a subset of expiration dates.

Here is a video showing what I want to do. I stopped at the 2nd expiration date for the sake of not recording too long. But I want to iterate all expiration dates as I mentioned.

But I can tell you what I'm already thinking. I've done things like the following... I've created macros that tell the user (either by sound or pop-up message) to move the mouse to the "start location" and leave the mouse there for a second. Then the macro tells the user to move the mouse to the end location and leave the mouse there for a second. That gets the start and end locations of the actions to be taken. Then a series of mouse and/or keyboard actions will occur on each "row" from the start location to the end location.

That is exactly why I think my task is hard to do as I don't see a way to easily position mouse to each row correctly.

I saw it. That video was extremely helpful. In short, your problem looks like something I could solve in 15 minutes (if I had your app, which I don't, and was familiar with it, which I'm not.)

What you think is hard to do is easier than you think it is. But what you think is easy to do is harder than you think it is.

The hard part, from what I see, is deciding where the start and stop locations are. Using your eyes, you can see where each date range starts and stops. Your eyes and your brain work with your hands to decide where to stop and start. The easy part is the positioning from one row to the next, but the hard part is figuring out where to start and where to stop. In my last post I told you a way (that I've used before) to determine start and stop points.

You didn't actually indicate when you want to stop, or what mouse actions you take when you finally hit the end. Do you really want to go beyond the bottom of the screen? You didn't indicate this. The stopping process is just as important as the starting process. Maybe we can deal with "stopping" later.

Basically the code will look like this:

  1. Find the first row of the first block (method TBD)
  2. Find every other block header and the final block trailer (method TBD)
  3. For each block: (a) iterate over each row on the left hand side with a series of mouse clicks; then (b) iterate over each row on the right hand side with a series of mouse clicks
  4. Implement the end of macro code (using mouse clicks that you didn't include in your video.)

Do you understand my overall plan here?

There will be some tricky things here because the pop-up menus that you see sometimes appear on the left or right (or above or below?) where the mouse is. This will take some experimentation to resolve.

Do you understand that my role will be to design the code to the best of my ability, but you have have to do some "code-finessing" because some things may be dependent on details that I can't handle, like screen resolution, or windows positioning.

P.S. Youtube rendered your video in hi-res about a half hour after you posted it. This is a common feature of YouTube uploading.

P.S. When I looked at the number of entires in each block of items, the number of rows per block ranged from 15 to 778. Do you really want to process the sum of all those rows? Even if we could automate this twice as fast as the human hand can click, that's still going to take between 10 minutes and 30 minutes to execute the entire macro. Is this something you have tested by hand with that many entires and the app still worked? This is about 10,000 consecutive mouse clicks. When you have that many mouse clicks in a row you need to worry about reliability, which means adding some preventative code. For example, we can't just assume that there are no delays to the pop-up menu... we have to validate that the menu is there before we click on it.

You can register for a free account for ToS to access its paper trading account. Then, you can use the paper trading account on ToS desktop. If you have time (which shouldn't be too long), I'd suggest that you register one. This will make sure that you can make the macro works correctly with the actual ToS app.

You didn't actually indicate when you want to stop,

I just scroll down until the last expiration date then click each options until the last option. Then, I am done.

P.S. When I looked at the number of entires in each block of items, the number of rows per block ranged from 15 to 778.

I don't think there are so many rows in an expiration date. Usually, there are tens to a few hundred options for a stock. So if you have a few clicks for a stock, then there will be a few hundred to a few thousand clicks for all the options of a stock. It is OK for now. I will see how it works in practice once the macro works.

I've created macros that tell the user (either by sound or pop-up message) to move the mouse to the "start location" and leave the mouse there for a second.

I want to completely avoid human interaction. So asking the user to point the mouse somewhere is not an acceptable solution.

That makes sense now that you've told me you want ALL the items clicked on. At first I didn't realize you wanted them all. Most people want "a range" when they do something like this.

Okay, but in the original video you posted at the top the narrator somehow switched to a graphical options chart. I didn't realize you don't need that done, until just now. Or maybe you can handle that part manually.

Okay, but in the sample you showed, there were about 1400 entires. There are at least 6 clicks per stock (at least 3 for each of the two column), possibly more. That's a bare minimum of 8400 clicks. Computers can handle numbers like that easily, but you didn't mention in your original post that your macro would require that many clicks. That does introduce reliability issues. We don't want the macro failing at 8,300 clicks. So it may have to work slowly at first until it's debugged. If we start with 1 second per click, that will take 2.3 hours for a single run. I hope you have some smaller data sets that you can test on before you go to the set you want.

The main problem with this is not that the number 8000 is large, but that we will have to deal with scrolling. At no point did you mention that the solution would require scrolling through dozens of pages of line items. I have indeed solved problems exactly like this in the past, so I'm not intimidated, but in that case I had the software in my possession so I could easily troubleshoot the scrolling controls. But in this case I don't have access to the program, so that makes things harder.

Let me ask you some questions. If the answers to these questions are favourable, that would simplify the required programming.

  1. If the macro we create did a right click on the same entry twice in a row, and clicked on its submenus too, would that be a harmful bug, or would that simply be a slight waste of time? THIS ITEM IS CRUCIAL.
  2. If the macro we create did a right click on the blank line between rows, what would pop-up, if anything?
  3. Are you able to change the rows' height using any sort of controls?
  4. Does the pop-up window appear outside the frame of the app window if you right click on items near the edge of the app?
  5. Can you resize the columns if we need that?
  6. Can you scroll the window with both the mouse wheel and the scroll bar? Can you also scroll with "page down" or any other key? Exactly how much of the page is scrolled when you scroll with the "page down" key? THIS IS EXTREMELY IMPORTANT.
  7. What happens when you get to the bottom of all the items using the scroll bar? Does the bottom item reach the top of the screen or does it stay at the bottom?
  8. What is the minimum number of lines that could possibly occur in each block? And the maximum?
  9. What is the minimum number of blocks that can occur? And the maximum?
  10. Does the window auto-scroll if you click on an item near the bottom of the window? (If so, this could save us a lot of work.)
  11. Does the DOWN-ARROW key provide any sort of down arrow motion on the window? (Allow me to hope!)
  12. Can you change the window to non-full-screen and can you resize the window in such a way that there is an exact integral number of line items on the screen?

Any solution we write has to anticipate all kinds of cases. I've written macros that solved problems exactly like yours, and succeeded. The hard part was dealing with understanding how the window was being scrolled. Up until now, I didn't know you were processing so much data that you needed to scroll the window for any amount, let alone many pages. This is no longer in the "15 minute" category that I previously mentioned because of that. Most of the code will be dealing with the scrolling.

But I've got lots of time. I'm just not sure how much time you want to spend at your end. You're going to be handling many of the details because I don't have your app. I really can help a lot, but you will have a lot of work too.

Okay, but in the sample you showed, there were about 1400 entires.

OK. I just checked. The first video is about a popular ticker MSFT (microsoft). As of today, it has 2335 options. But that is almost the maximum one can I get. In my own video, for BNGO, there are only 145 options. There are only a few tickers that are more than a thousand options. Most other tickers have less options.

So it may have to work slowly at first until it's debugged. If we start with 1 second per click, that will take 2.3 hours for a single run. I hope you have some smaller data sets that you can test on before you go to the set you want.

I actually need to click each option then do something else then save a file. The latter part I consider as easy, so I did not bother to ask. As long as a file is saved, if KM failed afterward, I don't care anymore. But then the problem becomes, how to read all the expiration dates and strike prices, then I can check what has been downloaded via an external program, then go back to KM to just download the ones that have not been downloaded. But that is another question. So for now, you can assume the click can be much faster, it is OK to fail after say saving 100 options or so.

But in this case I don't have access to the program, so that makes things harder.

Could you download ToS desktop and register an free account with paper trading? It sounds like this is the only way to get the problem solved perfectly.

If the macro we create did a right click on the same entry twice in a row, and clicked on its submenus too, would that be a harmful bug,

I don't think it matter by command-clicking a row twice. I am not sure what you mean by "clicked on its submenus too." Do you mean "clicked on its submenus [twice] too"? If so, only the 2nd click matters.

  1. If the macro we create did a right click on the blank line between rows, what would pop-up, if anything?

I am not sure what the black lines you refer to. There are no blank lines between options with the same expiration date.

  1. Are you able to change the rows' height using any sort of controls?

No.

  1. Does the pop-up window appear outside the frame of the app window if you right click on items near the edge of the app?

Yes. But the pop is still in the current computer screen.

  1. Can you resize the columns if we need that?

There is no need to resize the columns, as you can command-click any point a row to bring the popup.

Can you scroll the window with both the mouse wheel and the scroll bar?

Both work.

Can you also scroll with "page down" or any other key?

This is not possible.

What happens when you get to the bottom of all the items using the scroll bar? Does the bottom item reach the top of the screen or does it stay at the bottom?

There is something below "Options", so it would not stop at the last option. You'd better to download the ToS desktop give it a try.

What is the minimum number of lines that could possibly occur in each block? And the maximum?

minimum I would say a few. maximum I would say tens.

What is the minimum number of blocks that can occur? And the maximum?

minimum a few. maximum tens.

Does the window auto-scroll if you click on an item near the bottom of the window?

No.

Does the DOWN-ARROW key provide any sort of down arrow motion on the window?

No. I only moves highlight of rows. But do not move rows. There is no use for my purpose.

Can you change the window to non-full-screen and can you resize the window in such a way that there is an exact integral number of line items on the screen?

It can be done. But I don't know what size to use.

Unfortunately, ThinkorSwim is not available in Canada . TD Ameritrade, which provides the platform to all its users does not provide it in Canada.

So would you rather ask someone else on this forum from America to replace me, or would you rather get involved yourself in helping me write this program. Every time I write some code you're going to have to test and and tell me the results, or perhaps show me the results in a video.

I've written programs exactly like this before, but I've never done it by helping someone else who has to be the person to test and run the program.

Perhaps a better approach is for me to give you advice on how to write this code. If you are adept with KM, I could guide you through some possible solutions.

Perhaps a better approach is for me to give you advice on how to write this code.

I can try.

That's great! I just woke up and I can spend time on this today, but right now I'm required for health reasons to make and eat some food. I'll be back.

Let me rephrase my question so we get an answer to this important question. In the video you uploaded, the user clicks (command clicks) on an item on the screen then the user clicks on a menu item then clicks on another menu item for Red.

So my question is this: if the macro you write does this series of actions twice for the same row, is that a problem for you, or is that just going to be a waste of time? This is important because it tells me if the macro can fail in this way. In a real world, macros never make mistakes, but you want this macro to run unattended for a long time, so we have to understand if there is a problem with this kind of mistake.

if the macro you write does this series of actions twice for the same row , is that a problem for you, or is that just going to be a waste of time?

Not a problem. It just doubles the time. Slowing it down by a factor of 2 is acceptable.

That's good news. I'm only worried about occasional mistakes if my code processes the same line twice. Okay, I'm back from breakfast, and I will ponder this issue right now.

Okay, the key thing to solving this is understanding that we have to process each line on the screen (twice, once for the left side and once for the right side) and we have to scroll down, and repeat this until we reach the bottom. Determining what the bottom is is actually one of the really tricky things (it seems easy to you because you are a human who can see the big picture, but the macro is a really dumb machine.)

One bit of great news is that it does NOT hurt if we process the same line twice. This is wonderful, because it means we don't really have to understand what's on the screen. Any mistake of this type just slows us down.

One question mark that remains is that you didn't answer one of my questions about what happens when you click on a blank line between rows on the screen. I guess we'll find out.

I have an interesting approach to solving this. It will make some mistakes, but I don't think any of the mistakes are fatal in any way. My method is so elementary, it has got hardly any intelligence at all. This will save us a lot of pain. If there's something wrong with my approach, we will have to add some smarts to account for unforeseen problems.

Basically, we are going to write a program using a mode I call "Helen Keller Mode." This is a mode which repeats an action over and over without looking at the screen. It's operating blindly and it will still work. This will make the macro easy to write, and fairly short.

Actually part of it will operate blindly, and part of it will operate with a Find Image action, because locating the pop-up menus is something that can be done reliably using that action. So it will operate partly in Helen Keller Mode, and partly in Where's Waldo Mode. Here's my pseudocode for my initial solution:

Step 1. Assume that the user has opened all the lines using the double arrow button.
Step 2. Assume that the location of the lines of items are in a fixed location and known to us.
Step 3. For each line of items, call a sub macro which processes the mouse clicks for that line.
Step 4. When you get to the last line, use the mouse to go down a page, then return to Step 3.

Basically, that can solve the problem. It's not much code. As it stands, it is an infinite loop, but a harmless one because you said repeating mouse clicks on the same line doesn't harm anything. But we can probably add an end of program detection statement in step 4 once we get this working, if you think you need it. (The simplest way to end the program is simply to abort the macro manually when the human user sees it's at the end. but we might be able to automate this.)

I'm willing to draft some actual KM code to start implementing the above, but I want your feedback and your questions right now before I do that. You don't have to understand everything at this point, I just want you to tell me if the method to my mad approach is starting to make sense? I can make it work, I just want a little feedback at this point.

EDIT: I think (maybe) I can do it without a Find Image (Where's Waldo) action. Which will be cool.