After Reading the Relevant KBM Wiki Page I'm Still Not Clear on What a Token Is

That's not how I read it:

That's just background.

That is the question/issue. Did NOT ask about difference between "variables" and "tokens".

IAC,

I disagree. Trying to show how "Tokens" are like "Variables" only adds confusion to the issue, IMO.

It seems like we now have a clear (to all) definition of "Tokens". Let's not add any confusion. Of course, if anyone still is confused about KM "Tokens", please speak up.

2 Likes

No, Jim, it’s all fine.

1 Like

Fine: I will say that our wiki saying "Tokens in Keyboard Maestro represent placeholders for information about a specific object.", encourages this confusion though? This is exactly how people shorthand their definition of 'variable' very often; as a 'placeholder' or some cognate. I did read the question as @Tom did, though not directly asking for an amplification of the difference as, at least, implying it in some way.

@tudor_eynon, I agree that used of the term "placeholders" is unclear and misleading.

I will revise the KM Wiki in the next day or so, and post update here for everyone's review/comment.

1 Like

The Wiki’s definition explains it from the point of view of a variable (“placeholder”), as I did in this thread, tailored to @ronald.

I don’t think the WiFi definition is necessarily bad, even if it’s “incorrect”. Many people perceive the tokens as a kind of weird variable (me too at the beginning, IIRC).
Reword it, if you want, so that it is clear that the explanation is a didactic approximation.

But I would leave the “variable” concept in for explanation, and then add a more in-depth explanation that explains that tokens basically more work like functions. (For the people familiar with the functions concept.)

Keep in mind, many people new to KM have no concept at all of functions and barely have a concept of variables. But the “variable” concept is way easier to grasp than the “function” one. IMO.

That’s why I’m proposing this. No offense meant.

Tom, we will have to agree to disagree on this.

Most non-programmer types have NO concept of what a "variable" is or how you would use it.
However, I would submit that since Excel is one of the World's most popular apps, more users are likely to have some exposure to "functions" than "variables".

1 Like

Well, I don’t see that many Mac users are also macro writers in Excel (but I agree that in Excel the “function” term is more predominant than the “variable”, since the variable is mostly an already existing cell),

And I also agree, quoting you “Most non-programmer types have NO concept of what a "variable" is or how you would use it.”

Though: Most non-programmer types have even less concept (except the Excel-ones, though I doubt they are going to translate the Excel “function” term into a generic one without comprehension problems) of a function.

So, I think it would be beneficial to have both definition approaches in the Wiki.

But I already said that :slight_smile:

And I won’t say anything more on this.

Do what you consider best.
Thanks for reading (and maybe considering) my point of view.

And yes, I also appreciate that you are open for discussion on the Wiki entries. Thanks for that.

Using functions in Excel does NOT require writing Excel macro. Almost every Excel user does this when entering a formula in Excel, as simple as =SUM(A1:A10).

Excel provides lots of exposure to "functions":

image

OK, against what I promised you convinced me to write another comment:

Your example (and the GUIness of the example) underlines what I meant when I said that even if an Excel user (macro writer or not) that is using functions will not necessarily have a concept of functions. At least not one that would be transferable to KM or another language.

My original wording:

though I doubt they are going to translate the Excel “function” term into a generic one without comprehension problems)


But I don’t really think this is important.

Let’s say, there are some KM newcomers that have a concept of “variable” and not of “function”, and other ones that have a concept of “function” (from Excel maybe) but no clue of “variable”.

So why not be helpful to both?

Anyway, an interesting fragment on didactic concepts and on what one expects from others we don’t know :wink:


PS:

With “fragment” I refer to the last part of this thread, not to your last post.

1 Like

Tom, you are beating a dead horse.

To understand tokens, you have to simply understand their purpose, which is primarily to allow non-fixed text to be inserted inline into plain text. This process is sometimes called variable interpolation or string interpolation, or template processing.

An early use of tokens was the standard Mail Merge facility - you want to send a customised letter to a hundred customers, and you want each addressed to an individual name from your customer database, so instead of writing your letter as:

Dear Customer

You write it as:

Dear %Customer%

Somehow the “Customer” part has to be marked up such that the software can know that that part is the part that needs to be replaced. In Keyboard Maestro, it is done by surrounding tokens with percent characters, %TokenName%, but in places it is different, perhaps $VariableName, ${VariableName}, etc. There are many different ways, but the important thing is they are marked up in a specific way.

In many languages, you can only interpolate a variable name, but in Keyboard Maestro you interpolate any kind of tokens which can include all sorts of different values, including the values from variables or named clipboards or fixed values or values that change dynamically (like dates or times).

While variables and tokens are independent concepts, you often want to insert the value of a vairable in to a location that wants text, and the Variable token is how you do that.

3 Likes

The KM Variable Token is simply returning the value of a KM Variable. It could have been a function getVariable("myVarName"), and in fact, I suspect internally the Objective-C software is using just such a function (method).

I see no distinction between KM Tokens and KM Functions, other than that KM Functions have been arbitrarily limited to returning numbers.

So, Peter, I understand you needed some way to identify a function call in the plain text of a KM action, so you labeled them "Tokens". They could have been called functions, simply delimited by "%" characters to so identify. For example, we have both a CALCULATE() function and a %calculate% token.

I think a lot of the confusion about KM Tokens comes from the understanding most of us have of the general use of a token to represent a fixed value of money.

From Token Definition & Meaning - Merriam-Webster

a : a piece resembling a coin issued for use (as for fare on a bus) by a particular group on specified terms

b : a piece resembling a coin issued as money by some person or body other than a de jure government

Given that common understanding, it is easy to see why the term "token" is confusing for something like %WindowFrame%1%, which actually varies in its return value, according to which window is currently frontmost.

I don't believe I have ever heard or seen the term "token" associated with the example you gave in processing mail merge lists. "Placeholders" or "fields" are the terms I've seen used.

Moving Forward

Be that as it may, the concrete has been poured, and we have the term "Token", for better or worse.
The thing that concerns me now is how to clearly define this term, especially for those users with little or no programming experience. This is what led me to the analogy of "functions", since so many users world-wide are likely to have used Excel, and its functions. They understand the notion of a function returning a value, or of having an output.

I just reread the KM Wiki on Tokens :

Tokens allow you to add textual information to token text fields. You can tell that a field accepts tokens because a small T shows in the field while editing it.

For example, %SystemClipboard% will be replaced by the text in the clipboard.

It is no wonder many are confused about Tokens. I think I clearly understand the term, but that statement confuses me.

  • "add textual information" -- what a mouthful just to say Tokens only return text
  • "token text fields" -- first rule in definitions: never use the term in defining the term.

Of course, it is also confusing to say that Token only return text, when a window frame is clearly made up of numbers. LOL

So, we clearly have our work cut out for us to rewrite the Wiki Tokens page and make it very clear to all.
I'm make a first draft, and submit it for all to review and comment on.

2 Likes

It would have been really nice to have this kind of discussion in person or a seminar room. Makes me almost nostalgic for those days of face to face. To help you @JMichaelTX; that meaning of 'token' in normal English is not, as far as I can see, the source of the problem, not for me anyway. I take "token' in its meaning of 'standing for' other things. Again almost opposite meanings shared by one term. I took it as a 'term of art' in KM and a good one really.
I really hope I am not skewing this discussion from my very narrow niche. I hope @tjluoma might chip in!

1 Like

English is tough language for everyone, because so many words can vary greatly in meaning according to the context in which they are used. And, one of the first things I was taught in technical writing was to avoid these two things:

  1. Use of common terms in an uncommon way
  2. Use of uncommon terms in a common way.

Add to that the fact that KM very much has users from all over the world, make it be quite challenging to produce clear writing.

But in your case, I see your definition of "standing for" to be very close to the dictionary definition of "a piece resembling a coin issued for use (as for fare on a bus) by a particular group on specified terms".
IOW, the token for bus fare is standing for the actual cash one might have to pay.

IMO, it is not good, not intuitive, for any software to use a term of art that is inconsistent with common usage in the computing world. So, to use "token" for something that "walks, talks, and acts" like a "function" is confusing.

By that definition, since functions typically can also have side effects, everything is a function.

Actions are a function. Certainly Conditions are just functions that return booleans and Collections are functions returning lists. Heck, with side effects, macros are functions.

Yes, tokens are sort of functions that return text, but they are also different to Functions based on their usage, context and interpretation. Calling them functions just confuses the issue.

There is a Hex token and a HEX function and they are very different because their context is very different. Since functions return only numbers, you know that a HEX function is going to return a number, so you know it is for turning hex characters in to a number. Tokens are for providing text, so then it is clear that the Hex token will provide text characters.

No, they do quite different things. One produces a numerical answer from text, one produces text, especially the CalculateFormat version, from a numerical expression.

There is a reason for the terminology, and while some people on earth may have a different interpretation of what a token means in some other dictionary, using a token to mean something that stands in for something is certainly the common usage here.

And Keyboard Maestro defines terminology for a purpose, so that clarity can be provided when talking about its behaviour. It has very specific terminology which is worth learning if you want to use Keyboard Maestro well. As does any subject of any complexity.

  • Editor
  • Engine
  • Macro Groups
  • Smart Groups
  • Macros
  • Trigger
  • Actions
  • Conditions
  • Collections
  • Tokens
  • Functions
  • Filters

There is a purpose to the terminology, and arguing that everything is a function runs entirely counter to that purpose.

3 Likes

Isn't this perhaps a source of confusion ?

image

1 Like

A day ago, I did not understand anything. Thanks to re.reading the wiki and Tom's explanations I had a mental flash and now think that I understand. Like you, I graduated in math and physics years ago.

  • forget about constants (speed of light in a void). There are no constants in this discussion.
  • you already know what variables are : numbers, words or groups (arrays, like (pencil, eraser, pen))

Consider variables are something personal defined by the user: name of Tom's cat, number of apples in his fridge
Consider tokens as similar to variables but common to all observers. For purposes of practicality, names of tokens are 'reserved'. Example: all participants in this discussion use computers: we all have browser front windows, files with filenames and paths, etc. Just look at the list of tokens below and it will be obvious. Wee all have front windows which have a certain size, we all have emails to 'to' recipients, etc

I think that you may be confused by reading that tokens are unchangeable ... but cannot be changed. The reason is simple and obvious.
We all use browsers and are in a front browser window at any point in time:

  • Tokens cannot be changed: the name %FrontBrowser% is reserved by KBM to describe the front browser window. You cannot use that same name %FrontBrowser% as a variable equal to the name of Tom's cat. You have many equivalents in math. You can call a mathematical variable x,y,z etc, but you will not call a variable in an equation LOG or EXP. LOG and EXP are like tokens: the are 'reserved' to describe specific things. You can write x = a+b but you will never write LOG = a+b
  • Tokens change all the time. Although the token name is reserved, its value changes. If your front browser window changes from KBM forum to Youtube, the value assigned to %FrontBrowser% will change

@Tom @peternlewis @JMichaelTX can correct me.

2 Likes

Well yes they can, even I can do it with the one token, dates, I seem to be able to use and find very useful.

My example:- the token " short date " in the drop down menu " insert token " is, as a string,

%ShortDate%

You can change that to

%ICUDateTime%MMM d, yyyy%

In fact it will take any ICUDateTime properly written string and change in fact "short date" to "long date" if you wish to create that kind of confusion for yourself using ICU syntax.

I would say that the 'MMM' in that expression is a kind of variable within the Token. All you are doing probably is entering the machinery at a slightly lower level as it were? I wish now I knew programming, Peter's comment on string interlopation and the link he provided was illuminating: as you say ronald, it is a sequence of 'flashes' !

The change to the ICU formatted string produces exactly the same result as the %ShortDate% token. Now I have no idea where or when I got that other format from or why I tried it instead of the date token. I suppose I inferred or intuited that the ICU formats were what Peter is calling 'tokens'. I have the impression I was led to the ICU format from the KM wiki itself in fact.

My guess is that one is entering the machinery as it were at a different level. I take @peternlewis ' point about terminology and he is right. It is very hard to get terms which are fail safe though in any field. Mine is full of similar problems. I have actually taught people excel, very few went away with any understanding of 'function'. I returned to one group I showed stuff to some time later and they just had a kind of hard copy sheet with the formulae I had used written out for mechanical copying into a cell. Some had already degenerated into ill formed versions, brackets left out that kind of thing... By then the formulae were already, anyway, built in to excel and one just had to insert them from a drop down: they required no understanding what so ever of the principles or issues we are discussing. I can't seem to apply that 'can do' method in KM though.

I take points here from all of you and I don't want to skew the discussion to my very particular, in fact peculiar niche of experience. Until I have thought or worked at this a bit more I will stay schtumm and try to pinpoint what I don't understand. Because I still don't really 'get it', I think the fault is in me frankly.

By the by @ronald , a constant is a symbol that stands for an unique number too, it is not that number. In your example if we found we had made a mistake in calculating the speed of light, we could still use "c" for that new number but only that number. In fact constants have been revised. It isn't, as you say, relevant and I didn't mention constants anyway did I?

2 Likes