Conventions for Option Keys

[rpatters1]

As part of listing the differences between menu items, configuration items, and user settings, I realized that another tool in the tool box is Option keys when invoking the script. When Jari, Tobias, and I originally started our plugin projects, almost every plugin had a (modal) dialog box. We early on agreed on a convention that invoking a plugin with Option or Shift key should suppress the dialog box and use defaults or most recent settings. I subsequently added the ability to reverse that, which I see as the preferable choice in many cases (especially modal dialogs).

I did not resist @cv-on-hub's choice of creating extra menu items to suppress dialog boxes, because that's easier for a new user, but it would make for much less cluttered menus if we could adopt this convention of option key invoking (or suppressing) the dialog. Once users are trained on this it becomes automatic. In a user interface, "intuitive" equals "what the interface trains users to do" and if we could agree on consistent behavior of the option keys, they would be more useful.

I would propose the following conventions:

1. If a dialog box is modeless and if the user is likely to be changing the settings for each invocation of the script, then the default should be to open the dialog and option/shift key should suppress the dialog and use the last settings. An example of this case is transposition dialogs.
2. If a dialog box is modal or if the user is unlikely to be changing the settings for each invocation of the script, the default should be to suppress the dialog and use the last settings and option/shift should open the dialog. An example of this case is the Patterson Beams plugin (though it doesn't actually behave that way by default because reasons.)
3. In either case, if the dialog box has never been opened by the user it might open the first time regardless of option/shift state, unless there is a sensible default. Persistent user settings could potentially obviate the need ever to open it again.

FWIW: the current transpose_* scripts have already implemented case 1. above, and it is embodied in the FCXCustomLuaWindow:RunModeless method. So if you use RunModeless you get it automatically by default. (RunModeless forces the dialog to open if finenv.RetainLuaState is false, but it could be enhanced to use user settings as well.)

A concern is whether Keyboard Maestro can access a menu option with option key pressed.

@jwink75 @ThistleSifter

[cv-on-hub]

This use of modifier keys to replace duplicate menu items would be great if Keyboard Maestro can be shown to pass on modifier information while invoking Finale menus. (Tried for 3 days without success, but that doesn't prove impossibility). I don't think I've selected a Finale menu by mouse for years.

[rpatters1]

I found this post that might help (replace the Return key with the menu invokation in the solution script.)
https://forum.keyboardmaestro.com/t/simulate-keystroke-not-respecting-option-key/18182/8

[cv-on-hub]

That looked promising, and I'm sure works on keystrokes, but I can't get it to work on Finale menu selection. Even tried (again) replacing KM hold-modifier with AppleScript actions without success. (I had tried various combinations of "delay" action before). In a related topic response PeterNLewis says:
... it is possible your target application is looking at the keyboard modifiers directly, rather than the keystroke events (this would generally be bad behaviour, which you would expect only from a cross platform/Java/etc application of some sort).

[rpatters1]

The PDK Framework, on Mac at least, is looking at the current event to get the modifier keys. (I know because I wrote the code that does it.) For menu selection, the current event is the current menu command. If you want to try relaying this to the Keyboard Maestro folks, specifically, the code is this:

uint32_t _cocoalink_GetCurrentModifierKeys()
{
    const NSUInteger flags = [[NSApp currentEvent] modifierFlags];
    const FLAG_32 shiftkey = (flags & NSEventModifierFlagShift) ? __FCUserWindow::CMDMODKEY_SHIFT : 0;
    const FLAG_32 fnkey = (flags & NSEventModifierFlagFunction) ? __FCUserWindow::CMDMODKEY_FUNCTION : 0;
    const FLAG_32 altkey = (flags & NSEventModifierFlagOption) ? __FCUserWindow::CMDMODKEY_ALT : 0;
    const FLAG_32 ctrlkey = (flags & NSEventModifierFlagControl) ? __FCUserWindow::CMDMODKEY_CTRL : 0;
    const FLAG_32 cmdkey = (flags & NSEventModifierFlagCommand) ? __FCUserWindow::CMDMODKEY_COMMAND : 0;
    
    return shiftkey | fnkey | altkey | ctrlkey | cmdkey;
}

[cv-on-hub]

Got a response from PeterLewis himself today (KM creator) but no joy unfortunately:
https://forum.keyboardmaestro.com/t/still-failing-to-select-a-menu-in-finale-with-modifier-keys/28323/2?u=cv186

[rpatters1]

To be clear, it isn't Finale that using option-select, it's third party plugins. But there are plenty of Mac apps that use Option-menu item to do things differently. Like the Finder.

I'm looking at some old code of mine and realizing that the code would probably work better with macro problems if in fact it did look at the direct real-time shift state. You'd probably be able to get it to work then. It appears that I had a similar problem with yKey (an older keyboard macro program for macOS that I still use because inertia).

[cv-on-hub]

PeterNLewis responds amazingly to user comments and may yet add something to KM offering an alternative mechanism. For now is this a tolerable exception case to allow a second menu item without a modifier-key menu?

[rpatters1]

I mean, I merged the PRs that had them, so I suppose the answer is yes. I just hate to see it proliferate. For example, if KM is operating the computer, why do you even care if the db appears? Does the no-db option speed it up that significantly?

[cv-on-hub]

Good point - saves just one enter keystroke., so will omit this from future iterations. I just checked and only my cross_staff_offset script offers a NO DIALOG menu option. All others with extra menu options offer substantively different operation. (I could also save one menu count by merging my midi_velocity and midi_duration scripts into a single midi_values script - still greatly simpler than Finale's built-in version).

[cv-on-hub]

The option-menu-selection issue for Keyboard Maestro is now solved, and the solution is described here: http://carlvine.com/km_option/

Meanwhile I've noticed with mixin modeless dialogs that OkButtonCanClose() is only respected when the option key is held down. @ThistleSifter confirms that this is intentional, and I understand that there may be a logical reason, but this remains strangely prescriptive behaviour in a process that should make no (or at least few) assumptions about how the user will use it. @rpatters1?

One caveat ... instructions in this counter-intuitive order actually enable OkButtonCanClose(), though I would have thought this was hard to present as "standard procedure"!
global_dialog:RunModeless()
global_dialog.OkButtonCanClose = true

[rpatters1]

The idea is that there are two modes of operation for a modeless dialog. One where it stays open on the desktop and one where you set the value and then never open it again (by opening it with the option key always.)

For the second mode, you don't want to have it stay open when you hit okay. You're only opening it at all because it has no settings yet.

That said, I'm not married to the idea. While personally I am very used to it, it's definitely not a standard behavior. Also, there's nothing stopping you from setting OkButtonCanClose yourself in the script. In fact, if you want that behavior all the time, I think it should be in the script.

[cv-on-hub]

The problem, from my perspective, is that OkButtonCanClose has no effect anywhere within the calling script except AFTER RunModeless(). (I've tried every other conceivable configuration).

Also, for me, I hate dialogs hanging around after they've done their job. I might use a dozen in half an hour and don't want the windows lingering, or to have to explicitly close them all. I want modeless because modal don't remember their positions!