More Context-sensitve and Consistant link hints

[DNGros]

First off, let me say I'm very new to vimium, but after a just few days, I already love it! Though I have a few ideas / frustrations with link hints that might be able to make vimium even better.

The most problem I have with link hints is in in Google search results. The first result is almost always different from search. Additionally, either the first, second, or third links is essentially always what I click on, and it seems like these links should get one character link-hints such as the first, second, and third result always being "J","K", and "L" respectively.

The "google problem" was the first thing I noticed, but after that I began wondering how else could link-hints be improved. I began to realize by applying some understanding of how people make websites and the DOM, link-hints could be improved.

Picking what gets a single, home row character link-hint:

• Links inside a <"h"> links or in a <"strong"> matter and most probable for what to click on. These should likely be given priority of single letter hints
• Size matters. Large click areas or large font sizes are likely important and should given should be given priority over small links.
• Inputs matter. Especially if there is a singular, large input on a websites. Also, the first input in a form is important. You likely want to get there and then can use tab to move through the form.

Identifying sequences:

• Alignment matters. Going back to the google results example, each result is both in a <"h3"> and share the same exact same x-value. This could be a strong indicator of a important and sequential list that might deserve a "J", "K", "L" sequence or other sequence. Similarly, if links share the same y-value they could also be sequential list.
• Should make sure to understand priorities when considering whether a sequence should be given a home row sequence; The small site nav links at the bottom of a page may be in line, but that doesn't mean you're likely to click or that they should get a home row sequence.

Maintaining Consistency

• Site Consistency. This is sort of similar to f function #1850. One is the the google example where the first link should always be the same, but another is like when browsing a multi page article, slides, etc. the "next page" link should be the same each time. This is sort of like ]] but can be extend to things like the "Issues" tab on github should ideally keep the same link-hint. This can be detected with looking at the value of the content of an element or as brought up in f function #1850 maybe something with hashing the css.
• Global consistency. Websites share common element functions, shouldn't they share link hints? Almost every website has a "top-left-corner-logo-go-home" link. It would be nice if the link-hint was always the same website to website. Github usually (but not always, even in the same site) has it as "aa" which I kind of like, but maybe keys more closely to context of vim might be better like "gg" or something. Similarly, many websites have a large input thats high in the page with something like "search" in the class or id name. These usually serve the same function site to site and should get the same ideally vim inspired link-hint.

What about filter mode?
Turning on filtering link-hints makes this sort of not matter, but I find it takes more key strokes and is slower than the one or two character hints. This is especially true in google results where links often share first letters or even word. Also, in small links the pop-up can cover up the first letter, which takes a moment longer of thought and slows it down. Having context-sensitive and consistent link hints I think could make single and double character link-hints even faster by removing the aspect allowing you to predict what you will have to click, like always hitting "j" for first google result.

Implementation Ideas
I am interested helping implement this. What I am imagining now with only looking at the code a bit is a couple of heuristics like those mentioned above (font size, in header, strong, click size, etc) that are based on the link element and maybe its immediate parent. Links are put in a priority queue by "link importance score". There is also something watching for links that should be sequence or should have specific value for consistency and would add these to a "preassigned dictionary" instead of the queue. Things in the queue are poped off and get sequentially more complex link hints unless they are in the preassigned dictionary.
Still a lot thinking and looking through to better understand the current code to make this work. I didn't want too deep into without hearing some feedback from current contributors first. This would be my first time working with CoffeeScript, and I don't have much open source experience, but I am really interested in learning and trying to contribute.

These are just some ideas about where to start with context-sensitive and consistent link hints. I look forward to hearing what concerns, ideas, and comments actual contributors and the community have.

tl;dr: Links that are most likely to be clicked should be given priority for single character, home row link-hints, and there should be consistency with things the first, second, and third google result getting the same hint every time.

[smblott-github]

Some interesting ideas there, @DNGros. The issues you raise are problematic, but nobody has yet come up with a good general solution. One of the problems is that what seems to be a good solution for one use case, turns out to be exactly the wrong thing for another use case.

However, I do think this is something worth looking at again. Do you want to have a go? The place to look is probably to assign a score somewhere here, add the necessary plumbing here, and use the score here and here. You can mimic the way linkText is passed around. It's part of the hint descriptor. Hint descriptors are unpackaged here.

[DNGros]

@smblott-github I agree there is potential for it not to always generalize. The way I figure, with link-hints being essentially random now, even if system to completely fell apart on a particular site, it wouldn't do any worse than the randomly selected link-hints. There is some concern that if it ends up almost always working, the times where it doesn't work could lead to more following of the wrong link. But for that to happen it would have to be working pretty successfully in the first place, and hopefully a consistency can be reached that the collective fractions of a second saved each time can outweigh the rare misfollow.

But yeah, the only way to find out is to try it. Thanks for the links and hints, that will help. I'll start playing with it probably around beginning of next week. I have a feeling that if just a good scoring system can be put in place, consistency and sequences will appear naturally.

Also, apologies for any duplication from #2011 / not originally linking to it with #1850. Did not see it earlier.

[lydell]

@DNGros It might be helpful to look how VimFx does things (hints.coffee). It actually implements some of your ideas. Here is a relevant quote from its documentation about the feature:

Which elements get hints depends on the command as well:

• f and af: Anything clickable—links, buttons, form controls.
• F, gf and gF: Anything that can be opened in a new tab or window—links.
• yf: Anything that has something useful to copy—links (their URL) and text inputs (their text).
• zf: Anything focusable—links, buttons, form controls, scrollable elements, frames.
• zF: Browser elements, such as toolbar buttons.

It might seem simpler to match the same set of elements for all of the commands. The reason that is not the case is because the fewer elements the shorter the hints. (Also, what should happen if you tried to F a button?)

(You can also customize which elements do and don’t get hints.)

Another way to make hints shorter is to assign the same hint to all links with the same URL. So don’t get surprised if you see the same hint repeated several times.

VimFx also tries to give you shorter hints for elements that you are more likely to click. This is done by the surprisingly simple rule: The larger the element, the shorter the hint.

There are standardized elements which are always clickable—semantically clickable elements. Unfortunately, many sites use unclickable elements and then make them clickable using JavaScript—unsemantically clickable elements. Such elements are difficult to find. VimFx has a few techniques for doing so, which works many times but not always, but unfortunately they sometimes produce false positives. Many times those false positives are pretty large elements, which according to the last paragraph would give them really short hints, making other more important elements suffer by getting longer ones. Therefore VimFx favors semantic elements over unsemantic ones and takes that into account when deciding the hint length for elements.

Some hint characters are easier to type than others. The ones on the home row are of course the best. When customizing the [hint chars] option you should put the best keys to the left and the worst ones to the right. VimFx favors keys to the left, so that should give you the optimal hints.

Some key points from the above:

• Prioritize the best hint chars.
• Prioritize "semantic" elements.
• Prioritize larger elements.

Please do share whatever you come up with! I'd love to "steal" good ideas into VimFx :)

[DNGros]

@lydell
I spent some time using VimFx and can notice the gain. Single character links feel a lot more deliberate. Thanks for pointing that out.

Though while it is an improvement, I don't think that exclusively size is the optimal scoring method. It does not the have the consistency aspect that I view as a really important feature. It can be thrown off by things like how many characters are in a search result or any button, link, etc, when in reality, the number of characters is rarely indicative of the importance of the link. Also, it tends to favor large images which may never be clicked on and without ad block, ads.

I'm not sure other methods will always be better, but with experimentation and comparison hopefully we can find out.

[DNGros]

As an update of what progress I have made so far:
I able to get it to assign scores and sort the link-hints by them. The current status can be seen at the link-hint-scoring branch here. Right now scoring is based off of screen position, element and parent tag, and a little bit for height. Right now it breaks on high link count sites like redit, wikipedia, or even sometime on github. Will need to fix that.

I have not tested with it extensively yet. The single character link hints definitely feel more deliberate than just random as before. Brief observations suggest while there is improved consistency, there is still a fair amount to be done. It seems to When I get the chance, I'll continue to tweek it and implement some other ideas I have for scoring.