check x/ y coordinate rescaling in create_aoi_timepoints

[mzettersten]

I'm not sure I understand the scaling to the x and y coordinates done in create_aoi_timepoints()

Naively, I would expect this to look like this:

x = x * video_width / screen_width
y = y * video_height / screen_height

Instead, we currently rewrite x and y coordinates as follows (l. 87-88):

x = x - (screen_width-video_width)/2
y = y - (screen_height-video_height)/2

@adriansteffan Could you elaborate on this step? I might be missing something, but I don't quite follow why we are rescaling this way, and it would seem to lead to incorrect values (e.g., if x reaches its maximum value, the screen width, this would be converted to mean of the video width and the screen width (e.g., 1920 if screen_width = 2560 and video_width=1280), rather than the max video width..... so something seems off?). I wonder if this might also have something to do with the remaining heat map issues in #111.

[adriansteffan]

I could be off here, but the formula was a result of the videos not being stretched to fullscreen, but rather centered using their original resolution. 1px video is 1px screen, but with black/white space around it.

if x reaches its maximum value, the screen width, this would be converted to mean of the video width and the screen width (e.g., 1920 if screen_width = 2560 and video_width=1280), rather than the max video width

In this scenario, the video stretches from 640 to 1920 in screen coordinates, so

x - (2560-1280)/2 =
x - 640

x = 640 -> 0 (min video)
x = 1280 -> 640 (mid screen == mid video)
x = 1920 -> 1280 (max video)

x < 640 and x > 1920 land in the space outside the video, so they should not fall within the valid video coordinates

[mzettersten]

Gotcha, I figured there must be something like this that I was missing. How sure are we that this is true for all labs? E.g. are we sure this is also true for labs reporting screen coordinates of 2560 x 1440 , like PKU?

[adriansteffan]

Maybe @lucie-zimmer can help, since she was involved in the greenlighting process - did the labs send in any info on their setup that we could double-check?

[lucie-zimmer]

Labs were required to present the stimuli centered on the screen. This was confirmed by all labs in our lab questionnaire; see column AF in the document here: https://docs.google.com/spreadsheets/d/16TQ3fOGjATD0YhFPAw4I6JBTCka3Vh994nMQAFtWAiw/edit#gid=1390267299

[lucie-zimmer]

Another possibility to confirm if the stimuli were really presented "centered on the screen" is by checking the walkthrough-videos: https://drive.google.com/drive/u/0/folders/1nI1HH_F_zt-jZD0zrttWs9fz6K-koW_4
In these videos, the labs were instructed to record the stimuli presentation.

[adriansteffan]

Ahh, I think I found the issue, the video looks stretched to fullscreen:

[msschreiner]

Exactly, that's what Lucie and I discovered this morning. I already asked Yanwei to check the settings within their Tobii project -> Lucie remembered that this lab created their own Tobii project and did not use the one provided by the coordination team ...

[mzettersten]

this is some real detective work folks!!!

[msschreiner]

This is what Yanwei reported back:

"We've checked our Tobii projects. It seems that the stimuli might have been displayed centered and fit to screen (see the screenshot). Maybe it's the problem. As for the screen dimensions, the width and the height of our screen was 1920 x 1080, while in the exported data files they were reported as 2560 x 1440. It's quite confusing and we are trying to check what's wrong. Do you have any idea of it?"

[mzettersten]

WRT the question about 2560 x 1440, I think it just depends on the scale that the x-y coordinates, which should be clear from the data. I think maybe we should reimport that dataset and potentially rescale values directly there.

The one thing I'm not sure about is how Fit to screen works exactly. Here's what I found in a Tobii manual:

If you select Fit to screen, the video is either enlarged or shrunk (keeping its aspect
ratio) to fit on the screen during stimuli presentation.

So I think we assume preserved aspect ratio. It also looks like the width wasn't fully extended out to the edge of the screen, so presumably the video is presented as 1440x1080 ? (preserving the 1.3333 aspect ratio). And then we just need to rescale accordingly. Should I take another stab at this? @msschreiner

[msschreiner]

Yes, please!

This probably happened, right?

[mzettersten]

PKUSu now fixed I believe. Closing this issue.