Fix time-out issue in last "Index" stage

[andy1li]

• There're two ways in which the tester can hang, both related to companies.db

  • cping it
  • querying it to prepare the expected result

• Fix / Workaround:

  • use hard link instead of cp
  • hardcode the results

[rohitpaulk]

@andy1li can you elaborate a bit more on how long each of these actions takes (copying vs. querying)? Which is the dominant factor?

It's weird that this hasn't happened so far but is suddenly a problem. Wonder if I should contact fly to see if it's an issue on their end.

[andy1li]

@rohitpaulk Both copying and querying could time out (20s) on their own.

An inconsistency is that user's code could query the db without any issue.

Yeah, fly might provide more insights.

Just FYI, here're some disk and memory usage info that I gathered during debugging (Nothing out of the ordinary):

df -h:
Filesystem                Size      Used Available Use% Mounted on
devtmpfs                  1.9G         0      1.9G   0% /dev
none                      7.8G     35.3M      7.3G   0% /
/dev/vdb                  7.8G     35.3M      7.3G   0% /.fly-upper-layer
shm                       1.9G         0      1.9G   0% /dev/shm
tmpfs                     1.9G         0      1.9G   0% /sys/fs/cgroup

free -h:
              total        used        free      shared  buff/cache   available
Mem:           3.8G      131.0M        3.6G       21.9M      121.1M        3.5G
Swap:             0           0           0

[rohitpaulk]

Cool let's work with fly on this, definitely seems off. We'll focus on the copy issue first.

@andy1li could you help me out here a bit please before I contact:

• Add 2 simple logs:
  • One before copy
  • One after copy (that also includes the time elapsed)
• Remove all extra logs we only want these two
• Deploy and then run tests ~10 times to get values of how long it took
• I'll take it from there - will gather screenshots and fly machine IDs and contact them

[andy1li]

On it! Just to clarify:

One after copy (that also includes the time elapsed)

@rohitpaulk Will our current time-out err msg suffice, or do I need something special here?

[andy1li]

10 runs completed:

https://app-staging.codecrafters.io/courses/sqlite/admin/submissions

Oops, it's more accurate to log cp /var/opt/tester/companies.db test.db.

[rohitpaulk]

@andy1li so all of them failed on cp exceeding the allotted time? cp never went through and caused the failure where it occasionally goes through quick and the next one causes failures?

(Trying to ascertain if this is intermittent or not - it's harder to make a case with fly if it isn't intermittent)

[andy1li]

@rohitpaulk Yes, all 10 of my recent submissions look like this (cp never went through):

⛳ Before running `cp companies.db test.db`.
[tester::#NZ8] timed out, test exceeded 20 seconds

Furthermore, not a single submission (out of about a 100) showed signs of going through cp since July 24 (expect for a couple where I created a symlink to replace cp when debugging.)

(For the user submissions, it's possible that cp went through but their own code hung before outputting anything. But it's impossible to tell for sure.)

[rohitpaulk]

I've reached out to Fly.io about this with a repro: https://github.com/codecrafters-io/flyio-disk-performance-test

[rohitpaulk]

Also pushed a commit here that uses symlinks for now: c2598c7

Checked major languages and all of them seem to follow symlinks