Frequent zypper timeouts during package download (Curl error 56)

[ximGBu4cyQss5P]

For the past several months, I've been experience frequent zypper timeouts during package downloads, to the point that OpenSUSE distros are almost unusable for me. This happens in all the scenarios you would expect. Doing a zypper dup. Doing a package install. The worst is when it happens when doing a clean install of Tumbleweed or Slowroll using the net install media or Agama. A few weeks ago I was installing Tumbleweed via Agama and zypper stopped on a package download. I left the house, and two hours later when I came back it was still sitting there waiting to download that package. This is on a gigabit fiber connection, and I have no issues downloading any files or packages with other distros (Fedora, NixOS, Debian, Arch, etc.).

For this issue, I've reproduced it in a fresh Tumbleweed container. Here is the relevant section from /var/log/zypper.log after running ZYPP_MEDIA_CURL_DEBUG=2 sudo zypper -vvv install wine.

2024-12-23 13:13:51 <1> Z840(10410) [zypp] PackageProvider.cc(providePackage):525 provided Package (20596)libxml2-2-32bit-2.12.9-2.1.x86_64(repo-oss) at /var/cache/zypp/packages/repo-oss/x86_64/libxml2-2-32bit-2.12.9-2.1.x86_64.rpm
2024-12-23 13:13:51 <1> Z840(10410) [zypp] PackageProvider.cc(providePackage):424 provide Package (19899)libunwind8-32bit-1.8.1-2.1.x86_64(repo-oss)
2024-12-23 13:13:51 <1> Z840(10410) [zypp++] DeltaCandidates.cc(deltaRpms):83 package: (19899)libunwind8-32bit-1.8.1-2.1.x86_64(repo-oss)
2024-12-23 13:13:51 <1> Z840(10410) [zypp::media] RepoProvideFile.cc(provideFile):253 [1]./x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm{57.5 KiB|sha512-5ef52e0cdb45b216178ae6b00a191f23cc1e338272229a5dbebabc0710157ce092c022857e1fdab71fb09c69bd41744dd336fe423d9a844b3d56e0e3afe174e5|}
2024-12-23 13:13:51 <1> Z840(10410) [zypp::fetcher++] Fetcher.cc(addCachePath):337 Adding fetcher cache: '/var/cache/zypp/packages/repo-oss'.
2024-12-23 13:13:51 <1> Z840(10410) [zypp::media] RepoProvideFile.cc(provideFile):272 Added cache path /var/cache/zypp/packages/repo-oss
2024-12-23 13:13:51 <1> Z840(10410) [zypp::media] RepoProvideFile.cc(provideFile):306 Providing file of repo 'repo-oss' from http://download.opensuse.org/tumbleweed/repo/oss/
2024-12-23 13:13:51 <1> Z840(10410) [zypp::fetcher] Fetcher.cc(downloadAndReadIndexList):711 No indexes to read.
2024-12-23 13:13:51 <1> Z840(10410) [zypp::fetcher] Fetcher.cc(locateInCache):368 start fetcher with 1 cache directories.
2024-12-23 13:13:51 <1> Z840(10410) [zypp::fetcher] Fetcher.cc(provideToDest):539 Not found in cache, retrieving...
2024-12-23 13:13:51 <1> Z840(10410) [zypp::fetcher++] MediaSetAccess.cc(provide):275 Going to try to provide  file ./x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm from media number 1
2024-12-23 13:13:51 <1> Z840(10410) [zypp::media++] MediaManager.cc(checkDesired):157 checkDesired(1): desired (cached)
2024-12-23 13:13:51 <1> Z840(10410) [zypp::media] MediaNetworkCommonHandler.cc(getFileUrl):83 Redirecting ./x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm request to geoip location.
2024-12-23 13:13:51 <1> Z840(10410) [zypp::media++] MediaMultiCurl.cc(doGetFileCopy):1510 dest: /var/tmp/AP_0xyh19Io/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm
2024-12-23 13:13:51 <1> Z840(10410) [zypp::media++] MediaMultiCurl.cc(doGetFileCopy):1511 temp: /var/tmp/AP_0xyh19Io/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm.new.zypp.9hIc48
2024-12-23 13:13:51 <1> Z840(10410) [zypp::media++] MediaCurl.cc(doGetFileCopyFile):1204 ./x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm
2024-12-23 13:13:51 <1> Z840(10410) [zypp::media] MediaNetworkCommonHandler.cc(getFileUrl):83 Redirecting ./x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm request to geoip location.
2024-12-23 13:13:51 <1> Z840(10410) [zypp::media++] MediaCurl.cc(doGetFileCopyFile):1214 URL: http://cdn.opensuse.org/tumbleweed/repo/oss/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm
2024-12-23 13:30:07 <3> Z840(10410) [zypp::media] MediaCurl.cc(doGetFileCopyFile):1279 curl error: 56: Recv failure: Connection timed out, temp file size 0 bytes.
2024-12-23 13:30:07 <1> Z840(10410) [zypp::media] MediaNetworkCommonHandler.cc(getFileUrl):83 Redirecting ./x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm request to geoip location.
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 MediaCurl.cc(evaluateCurlCode):972 THROW:    Download (curl) error for 'http://cdn.opensuse.org/tumbleweed/repo/oss/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm':
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 Error code: Curl error 56
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 Error message: Recv failure: Connection timed out
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 MediaCurl.cc(doGetFileCopyFile):1295 RETHROW:  Download (curl) error for 'http://cdn.opensuse.org/tumbleweed/repo/oss/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm':
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 Error code: Curl error 56
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 Error message: Recv failure: Connection timed out
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 MediaMultiCurl.cc(doGetFileCopy):1539 RETHROW:  Download (curl) error for 'http://cdn.opensuse.org/tumbleweed/repo/oss/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm':
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 Error code: Curl error 56
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 Error message: Recv failure: Connection timed out
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 
2024-12-23 13:30:07 <1> Z840(10410) [zypp-core] PathInfo.cc(unlink):711 unlink /var/tmp/AP_0xyh19Io/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm.new.zypp.9hIc48
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 MediaCurl.cc(getFileCopy):802 RETHROW:  Download (curl) error for 'http://cdn.opensuse.org/tumbleweed/repo/oss/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm':
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 Error code: Curl error 56
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 Error message: Recv failure: Connection timed out
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 MediaSetAccess.cc(provide):285 CAUGHT:   Download (curl) error for 'http://cdn.opensuse.org/tumbleweed/repo/oss/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm':
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 Error code: Curl error 56
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 Error message: Recv failure: Connection timed out
2024-12-23 13:30:07 <5> Z840(10410) [zypp-core] Exception.cc(log):219 
2024-12-23 13:30:07 <1> Z840(10410) [zypp::media++] MediaHandler.cc(getDetectedDevices):1348 No devices for this medium
2024-12-23 13:30:07 <1> Z840(10410) [zypp::media] MediaManager.cc(releaseAll):554 Releasing all attached media
2024-12-23 13:30:07 <1> Z840(10410) [zypp::media++] MediaManager.cc(operator()):560 Releasing media id 1
2024-12-23 13:30:07 <1> Z840(10410) [zypp::media++] MediaHandler.cc(release):714 Request to release attached media http<http://download.opensuse.org/tumbleweed/repo/oss/>, use count=1
2024-12-23 13:30:07 <1> Z840(10410) [zypp::media++] MediaHandler.cc(release):721 Releasing media http<http://download.opensuse.org/tumbleweed/repo/oss/>
2024-12-23 13:30:07 <1> Z840(10410) [zypp::media] MediaHandler.cc(disconnect):693 Disconnected: http://download.opensuse.org/tumbleweed/repo/oss/ attached; localRoot "/var/tmp/AP_0xyh19Io"
2024-12-23 13:30:07 <1> Z840(10410) [zypp::media++] MediaHandler.cc(removeAttachPoint):150 MediaHandler - checking if to remove attach point
2024-12-23 13:30:07 <1> Z840(10410) [zypp-core] PathInfo.cc(recursive_rmdir):439 recursive_rmdir /var/tmp/AP_0xyh19Io 
2024-12-23 13:30:07 <1> Z840(10410) [zypp::media] MediaHandler.cc(removeAttachPoint):162 Deleted default attach point /var/tmp/AP_0xyh19Io
2024-12-23 13:30:07 <1> Z840(10410) [zypp::media] MediaHandler.cc(release):775 Released: http://download.opensuse.org/tumbleweed/repo/oss/ not attached; localRoot ""
2024-12-23 13:30:07 <1> Z840(10410) [zypp::media] MediaManager.cc(releaseAll):584 Exit
2024-12-23 13:30:07 <1> Z840(10410) [zypper++] media.cc(requestMedia):204 medium problem, url: http://download.opensuse.org/tumbleweed/repo/oss/, error 3, label 'openSUSE-Tumbleweed-Oss', #1
2024-12-23 13:30:43 <1> Z840(10410) [zypper] prompt.cc(get_prompt_reply):304 reply: r (1)
2024-12-23 13:30:43 <1> Z840(10410) [zypp::fetcher] MediaSetAccess.cc(provide):336 ProvideFile exception caught, callback answer: 1
2024-12-23 13:30:43 <1> Z840(10410) [zypp::fetcher++] MediaSetAccess.cc(provide):369 Going to try again
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaManager.cc(close):357 Close to access handler using id 1 requested
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaHandler.cc(release):707 Request to release media - not attached; eject ''
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaManager.cc(close):116 Close: http(http://download.opensuse.org/tumbleweed/repo/oss/ not attached; localRoot "") (OK)
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaHandler.cc(release):707 Request to release media - not attached; eject ''
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaManager.cc(close):116 Close: http(http://download.opensuse.org/tumbleweed/repo/oss/ not attached; localRoot "") (OK)
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaHandler.cc(release):707 Request to release media - not attached; eject ''
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaHandler.cc(removeAttachPoint):150 MediaHandler - checking if to remove attach point
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaHandlerFactory.cc(createHandler):59 Trying scheme 'http'
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaNetworkCommonHandler.cc(findGeoIPRedirect):42 Found GeoIP file for host: download.opensuse.org
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaNetworkCommonHandler.cc(findGeoIPRedirect):57 Found GeoIP rewrite: download.opensuse.org -> cdn.opensuse.org
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaCurl.cc(MediaCurl):357 MediaCurl::MediaCurl(http://download.opensuse.org/tumbleweed/repo/oss/, )
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaMultiCurl.cc(MediaMultiCurl):1375 MediaMultiCurl::MediaMultiCurl(http://download.opensuse.org/tumbleweed/repo/oss/, )
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaHandlerFactory.cc(createHandler):170 Opened: http://download.opensuse.org/tumbleweed/repo/oss/ not attached; localRoot ""
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaManager.cc(open):327 Opened new media access using id 2 to http://download.opensuse.org/tumbleweed/repo/oss/
2024-12-23 13:30:43 <1> Z840(10410) [zypp::fetcher++] MediaSetAccess.cc(provide):275 Going to try to provide  file ./x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm from media number 1
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaManager.cc(attach):445 attach(id=2)
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaHandler.cc(createAttachPoint):360 Create attach point: attach root is not a writable directory: '/var/adm/mount'
2024-12-23 13:30:43 <1> Z840(10410) [zypp-core] TmpPath.cc(Impl):53 /var/tmp/AP_0xlQ7fxJ
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaHandler.cc(createAttachPoint):343 Created default attach point /var/tmp/AP_0xlQ7fxJ
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaCurl.cc(setupEasy):622 Proxy: not explicitly set
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaCurl.cc(setupEasy):623 Proxy: libcurl may look into the environment
2024-12-23 13:30:43 <1> Z840(10410) [zypp-core] PathInfo.cc(chmod):1103 assert_file_mode 00600 /var/lib/YaST2/cookieschmod /var/lib/YaST2/cookies 00600
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaCurl.cc(setupEasy):650 No cookies requested
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaHandler.cc(attach):654 Attached: http://download.opensuse.org/tumbleweed/repo/oss/ attached; localRoot "/var/tmp/AP_0xlQ7fxJ"
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaManager.cc(checkDesired):155 checkDesired(2): desired (report by zypp::media::NoVerifier)
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaManager.cc(checkDesired):157 checkDesired(2): desired (cached)
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaNetworkCommonHandler.cc(getFileUrl):83 Redirecting ./x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm request to geoip location.
2024-12-23 13:30:43 <1> Z840(10410) [zypp-core] PathInfo.cc(assert_dir):358 mkdir /var/tmp/AP_0xlQ7fxJ/x86_64 00755
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaMultiCurl.cc(doGetFileCopy):1510 dest: /var/tmp/AP_0xlQ7fxJ/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaMultiCurl.cc(doGetFileCopy):1511 temp: /var/tmp/AP_0xlQ7fxJ/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm.new.zypp.7TUb55
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaCurl.cc(doGetFileCopyFile):1204 ./x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media] MediaNetworkCommonHandler.cc(getFileUrl):83 Redirecting ./x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm request to geoip location.
2024-12-23 13:30:43 <1> Z840(10410) [zypp::media++] MediaCurl.cc(doGetFileCopyFile):1214 URL: http://cdn.opensuse.org/tumbleweed/repo/oss/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm
2024-12-23 13:30:44 <1> Z840(10410) [zypp::media++] MediaMultiCurl.cc(progressCallback):1460 looks_like_meta_file: 0
2024-12-23 13:30:44 <1> Z840(10410) [zypp::media++] MediaMultiCurl.cc(doGetFileCopy):1549 HTTP response: 200
2024-12-23 13:30:44 <1> Z840(10410) [zypp-core] PathInfo.cc(rename):753 rename /var/tmp/AP_0xlQ7fxJ/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm.new.zypp.7TUb55 -> /var/tmp/AP_0xlQ7fxJ/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm
2024-12-23 13:30:44 <1> Z840(10410) [zypp::media++] MediaMultiCurl.cc(doGetFileCopy):1713 done: /var/tmp/AP_0xlQ7fxJ/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm{- 0644 0/0 size 58859}
2024-12-23 13:30:44 <1> Z840(10410) [zypp::media++] MediaHandler.cc(provideFile):977 provideFile([1]./x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm{57.5 KiB|sha512-5ef52e0cdb45b216178ae6b00a191f23cc1e338272229a5dbebabc0710157ce092c022857e1fdab71fb09c69bd41744dd336fe423d9a844b3d56e0e3afe174e5|})
2024-12-23 13:30:44 <1> Z840(10410) [zypp::fetcher] Fetcher.cc(validate):387 Checking job [/var/tmp/AP_0xlQ7fxJ/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm] (2 checkers )
2024-12-23 13:30:44 <1> Z840(10410) [zypp-core] PathInfo.cc(hardlinkCopy):921 hardlinkCopy /var/tmp/AP_0xlQ7fxJ/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm -> /var/cache/zypp/packages/repo-oss/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm
2024-12-23 13:30:44 <1> Z840(10410) [zypp::fetcher++] MediaSetAccess.cc(releaseFile):91 Going to release file ./x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm from media number 1
2024-12-23 13:30:44 <1> Z840(10410) [zypp-core] PathInfo.cc(unlink):711 unlink /var/tmp/AP_0xlQ7fxJ/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm
2024-12-23 13:30:44 <1> Z840(10410) [Progress++] progressdata.cc(report):75 {#413|} START
2024-12-23 13:30:44 <1> Z840(10410) [Progress++] progressdata.cc(report):95 {#413|} END
2024-12-23 13:30:44 <1> Z840(10410) [zypp::media] RepoProvideFile.cc(provideFile):319 provideFile at /var/cache/zypp/packages/repo-oss/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm
2024-12-23 13:30:44 <1> Z840(10410) [zypp] PackageProvider.cc(providePackage):525 provided Package (19899)libunwind8-32bit-1.8.1-2.1.x86_64(repo-oss) at /var/cache/zypp/packages/repo-oss/x86_64/libunwind8-32bit-1.8.1-2.1.x86_64.rpm
2024-12-23 13:30:44 <1> Z840(10410) [zypp] PackageProvider.cc(providePackage):424 provide Package (17733)libodbc2-32bit-2.3.12-3.2.x86_64(repo-oss)
2024-12-23 13:30:44 <1> Z840(10410) [zypp++] DeltaCandidates.cc(deltaRpms):83 package: (17733)libodbc2-32bit-2.3.12-3.2.x86_64(repo-oss)

You can see that it got stuck for ~16 minutes waiting to download libunwind8-32bit-1.8.1-2.1.x86_64.rpm. It eventually bails out and asks if I want to retry, abort, etc., at which point I clicked "r" to retry and the package then downloads successfully. I'm happy to spend more time debugging and providing more information so we can figure out what's going on and get it fixed for myself and anyone else that may run into this. I'm also wondering if there is a short term config or environmental variable I can set to make libcurl not wait 15+ minutes to error out (ideally just retry after a few seconds). Right now I have to CTRL-C out of zypper dup all the time and restart it when it hangs like this waiting on a package download.