Missing support fr RPC API over HTTPS

[ze42]

Checklist

• This is a bug report, not a question. Ask questions on discuss.ipfs.tech.
• I have searched on the issue tracker for my bug.
• I am running the latest kubo version or have an issue updating.

Installation method

ipfs-update or dist.ipfs.tech

Version

/ipfs version --all
Kubo version: 0.30.0
Repo version: 16
System version: amd64/linux
Golang version: go1.22.7

Config

(probably not relevant)

{"API":{"HTTPHeaders":{}},"Addresses":{"API":"/ip4/127.0.0.1/tcp/5001","Announce":[],"AppendAnnounce":[],"Gateway":"/ip4/127.0.0.1/tcp/8080","NoAnnounce":[],"Swarm":["/ip4/0.0.0.0/tcp/4001","/ip6/::/tcp/4001","/ip4/0.0.0.0/udp/4001/quic-v1","/ip4/0.0.0.0/udp/4001/quic-v1/webtransport","/ip6/::/udp/4001/quic-v1","/ip6/::/udp/4001/quic-v1/webtransport"]},"AutoNAT":{},"Bootstrap":["/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN","/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa","/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb","/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt","/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ","/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"],"DNS":{"Resolvers":{}},"Datastore":{"BloomFilterSize":0,"GCPeriod":"1h","HashOnRead":false,"Spec":{"mounts":[{"child":{"path":"blocks","shardFunc":"/repo/flatfs/shard/v1/next-to-last/2","sync":true,"type":"flatfs"},"mountpoint":"/blocks","prefix":"flatfs.datastore","type":"measure"},{"child":{"compression":"none","path":"datastore","type":"levelds"},"mountpoint":"/","prefix":"leveldb.datastore","type":"measure"}],"type":"mount"},"StorageGCWatermark":90,"StorageMax":"10GB"},"Discovery":{"MDNS":{"Enabled":true}},"Experimental":{"FilestoreEnabled":false,"Libp2pStreamMounting":false,"OptimisticProvide":false,"OptimisticProvideJobsPoolSize":0,"P2pHttpProxy":false,"StrategicProviding":false,"UrlstoreEnabled":false},"Gateway":{"DeserializedResponses":null,"DisableHTMLErrors":null,"ExposeRoutingAPI":null,"HTTPHeaders":{},"NoDNSLink":false,"NoFetch":false,"PublicGateways":null,"RootRedirect":""},"Identity":{"PeerID":"12D3KooWHmUkKz2ys3uQpNStsCwDP7DBpvPMMF4182KsBQ55xb1A"},"Internal":{},"Ipns":{"RecordLifetime":"","RepublishPeriod":"","ResolveCacheSize":128},"Migration":{"DownloadSources":[],"Keep":""},"Mounts":{"FuseAllowOther":false,"IPFS":"/ipfs","IPNS":"/ipns"},"Peering":{"Peers":null},"Pinning":{"RemoteServices":{}},"Plugins":{"Plugins":null},"Provider":{"Strategy":""},"Pubsub":{"DisableSigning":false,"Router":""},"Reprovider":{},"Routing":{"Methods":null,"Routers":null},"Swarm":{"AddrFilters":null,"ConnMgr":{},"DisableBandwidthMetrics":false,"DisableNatPortMap":false,"RelayClient":{},"RelayService":{},"ResourceMgr":{},"Transports":{"Multiplexers":{},"Network":{},"Security":{}}}}

Description

When trying to connect to a distant API over https, it fails and try to connect with http.

/ipfs --api /dns/myapihost.mydomain.com/tcp/443/https files stat /
2024-10-07T16:00:52.505+0200	ERROR	cmds/http	http/parse.go:212	could not guess encoding from content type "text/html"
Error: unknown error content type: text/html

When checking logs from the RP we have in place receiving the request, we see "scheme": "http".

When trying to access directly the same URL with curl and https, it works properly.

[lidel]

Thank you for reporting this bug.

Indeed, /https and /tls/http are not supported. https:// hint is not forwarded to go-ipfs-cmds client.

The host value in:

kubo/cmd/ipfs/kubo/start.go

Line 357 in 091bc08

remoteVersion, err := getRemoteVersion(tracingWrappedExecutor{cmdhttp.NewClient(host, opts...)})

likely does not include protocol, just host, and http:// is added in:

https://github.com/ipfs/go-ipfs-cmds/blob/ae8443f905e4604e2a0c001ac20a7c8fe4dd7948/http/client.go#L87-L89

Potential quick fix is to make sure kubo/cmd/ipfs/kubo/start.go prepends https:// to host if apiAddr contains /https or /tls/http

@ze42 would you have time to open PR with a fix?

[Sairaviteja27]

@lidel I’m new to this repository, but I believe this fix requires a small change, and I would like to contribute to it. Can I take this?

[lidel]

@Sairaviteja27 sure, submit PR once you have something ready for review 👍

[Sairaviteja27]

Hi,

Just wanted to share an update. I was occupied with some personal work and initially faced a few challenges due to the earlier version. However, I updated to the latest version (0.32.1) and tried to reproduce the issue, but I didn't encounter any errors.

Next, I will try downgrading my IPFS version to check if the issue persists and also investigate further by cloning the repository.

[Sairaviteja27]

After some investigation, I found that if I prepend https:// when the apiAddr contains /https or /tls/http, the cmdhttp.NewClient method call prepends http:// again to the address. I tried changing the serverAddress, but it’s not possible because both the serverAddress and client are unexported. The only solution I can think of is adding a function like this in the go-ipfs-cmds library and using it as an option in start.go:

func ClientWithHTTPSServerAddress() ClientOpt {
    return func(c *client) {
        c.serverAddress = strings.Replace(c.serverAddress, "http://", "https://", 1)
    }
}

This would allow the client to send HTTPS requests, but additional changes may be needed for handling HTTPS responses. When I tried this, I encountered the error: http: server gave HTTP response to HTTPS client

@lidel @gammazero
Correct me if I’m mistaken or overlooked anything, and please advise on the next steps for resolving this issue.