Error registering repository from org #729

magandrez · 2024-11-25T20:14:05Z

I expected to add a repository for the organization I belong in Github upon issuing forge-add-repository, however, I see the following error:

ghub-repository-id: Repository "ORG/REPO" does not exist on "api.github.com".
Maybe it was renamed and you have to update "remote.<remote>.url"?

I re-generated the token and made sure the roles required were included.
I run password-store, my entry looks like follows

pass show api.github.com/magandrez^forge
machine api.github.com login magandrez^forge password TOKEN

where TOKEN has been anonymized.

The trace below (anonymized organization and repo with ORG and REPO respectively):

Debugger entered--Lisp error: (error "Repository \"ORG/REPO\" does not exist on \"api....")
  transient--exit-and-debug(error (error "Repository \"ORG/REPO\" does not exist on \"api...."))
  error("Repository %S does not exist on %S.\n%s%S?" "ORG/REPO" "api.github.com" "Maybe it was renamed and you have to update " "remote.<remote>.url")
  ghub-repository-id("ORG" "REPO" :host "api.github.com" :auth forge :forge github :noerror nil)
  #f(compiled-function (class host owner name &optional stub noerror) "Return (OUR-ID . THEIR-ID) of the specified repository.\nIf optional STUB is non-nil, then the IDs are not guaranteed to\nbe unique.  Otherwise this method has to make an API request to\nretrieve THEIR-ID, the repository's ID on the forge.  In that\ncase OUR-ID derives from THEIR-ID and is unique across all\nforges and hosts." #<bytecode -0x18db51c69e733ccd>)(forge-github-repository "github.com" "ORG" "REPO" nil nil)
  apply(#f(compiled-function (class host owner name &optional stub noerror) "Return (OUR-ID . THEIR-ID) of the specified repository.\nIf optional STUB is non-nil, then the IDs are not guaranteed to\nbe unique.  Otherwise this method has to make an API request to\nretrieve THEIR-ID, the repository's ID on the forge.  In that\ncase OUR-ID derives from THEIR-ID and is unique across all\nforges and hosts." #<bytecode -0x18db51c69e733ccd>) forge-github-repository ("github.com" "ORG" "REPO" nil nil))
  forge--repository-ids(forge-github-repository "github.com" "ORG" "REPO" nil nil)
  #f(compiled-function (&rest rest) "((HOST OWNER NAME) &optional REMOTE DEMAND)\n\nReturn the repository identified by HOST, OWNER and NAME.\nSee `forge-alist' for valid Git hosts." #<bytecode -0x18215f7ee6ba562a>)(("github.com" "ORG" "REPO") nil :insert!)
  apply(#f(compiled-function (&rest rest) "((HOST OWNER NAME) &optional REMOTE DEMAND)\n\nReturn the repository identified by HOST, OWNER and NAME.\nSee `forge-alist' for valid Git hosts." #<bytecode -0x18215f7ee6ba562a>) ("github.com" "ORG" "REPO") (nil :insert!))
  forge-get-repository(("github.com" "ORG" "REPO") nil :insert!)
  #f(compiled-function (repo &optional noerror demand) #<bytecode -0x1ffe6e3d0eeb5362>)(#<forge-github-repository forge-github-repository-1579a03eabac> nil :insert!)
  apply(#f(compiled-function (repo &optional noerror demand) #<bytecode -0x1ffe6e3d0eeb5362>) #<forge-github-repository forge-github-repository-1579a03eabac> (nil :insert!))
  forge-get-repository(#<forge-github-repository forge-github-repository-1579a03eabac> nil :insert!)
  forge-add-repository(#<forge-github-repository forge-github-repository-1579a03eabac>)
  #f(compiled-function (repo) (interactive #f(compiled-function () #<bytecode 0x198c6fbd6dd2>)) #<bytecode -0xe0242eff4570ec6>)(#<forge-github-repository forge-github-repository-1579a03eabac>)
  apply(#f(compiled-function (repo) (interactive #f(compiled-function () #<bytecode 0x198c6fbd6dd2>)) #<bytecode -0xe0242eff4570ec6>) #<forge-github-repository forge-github-repository-1579a03eabac>)
  #f(compiled-function (fn &rest args) #<bytecode -0x4cfd49b6cfc1a00>)(#f(compiled-function (repo) (interactive #f(compiled-function () #<bytecode 0x198c6fbd6dd2>)) #<bytecode -0xe0242eff4570ec6>) #<forge-github-repository forge-github-repository-1579a03eabac>)
  apply(#f(compiled-function (fn &rest args) #<bytecode -0x4cfd49b6cfc1a00>) #f(compiled-function (repo) (interactive #f(compiled-function () #<bytecode 0x198c6fbd6dd2>)) #<bytecode -0xe0242eff4570ec6>) #<forge-github-repository forge-github-repository-1579a03eabac>)
  (lambda (fn &rest args) (interactive #f(compiled-function (spec) #<bytecode -0x1b55884e8d4a0747>)) (apply '#f(compiled-function (fn &rest args) #<bytecode -0x4cfd49b6cfc1a00>) fn args))(#f(compiled-function (repo) (interactive #f(compiled-function () #<bytecode 0x198c6fbd6dd2>)) #<bytecode -0xe0242eff4570ec6>) #<forge-github-repository forge-github-repository-1579a03eabac>)
  apply((lambda (fn &rest args) (interactive #f(compiled-function (spec) #<bytecode -0x1b55884e8d4a0747>)) (apply '#f(compiled-function (fn &rest args) #<bytecode -0x4cfd49b6cfc1a00>) fn args)) #f(compiled-function (repo) (interactive #f(compiled-function () #<bytecode 0x198c6fbd6dd2>)) #<bytecode -0xe0242eff4570ec6>) #<forge-github-repository forge-github-repository-1579a03eabac>)
  transient:forge-add-repository::16(#<forge-github-repository forge-github-repository-1579a03eabac>)
  funcall-interactively(transient:forge-add-repository::16 #<forge-github-repository forge-github-repository-1579a03eabac>)
  command-execute(transient:forge-add-repository::16)

The version I run is the following:

Magit 20241124.1509 [>= 20241124.1509], Transient 20241125.1302, Forge 20241123.2131, Git 2.45.2, Emacs 29.4, gnu/linux

The text was updated successfully, but these errors were encountered:

tarsius · 2024-11-25T20:52:34Z

Maybe it was renamed and you have to update "remote..url"?

You would not be the first user who read and quoted that, and could have fixed the issue by doing just that but still opened such an issue instead. Please double check that URL. Check whether the capitalization has changed, for example.

And while I am not sure not this is needed here, also make sure the token has the read:org permission.

magandrez · 2024-11-26T08:06:14Z

Maybe it was renamed and you have to update "remote..url"?

You would not be the first user who read and quoted that, and could have fixed the issue by doing just that but still opened such an issue instead. Please double check that URL. Check whether the capitalization has changed, for example.

Hmmm, I quite not get the informational message. This is a repo that I have just cloned. Below a bit more details that I hope help troubleshoot:

I post here the ./git/config for the repo, anonymized

[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
	hooksPath = .husky
[advice]
	forceDeleteBranch = false
[forge]
	remote = origin
[remote "origin"]
	url = [email protected]:ORG/REPO.git
	fetch = +refs/heads/*:refs/remotes/origin/*
	fetch = +refs/pull/*/head:refs/pullreqs/*

The remote has been added like:

$ git remote add origin -m develop [email protected]:ORG/REPO.git

I did notice that, when doing forge-pull on a magit frame for the repo in question for the first time, the dialog mentions https://github.com/ORG/repo. Not sure if this is just a string or a value that I can change somehow.

And while I am not sure not this is needed here, also make sure the token has the read:org permission.

Yeah, I double checked this (re-created the token making sure the roles were set).

tarsius · 2024-11-26T13:34:49Z

Hmmm, I quite not get the informational message. This is a repo that I have just cloned.

After a rename, when using git and the web interface, the old url redirects to the new. The same does not happen when using the API. So it is possible to copy an URL from some outdated wiki, use that to clone and succeed, but have the API calls still fail.

You had to anonymize, but note that in your message both ORG/REPO and ORG/repo appear. I've the same sort of discrepancy exists in the actual values, that would be the cause.

magandrez · 2024-11-26T14:43:23Z

Hmmm, I quite not get the informational message. This is a repo that I have just cloned.

After a rename, when using git and the web interface, the old url redirects to the new. The same does not happen when using the API. So it is possible to copy an URL from some outdated wiki, use that to clone and succeed, but have the API calls still fail.

I copied the URL to do git clone from here:

Note that this is the SSH version of the URL that Github provides. I still don't understand what are you referring to.

You had to anonymize, but note that in your message both ORG/REPO and ORG/repo appear. I've the same sort of discrepancy exists in the actual values, that would be the cause.

A typo. The capitals were meant to highlight that anonymized. I did not use sed for that string where ORG/repo appears to anonymize it, instead I wrote it manually here, in this issue.

All of the appearances where the anonymization happens match what I expect in terms of the organization and repository names.

tarsius · 2024-11-26T15:18:50Z

I copied the URL to do git clone from here:

In that case we can (almost) know that it is up-to-date.

I.e., it now makes sense to look at other possible causes.

I still don't understand what are you referring to.

This is what could have happened: (And I am not able to know whether it has. I can only ask you to double-check and then have to take your word for it.)

You went to your company's wiki and it said "Do git clone [email protected]:foo/Bar.git".
You copy-paste that into a terminal and it works. origin.url is set to [email protected]:foo/Bar.git in .git/config.
However, what actually happened behind the scene is that Github served you the repository for [email protected]:foo/bar.git. The repository used to be named Bar but was renamed to bar. Github automatically redirects on the web and when using git.
You tell Forge to fetch API data for that repository, resulting in the above error.
This did not work because Forge extracts the necessary information from the [email protected]:foo/Bar.git it finds in .git/config. The information stored there is incorrect.
The reason git clone succeeded is that Github redirects renamed repositories in that case.
The reason the API access does not succeeded is that Github does NOT redirects renamed repositories when using the API.

magandrez · 2024-11-26T15:55:59Z

I copied the URL to do git clone from here:

In that case we can (almost) know that it is up-to-date.

I.e., it now makes sense to look at other possible causes.

I still don't understand what are you referring to.

This is what could have happened: (And I am not able to know whether it has. I can only ask you to double-check and then have to take your word for it.)

You went to your company's wiki and it said "Do git clone [email protected]:foo/Bar.git".

You copy-paste that into a terminal and it works. origin.url is set to [email protected]:foo/Bar.git in .git/config.

However, what actually happened behind the scene is that Github served you the repository for [email protected]:foo/bar.git. The repository used to be named Bar but was renamed to bar. Github automatically redirects on the web and when using git.

You tell Forge to fetch API data for that repository, resulting in the above error.

This did not work because Forge extracts the necessary information from the [email protected]:foo/Bar.git it finds in .git/config. The information stored there is incorrect.

The reason git clone succeeded is that Github redirects renamed repositories in that case.

The reason the API access does not succeeded is that Github does NOT redirects renamed repositories when using the API.

I understand if that could be the case, the mapping that Github does under the hood.

What actually happened is the following:

I got access to the Github Organization.
I visited https://github.com/ORG/REPO and opened the cloning menu I screenshot in the previous post.
I copied the SSH version of the URL and cloned it as mentioned in prior messages.
I created a (Classic) Github token with the required roles.
I made an entry on my password-store as mentioned in the original message in this issue.
I made sure the auth-source is password-store.
I run forge-pull and received the error message I posted in the original message in this issue.

I still don't understand what have I missed, after re-reading forge's and ghub's documentation on setting up a new forge 😞

magandrez · 2024-11-26T16:02:39Z

For reference, the .git/config anonymized (again, capitals mine) is what I expect to be regarding the remote URL:

[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
	hooksPath = .husky
[advice]
	forceDeleteBranch = false
[forge]
	remote = origin
[remote "origin"]
	url = [email protected]:ORG/REPO.git
	fetch = +refs/heads/*:refs/remotes/origin/*
	fetch = +refs/pull/*/head:refs/pullreqs/*
[branch "develop"]
	remote = origin
	merge = refs/heads/develop

The line url = [email protected]:ORG/REPO.git is letter-by-letter, the same as the URL offered by Github to clone the repository via SSH.

tarsius · 2024-11-26T17:12:51Z

Organization owners can restrict API access. Contact the admin of your organization to learn whether (1) you have the permissions to use (2) classic tokens.

magandrez · 2024-11-27T08:40:14Z

Organization owners can restrict API access. Contact the admin of your organization to learn whether (1) you have the permissions to use (2) classic tokens.

This was activated now following this documentation with me being present.

I am not aware of anything else per Github documentation (enrollment or any other setting needed). If you are, please let me know.

One thing that strikes me is that, if I modify the token to be foobar, the error remains the same, so I wonder if the problem is somewhere else 🤔

tarsius · 2024-11-27T16:32:34Z

if I modify the token to be foobar, the error remains the same,

In that case the problem is that Github returns 404 for resources that exist but you are not allowed to access, to avoid disclosing the [non-]existence of said resources. That indeed often makes it harder to figure out what is going wrong.

magandrez · 2024-11-27T16:36:56Z

if I modify the token to be foobar, the error remains the same,

In that case the problem is that Github returns 404 for resources that exist but you are not allowed to access, to avoid disclosing the [non-]existence of said resources. That indeed often makes it harder to figure out what is going wrong.

Ok, but how come if I try to setup forge for my own repository under no organization, I get the same response? 🤔

tarsius · 2024-11-27T16:44:23Z

I've it's a private repository, then Github also wouldn't want to disclose its existence to someone claiming to be you, just like for a private organization repository. I've it's public, the issue could be that Github errs on the side of playing it safe too much.

Which reminds me, is the repository you originally wanted to access private?

magandrez · 2024-11-27T17:10:59Z

Which reminds me, is the repository you originally wanted to access private?

Yes, the org repo I would like to fetch is private. Any further consideration I need to take regarding that?

tarsius · 2024-11-27T18:39:49Z

Can't think of anything right now. Also, I likely will stop responding for a week or so, because I'm rather busy.

magandrez · 2024-12-01T12:35:56Z

Can't think of anything right now. Also, I likely will stop responding for a week or so, because I'm rather busy.

Ok, I understand. You own your agenda and decide where to put time.

However, let me explain once again that I believe this is not a "support" issue.

After having read all documentation from forge and ghub (including How-Forge-Detection-Works checked closed issues and having made sure the token has the necessary permissions and the authentication works as expected, I am confident in saying the problem is not a support problem, but rather a non-documented behavior or a bug.

Here, if it is of any help, the trace for attempted registering magit/forge (same behavior, same error as with other private repos).

Debugger entered--Lisp error: (error "Repository \"magit/forge\" does not exist on \"api.gi...")
  transient--exit-and-debug(error (error "Repository \"magit/forge\" does not exist on \"api.gi..."))
  error("Repository %S does not exist on %S.\n%s%S?" "magit/forge" "api.github.com" "Maybe it was renamed and you have to update " "remote.<remote>.url")
  ghub-repository-id("magit" "forge" :host "api.github.com" :auth forge :forge github :noerror nil)
  #f(compiled-function (class host owner name &optional stub noerror) "Return (OUR-ID . THEIR-ID) of the specified repository.\nIf optional STUB is non-nil, then the IDs are not guaranteed to\nbe unique.  Otherwise this method has to make an API request to\nretrieve THEIR-ID, the repository's ID on the forge.  In that\ncase OUR-ID derives from THEIR-ID and is unique across all\nforges and hosts." #<bytecode 0xa64aba46bdab733>)(forge-github-repository "github.com" "magit" "forge" nil nil)
  apply(#f(compiled-function (class host owner name &optional stub noerror) "Return (OUR-ID . THEIR-ID) of the specified repository.\nIf optional STUB is non-nil, then the IDs are not guaranteed to\nbe unique.  Otherwise this method has to make an API request to\nretrieve THEIR-ID, the repository's ID on the forge.  In that\ncase OUR-ID derives from THEIR-ID and is unique across all\nforges and hosts." #<bytecode 0xa64aba46bdab733>) forge-github-repository ("github.com" "magit" "forge" nil nil))
  forge--repository-ids(forge-github-repository "github.com" "magit" "forge" nil nil)
  #f(compiled-function (&rest rest) "((HOST OWNER NAME) &optional REMOTE DEMAND)\n\nReturn the repository identified by HOST, OWNER and NAME.\nSee `forge-alist' for valid Git hosts." #<bytecode -0x120926881f4bb0b2>)(("github.com" "magit" "forge") nil :insert!)
  apply(#f(compiled-function (&rest rest) "((HOST OWNER NAME) &optional REMOTE DEMAND)\n\nReturn the repository identified by HOST, OWNER and NAME.\nSee `forge-alist' for valid Git hosts." #<bytecode -0x120926881f4bb0b2>) ("github.com" "magit" "forge") (nil :insert!))
  forge-get-repository(("github.com" "magit" "forge") nil :insert!)
  #f(compiled-function (repo &optional noerror demand) #<bytecode -0x1140f6f93a8cb362>)(#<forge-github-repository forge-github-repository-1563b60c2878> nil :insert!)
  apply(#f(compiled-function (repo &optional noerror demand) #<bytecode -0x1140f6f93a8cb362>) #<forge-github-repository forge-github-repository-1563b60c2878> (nil :insert!))
  forge-get-repository(#<forge-github-repository forge-github-repository-1563b60c2878> nil :insert!)
  forge-add-repository(#<forge-github-repository forge-github-repository-1563b60c2878>)
  #f(compiled-function (repo) (interactive #f(compiled-function () #<bytecode 0x198718e5cce6>)) #<bytecode -0xe00d52b51057b95>)(#<forge-github-repository forge-github-repository-1563b60c2878>)
  apply(#f(compiled-function (repo) (interactive #f(compiled-function () #<bytecode 0x198718e5cce6>)) #<bytecode -0xe00d52b51057b95>) #<forge-github-repository forge-github-repository-1563b60c2878>)
  #f(compiled-function (fn &rest args) #<bytecode 0x1b5ecff65b5d76c8>)(#f(compiled-function (repo) (interactive #f(compiled-function () #<bytecode 0x198718e5cce6>)) #<bytecode -0xe00d52b51057b95>) #<forge-github-repository forge-github-repository-1563b60c2878>)
  apply(#f(compiled-function (fn &rest args) #<bytecode 0x1b5ecff65b5d76c8>) #f(compiled-function (repo) (interactive #f(compiled-function () #<bytecode 0x198718e5cce6>)) #<bytecode -0xe00d52b51057b95>) #<forge-github-repository forge-github-repository-1563b60c2878>)
  (lambda (fn &rest args) (interactive #f(compiled-function (spec) #<bytecode -0x1a8e39349eda4bf3>)) (apply '#f(compiled-function (fn &rest args) #<bytecode 0x1b5ecff65b5d76c8>) fn args))(#f(compiled-function (repo) (interactive #f(compiled-function () #<bytecode 0x198718e5cce6>)) #<bytecode -0xe00d52b51057b95>) #<forge-github-repository forge-github-repository-1563b60c2878>)
  apply((lambda (fn &rest args) (interactive #f(compiled-function (spec) #<bytecode -0x1a8e39349eda4bf3>)) (apply '#f(compiled-function (fn &rest args) #<bytecode 0x1b5ecff65b5d76c8>) fn args)) #f(compiled-function (repo) (interactive #f(compiled-function () #<bytecode 0x198718e5cce6>)) #<bytecode -0xe00d52b51057b95>) #<forge-github-repository forge-github-repository-1563b60c2878>)
  transient:forge-add-repository::2381(#<forge-github-repository forge-github-repository-1563b60c2878>)
  funcall-interactively(transient:forge-add-repository::2381 #<forge-github-repository forge-github-repository-1563b60c2878>)
  command-execute(transient:forge-add-repository::2381)

Magit 20241130.1707 [>= 20241130.1707], Transient 20241125.1302, Forge 20241201.700, Git 2.45.2, Emacs 29.4, gnu/linux

Let me know if there is any way I can help debugging this and find out the root cause. I think several tangents were already explored.

Duy-X · 2024-12-09T20:44:20Z

I was facing this issue earlier, then I realized I had to configure the SSO by clicking the button and specifically authorize the token's use for my organization:

Your problem might be different, but just wanted to share the solution that worked for me.

magandrez · 2024-12-13T08:07:16Z

@Duy-X Thanks for your take.

It seems like the screenshot you posted is taken from the Github tokens view, the step you mention is required for organizations that use or enforce SAML SSO, but the organizations I belong to are not using nor enforcing such, and so I don't even get to see the button shown.

magandrez · 2024-12-15T19:21:52Z

@tarsius I narrowed down the issue. IF I switch auth-source to an encrypted authinfo file (with the same content as the entry in my password-store, then I can make the pull.

The config I had that was causing trouble is below:

(use-package auth-source
  :no-require
  :config (setq auth-source-debug t
           auth-source-pass-extra-query t))
(auth-source-pass-enable)

I don't see anything weird, does this grant registering a bug upstream?

tarsius · 2024-12-15T21:32:36Z

Try this patch #720 (comment).

magandrez · 2024-12-16T07:51:50Z

Try this patch #720 (comment).

I did, and gave me a 401 error.

tarsius added the support User needs some help label Nov 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error registering repository from org #729

Error registering repository from org #729

magandrez commented Nov 25, 2024

tarsius commented Nov 25, 2024

magandrez commented Nov 26, 2024 •

edited

Loading

tarsius commented Nov 26, 2024

magandrez commented Nov 26, 2024 •

edited

Loading

tarsius commented Nov 26, 2024 •

edited

Loading

magandrez commented Nov 26, 2024

magandrez commented Nov 26, 2024 •

edited

Loading

tarsius commented Nov 26, 2024 •

edited

Loading

magandrez commented Nov 27, 2024

tarsius commented Nov 27, 2024

magandrez commented Nov 27, 2024

tarsius commented Nov 27, 2024

magandrez commented Nov 27, 2024

tarsius commented Nov 27, 2024

magandrez commented Dec 1, 2024

Duy-X commented Dec 9, 2024

magandrez commented Dec 13, 2024

magandrez commented Dec 15, 2024

tarsius commented Dec 15, 2024

magandrez commented Dec 16, 2024

Error registering repository from org #729

Error registering repository from org #729

Comments

magandrez commented Nov 25, 2024

tarsius commented Nov 25, 2024

magandrez commented Nov 26, 2024 • edited Loading

tarsius commented Nov 26, 2024

magandrez commented Nov 26, 2024 • edited Loading

tarsius commented Nov 26, 2024 • edited Loading

magandrez commented Nov 26, 2024

magandrez commented Nov 26, 2024 • edited Loading

tarsius commented Nov 26, 2024 • edited Loading

magandrez commented Nov 27, 2024

tarsius commented Nov 27, 2024

magandrez commented Nov 27, 2024

tarsius commented Nov 27, 2024

magandrez commented Nov 27, 2024

tarsius commented Nov 27, 2024

magandrez commented Dec 1, 2024

Duy-X commented Dec 9, 2024

magandrez commented Dec 13, 2024

magandrez commented Dec 15, 2024

tarsius commented Dec 15, 2024

magandrez commented Dec 16, 2024

magandrez commented Nov 26, 2024 •

edited

Loading

magandrez commented Nov 26, 2024 •

edited

Loading

tarsius commented Nov 26, 2024 •

edited

Loading

magandrez commented Nov 26, 2024 •

edited

Loading

tarsius commented Nov 26, 2024 •

edited

Loading