ENH: Answer the question - what's the difference between pip and conda and why does pip install . -e install a package that's not on pypi?

[lwasser]

This page begins to explain the difference but it doesn't answer the questions posed in the issue title above.

What is the difference between the two
How can pip install -e . install my package if it's not on pypi?

This may be related to the environments discussed in another issue.

cc @chenghlee

We might want to add this content on this page

And also it might be good to consider personas which would help a user find the things that they need to know ...

[lwasser]

the narrative idea of having personas and trying to present those stories with how to do the things that the persona probably needs to do...

[sneakers-the-rat]

i don't know anything about conda, so i would be bad at answering the first one aside from saying "conda manages system dependencies in addition to python dependencies"

I can def answer the second one. It looks like we already do actually: https://www.pyopensci.org/python-package-guide/tutorials/installable-code.html#step-5-install-your-package-locally

python-package-guide/tutorials/installable-code.md

Lines 531 to 551 in f31ba96

	:::{admonition} What does `python -m pip install -e .` do?
	:class: tip
	Let's break down `python -m pip install -e .`
	`python -m pip install -e .` installs your package into the current active
	Python environment in **editable mode** (`-e`). Installing your package in
	editable mode, allows you to work on your code and then test the updates
	interactively in your favorite Python interface. One important caveat of editable mode is that every time you update your code, you may need to restart Python.
	If you wish to install the package regularly (not in editable
	mode) you can use:
	- `python -m pip install . `
	**Using `python -m` when calling `pip`**
	Above, you use`python -m` to call the version of pip installed into your
	current active environment. `python -m` is important to ensure that you are
	calling the version of pip installed in your current environment.
	:::

the only thing we'd need to add there to answer this specific question re: on pypi/not on pypi is just that we are asking it to install "the current directory" when we ask to install ".". We can actually do that from any directory using any other directory, like pip install -e ~/Desktop/some_folder (and it's not specific to -e, pip install . also works). Our package has all the information that is needed to install it no matter where we get it from - pypi is just the thing that allows us to refer to it by name without specifying where it comes from (because pypi is the default place to look). When we instead use a package name, pip will assume we want something from pypi.

I'm not sure if it would be too much detail to add a link to the pip docs which explain this fully, and be like "you can also install stuff from URLs and git repos and stuff" with an example e.g. installing numpy directly from the URL to the .whl file

[ucodery]

Jonny's summary of the second point is excellent. This is exactly what I came here to say: don't make the . some magical target, it is simply a file path and pip works on all file paths (. happens to mean "the directory I am in" on the three most popular OSs). And -e doesn't alter how the user targets a package, only how it gets installed. Pip can even install directly from URLs, and supports a huge variety of protocols, not just http(s).

Also not a typical conda user, but I would say the bigger differences between pip and conda for the end-user developer are the scope of controlled packages (as Jonny said, conda goes beyond Python packages), the default index (PyPI vs anaconda), the expected isolation method (v[irtual]env vs conda) - I don't want to say default as pip is not an environment manager and that is another difference, and for advanced users and authors - the metadata specifications.

I don't know if conda has a concept of an editable install, but I'm sure it has a concept of a local install and in that way it is similar to pip install ..