How to use pipeline for monorepo?

[rababahmed]

I have a mono repo like this:

/project
  /front-end
  /back-end
   /copilot
   Dockerfile

However, when I'm deploying through codepipeline it fails with the following code:

[Container] 2021/11/09 19:44:12 Waiting for agent ping
--
2 | [Container] 2021/11/09 19:44:13 Waiting for DOWNLOAD_SOURCE
3 | [Container] 2021/11/09 19:44:13 Phase is DOWNLOAD_SOURCE
4 | [Container] 2021/11/09 19:44:13 CODEBUILD_SRC_DIR=/codebuild/output/src340334424/src
5 | [Container] 2021/11/09 19:44:15 Phase complete: DOWNLOAD_SOURCE State: FAILED
6 | [Container] 2021/11/09 19:44:15 Phase context status code: YAML_FILE_ERROR Message: stat /codebuild/output/src340334424/src/copilot/buildspec.yml: no such file or directory

Both my buildspec.yml and pipeline.yml are defaults generated by the copilot cli and yes they're pushed upstream to github.

What could be the possible solutions to this? Would really appreciate it!

[iamhopaul123]

Hello @rababahmed. Copilot creates a pipeline per workspace, so currently you would need to place each microservice in its own GH repo. For example, i guess you ran copilot init within "front-end" and "back-end" respectively before, you could then run copilot pipeline init within those two repo and give their GH repo URL respectively as well. Glad to answer any follow up questions about it!

[rababahmed]

But my front-end is hosted elsewhere (NextJS on Vercel). I only have copilot and pipeline initiated for my back-end. As I mentioned previously, it's a monorepo and it looks like code build can't find the yaml file. Is there any way I could point it towards my back-end directory? Perhaps in my buildspec or somewhere? Sorry I'm pretty noob when it comes to devops.

[iamhopaul123]

Gotcha, in this case can you run copilot pipeline init within the "back-end" directory and give the URL for the backend repo? On your GitHub remote repo, it should only include "backend" service like

/copilot (for back-end)
... your backend code

[iamhopaul123]

Is there any way I could point it towards my back-end directory?

That's not something we can do currently as we hardcoded the buildspec.yaml location to be "copilot/buildspec.yml". From the question it seems like your buildspec location is either "project/back-end/copilot/buildspec.yml" or "back-end/copilot/buildspec.yml".

[rababahmed]

Yes, my build spec is located in project/back-end/copilot/buildspec.yml Atm, I really don't want to break up my mono-repo as that'd mean managing multiple packages and all as code is shared, it's gonna be very hectic for me. I've seen in another discussion #2717 where someone was having the same problem and one of the maintainers suggested to double check if the buildspec and pipeline files were pushed upstream to solve his problem. In my case tho, it is pushed upstream.

So, there's no possible solution to this right now?

[iamhopaul123]

Yes there is actually. Since you don't want to change the mono-repo pattern, it would make more sense if you make the root directory to be the workspace instead of within the back-end dir. You could simply move the back-end/copilot folder out of it to the root dir and push the change to the remote. After that in root dir you can run copilot pipeline update to update the pipeline.