Skip to content

Conversation

SxMShaDoW
Copy link

I think the Github Action is awesome!

I was struggling to understand the use cases though, so I tried brainstorming a few of them. If I am off target, maybe you could write up a few better ones.

This PR was mostly to get the conversation started, I will not be offended if the PR doesn't get accepted.

@chiarng chiarng self-assigned this Jun 4, 2021
@chiarng
Copy link
Member

chiarng commented Jun 5, 2021

Hi @SxMShaDoW,
Thanks for sending over this PR! In typical PM fashion, I'd like to dive into your pain points around the use cases a bit better, see if we need to expand or edit the documentation here or in the other GitHub Actions. I sense that there's clarifications to be made around 3 topics:

  1. What is Instance Scan? In what cases would a dev use it?
  2. How are these GitHub Actions built on the SN CI/CD APIs intended to work together?
  3. Adopting scoped apps, global apps, and application customizations (store apps or plugin).

I've produced a few things to get the ball rolling for our dev users, would you like to take a look and see if it answers most of the questions?

Pipeline Sample

Setting up your first CI/CD pipeline with GitHub Actions (YouTube)

@dorian-cask
Copy link

dorian-cask commented Jun 6, 2021

Hi @chiarnglin,

P.S. I thought I would respond from my partner account instead of my personal one for this one - since this is where I use source control the most.

Happy to put on my "value add" hat and answer some pain points. I looked through the pipeline sample and your youtube video. You are doing great work! Hopefully my response helps articulate some of the pain points.

The biggest question I am struggling with is "When to use Github Actions vs Flow Designer + CICD Spokes/Custom Spokes?". I can't articulate the value of Github Actions over Flow Designer, just yet. Does Github Actions help in non git related cases (such as Instance Scans on Update Sets)?

The second question I am struggling with is "When do I move things to test vs doing things in dev". Your video shows that your actions in dev are: (1) Apply remote changes and (2) Publish. I believe that we should be incorporating Instance Scans and ATFs before we publish on Dev. This will prevent unnecessary rollbacks.

The third question I am struggling with is "How does a developer handle skipped records on application deploys?". Let's say you have a BR that is already in production, that needs some modification. For simplicity, let's say that a junior engineer modifies the change in an update set in dev and manually in test. In the "update set world", a "local update is newer" action will show up when retrieving update sets from dev. In the scoped app world, a skip record is generated. If an ATF is expected to pass for that BR, it may fail and cause a rollback. How would one handle that scenario with github actions?

My fourth question (un-related to CICD specifically) is: 'How does a "global scoped bundle" handle skip logs/upgrades?" Let's say I create a Global scoped app called "New Global". If I make a change in something that is in "legacy" global scope, I move the application file over to the "New Global" (reasoning: to take advance of CICD and source control). When I do an upgrade, how will those skip logs show up? Should I just be de-activate "legacy" global files and instead copy it over instead to "New Global" and deal with skip records later? Do I still use update sets in "legacy" global - if the file originated from "legacy" global?

You asked 3 questions below, so I thought I would add some clarity:

  • What is Instance Scan? In what cases would a dev use it? Instance Scan . I believe devs should use this before promoting code to any environment. This will be great for catching code smells (similar to a sonarqube)
  • How are these GitHub Actions built on the SN CI/CD APIs intended to work together? More specifically Flow Designer + Spokes
  • Adopting scoped apps, global apps, and application customizations (store apps or plugin). I think update sets are missing in this "new" world and desire a specific call out as there are still useful cases for update sets. Added a 4th question to highlight this.

@chiarng
Copy link
Member

chiarng commented Jun 7, 2021

Hi Dorian,
Thanks for going into detail on the questions you have, helps to understand what the average SN dev looking at this repo is going to be asking. :)

I've previously tried to clarify the use cases for both the CI/CD spoke on Flow Designer, as well as Team Development + Update Sets vs the new app-based model and Source Control Git branches in the following articles:

I fully get that it's hard to discover those, so perhaps we need to link them in a more discoverable way to these external integrations we are building.

But to answer a few of your questions succinctly:

  1. I recommend using Flow Designer to get acquainted with automating Application Repository based workflows, but bumping up to a dedicated pipeline tool of choice for production (unless you don't have access to said tooling).
  2. You can absolutely run Instance Scans and ATF tests in Dev, I'd suggest doing that before committing and pushing the branch up to GitHub and starting a PR. In fact, you can package your ATF tests with your application if you want to move them together. (Don't have to run them in Prod).
  3. Idea is to have your organizational processes around how changes are made to shift from Update Sets to commits on Git branches via Source Control. You can still move things via Update Sets if you want to, but assuming a record is already claimed by a global app post-Paris, whichever version of the application that already claims the file will take effect on that instance. You'd stash the changes on the next auto-deploy of a version of the app that is ready for ATF tests on the Test instance in that specific scenario. (because they weren't committed). Also, if you are using App Repo install instead of Source Control Apply Remote Changes to Test, then your app lives in sys_store_app, not sys_app table, so you wouldn't be making direct changes to the app at that point.
  4. Check out the claims resolution part of https://developer.servicenow.com/blog.do?p=/post/paris-source-control/. Again, the idea is to have folks move away from Update Sets as the mechanism to handle development and "promotion" workflows, and to move towards apps, Source Control, and app-repo based workflows in a more DevOps model. Those global files claimed by global apps would still show up in skips on upgrades, and would still have to be resolved. Then those apps would have to be committed/pushed, and re-published after resolution.

Thanks,
Chiarng

@dorian-cask
Copy link

The devproductivity docs you linked were really helpful. It answered most of my questions. The claims resolution is very interesting and I will need to sink my teeth in it more.

I am still confused on the following use case:

  1. Dev environment is linked to source control. Scoped App is called = "Awesome BR"
  2. A BR is added that adds a comment to an incident. An ATF is created for it to make sure only a comment is added. Committed. Published to local repository. Pushed to Test. Pushed to Prod.
  3. The scoped app is now in Test and Prod.
  4. A junior engineer goes into Test, changes the scope to "Awesome BR" and changes the BR from comment to work_notes (bad practice but can happen)
  5. A junior engineer goes into Dev, changes the scope to "Awesome BR" and changes the BR to add both a comment and work_notes (different from Test). Then modifies the ATF. Commits the change. Publishes to Test.
  6. My assumption is a skip record should show up because it doesn't know which one to trust. What happens when the ATF runs in Test? My assumption is it should fail because the change from dev wouldn't be applied automatically. This would cause a rollback. Based on this assumption, how does one automate "accepting" the scoped app as source of truth and just override what is in Test? If that isn't possible, what would be a successful workflow?

@chiarng
Copy link
Member

chiarng commented Jun 9, 2021

once you publish to application repository, then install to another instance (Test or Prod), it doesn't live in sys_app table anymore, it lives in sys_store_app, and you can no longer "develop" on it. the app client instance will always only have a version of the app, not a version of the app + changes made on that instance.

@dorian-cask
Copy link

@chiarnglin - I agree that it can't be "developed" on, but what is stopping an admin from changing scope to the app in the picker and then making changes in the default update set of that app (Test or Prod)? Am I missing something?

@chiarng
Copy link
Member

chiarng commented Jun 9, 2021

@dorian-cask hmmm i don't think you can make changes to a sys_store_app that way in test or prod, but @bmeeder22 can you help with describing any limitations we've implemented to ensure that applications don't end up with "merge conflicts" in the above case?

@bmeeder22
Copy link

Hi @dorian-cask,

This is a great question and one that is asked quite frequently from people moving from update sets. The entire idea of applications inside of ServiceNow is that they act as the "source of truth". What that means is that upon application load we do not check for "collisions" as you see in update sets. The idea is that your application contains the complete subset of records contained in the application and that is what will load.

So going back to your scenario where Junior Dev changes a BR on test and then loads the application which has a different version of the BR, we won't throw any conflicts we will just override that Junior Devs version with the version in the app. If any developers want to make changes to records belonging to an application they should ALWAYS update the application itself and push via SC or publish via app repo to promote those changes to other envs.

Thanks,
Ben

PS claims

Claims are designed to help negotiate conflicts between apps since multiple apps could modify the same file in the same scope. In the example above of a developer manually/update set editing in a test instance you would still observe the above behavior.

@bmeeder22
Copy link

PS pt2: If you want more clarification around claim resolution between apps I would be happy to provide that and answer your questions.

@bmeeder22
Copy link

Finally to answer this question:

what is stopping an admin from changing scope to the app in the picker and then making changes in the default update set of that app (Test or Prod)?

Nothing is stopping them, that is why they have admin power and with great power comes great responsibility.

@dorian-cask
Copy link

PS pt2: If you want more clarification around claim resolution between apps I would be happy to provide that and answer your questions.

Hi @bmeeder22 - thanks for the clarity. Claim is interesting.

The only question I have is around claim resolution is between Global and "Global Bundle" (a global scoped app to hold all the customizations we may make instead of using "Legacy" Global). Are there going to be claim issues on upgrades if we go that path?

@dorian-cask
Copy link

Hi @dorian-cask,

So going back to your scenario where Junior Dev changes a BR on test and then loads the application which has a different version of the BR, we won't throw any conflicts we will just override that Junior Devs version with the version in the app. If any developers want to make changes to records belonging to an application they should ALWAYS update the application itself and push via SC or publish via app repo to promote those changes to other envs.

^ Perfect - I like source of truth changes.

@bmeeder22
Copy link

Glad you like it!

So your question appears to be around the difference between things in the global scope and things in "globally scoped apps". So in your mind you don't have to think that there is any difference in terms of the scope things in globally scoped apps ARE in the global scope. It's just that we can associate them together into a package or application and thus say they are the source of truth.

Basically, global apps are a way to allow customers who need to modify global files to switch from update sets to an application driven model, but since everything is still in the same scope from a system level we need the extra protections of claims to handle those conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants