Changes from an agency job analysis exercise.

[wslack]

Edit: #162 (comment) has the organized task list; please ignore this comment!

- [ ] Create a general page with facilitation tips and best practices, to include
   - A checklist of supplies, printouts, and other tools to bring or have. Should include:
       - a room with a projector that enables live editing. Projector should be tested.
       - previous job analyses and sample competencies
       - a fan, if the room is small
       - stapled resumes
   - How to split the group up to accomplish subtasks, paying attention to strong and weak personalities.
   - Assigning roles in advance of the exercise - who is leading and driving, who is the scribe editing live on screen
- [ ] Add guidance about how to group tasks and then create competencies based on the task, intended to help facilitators
- [ ] Add guidance about how to two proficiencies - meets requirements, exceeds requirements.
- [ ] Update sample competencies to have proficiencies with "single word changes"
- [ ] Update sample competencies to be shorter
- [ ] Update sample competencies to have proficiencies with sections for exam-only
- [ ] Add alternative slides not about IT roles as samples
- [ ] Add slide Jenn made about purpose of exercise
- [ ] Add guidance about the purpose of resume review in the exercise
- [ ] Add guidance about practicing the questions
- [ ] Combine the agenda into one big list that agencies can break up into pieces. 
- [ ] Likewise, combine the day 1 and day 2 PPTs so that styling can be changed more easily and flexed to 3 days more easily.
- [ ] Add guidance for non-interview assessments being incorporated (for job analysis and the assessment strategy - strategy issue https://github.com/usds/SME-QA-Website/issues/166)```

[wslack]

cc @JennNoinaj - did I miss anything?

[wslack]

From another agency exercise:

• Only do 1-2 resumes at first to test competencies; five is too many
• Add interview practice guidance
• Add clarity around reducing from job tasks to competencies
• Day 3 powerpoint and agenda updates is 4 hours:
  • Fresh eyes review of competencies and proficiencies.
  • Evaluate if interview questions allow people to distinguish between proficiencies.
  • Good answers / best answers
  • Divide up into groups to ask questions of each other
  • Time the answers
  • Make sure questions are worded correctly.
  • Break up into pairs
  • What is exceeds and what isn't (give them space to decide that)

[wslack]

• Add competencies from other agencies, not their proficiencies.
• Align the PPT and the website agenda
• Creating a written assessment page for job analysis
• Powerpoint slide on the timeline idea
• Guidance about four vs two
  • For multiple grades, for example
• Add slide that SMEs cannot apply for the job (make it funny!)
• Edit the slide on time commitment to be clear that SME work is: 1) not full time; 2) not the selecting officials in the room
• Rework how to pull competencies from task groups
• Add photo of how to pull competencies from tasks
• in our draft agenda, have the confidentiality sheet signed right before they start creating proficiency levels since that is part of the confidential material
• power point, have the confidentiality sheet signed right before they start creating proficiency levels since that is part of the confidential material
• add slides showing output/use of the proficiencies
• Think more through ranking lists
• Feedback interview expectations in prep

[wslack]

Todos:

• Will to do the agenda update

[wslack]

Will: agenda
Jenn: ppt
Stephanie: samples

[wslack]

Combined tasks from the above comments:

PPT updates (top level tasks all owned by @JennNoinaj)

• Add guidance about how to group tasks and then create competencies based on the task, intended to help facilitators
  • @wslack to add content to website when on PPT
• Add alternative slides not about IT roles as samples
• Add slide Jenn made about purpose of exercise
• Add guidance about the purpose of resume review in the exercise
  • @wslack to add content to website
• Combine the day 1 and day 2 PPTs so that styling can be changed more easily and flexed to 3 days more easily.
• After agenda is updated, ensure PPT aligns to it (or push back on agenda updates)
• Create a slide explaining how rating resumes works (using the "timeline" image or something else - basically explaining why not to define a specific "does not meet" competancy
• Add slide that SMEs cannot apply for the job (make it funny!)
• Edit the slide on time commitment to be clear that SME work is: 1) not full time; 2) not the selecting officials in the room
• Rework slide on how to pull competencies from task groups
• Add photo of how to pull competencies from tasks
  • @wslack to add content to website
• have the confidentiality sheet signed right before they start creating proficiency levels since that is part of the confidential material
• Rename job analysis workshop to "job analysis exercise" or whenever possible, "job analysis." Workshop has a training connotation.
• Add the "setting the bar" slide https://github.com/usds/SME-QA/files/4277619/setting.the.bar.-.slide.pptx

Example updates (top level owned by @stephaniefaith)

• Add examples of two proficiencies - meets requirements, exceeds requirements
• Add guidance about using four vs two competancies
  • For multiple grades, for example
  • @wslack to add content to website
  • @JennNoinaj to update PPT slides
• Update sample competencies to have proficiencies with "single word changes" that better model best practices
• Update sample competencies to be shorter
• Update sample competencies to have proficiencies with sections for exam-only
• Add competencies from other agencies we've been working with, not their proficiencies.
• Write up ranking list guidance

Website updates (top level tasks all owned by @wslack)

• Create a general page with facilitation tips and best practices, to include
  • A checklist of supplies, printouts, and other tools to bring or have. Should include:
    • a room with a projector that enables live editing. Projector should be tested.
    • previous job analyses and sample competencies
    • a fan, if the room is small
    • stapled resumes
  • How to split the group up to accomplish subtasks, paying attention to strong and weak personalities.
  • Assigning roles in advance of the exercise - who is leading and driving, who is the scribe editing live on screen
• Combine the agenda into one big list that agencies can break up into pieces.
• Add "day 3" agenda updates:
  • Fresh eyes review of competencies and proficiencies.
  • Evaluate if interview questions allow people to distinguish between proficiencies.
  • Good answers / best answers
  • Divide up into groups to ask questions of each other
  • Time the answers
  • Make sure questions are worded correctly.
  • Break up into pairs
  • What is exceeds and what isn't (give them space to decide that)
• Only do 1-2 resumes at first to test competencies; five is too many
  • @JennNoinaj to add the guidance about this to PPT
• Add clarity around reducing from job tasks to competencies
  • @JennNoinaj to add the guidance about this to PPT
• Creating a written assessment page for job analysis
• in our draft agenda, have the confidentiality sheet signed right before they start creating proficiency levels since that is part of the confidential material
• add slides showing output/use of the proficiencies
• Rename job analysis workshop to "job analysis exercise" or whenever possible, "job analysis." Workshop has a training connotation.

[JennNoinaj]

Potential to include the following text in this presentation (from the HR CoP 3/24 meeting presented by someone from OPM):

• If the job does not reflect the position description, the employee may not be successful in the role.
• Job Analysis cannot be successfully completed just by staying in HR, typically HR, SMEs, Leadership, and Hiring Managers should be involved.

[wslack]

Add a note about making proficiencies look like bullets.

[stephaniefaith]

@JennNoinaj to make sure these are in the job analysis instructions

Use these examples to develop your own core competencies and proficiencies during a job analysis workshop. Common aspects are:
• Based on job tasks needed from day one on the job, create 4-5 required core competencies required for the position.
• Define at least two proficiency levels for “Meets” and “Exceeds”.
• A clear description of the possible proficiencies that applicants would demonstrate for the given competency.
• When complete, decide which proficiency level is required for competency at the grade being hired.
• Make sure your job analysis documentation shows 4-5 tasks listed with each required competency to justify why that competency is required to qualify.

[wslack]

Started the agenda update. It took 45 min of solid work to do about a 3rd of it, I estimate. Lots more to get the links all fixed.

[wslack]

Did a bunch of these, moved back to up next.

[wslack]

Content from the OMB workshops:

PPT updates

• Add a "start to finish" example into the ppt based on a past job analysis to go through at the beginning - tasks to competencies to proficiencies to assessments. These slides with ground the participant throughout the workshop, and should use the same language as the one-pager to do so.
• Create a slide showing the steps of the workshop (at each stage of the workshop, show a "you are here" slide marking the topic that folks.
• Include reference in the job analysis workshop to the kickoff presso (to be expanded to explain SMEQA in a mini-roadshow if there was no kickoff)
  • Can also be shared with folks who join the workshop late, so they can catch themselves up - will need significant speaker notes
  • Should include a compare/contrast with typical hiring processes so people understand we want to keep low-quality candidates off the cert, and that offices do fit interviews after)
• Changes req'd for Direct Hiring initiative
• Create a "before dot voting slide" about consolidating competency groups as much as possible before dot-voting.
• Add slide on what good and bad competencies look like
  • Show the impact of a long competency and how its hard to use
• Add slide on what good and bad proficiencies look like
  • Can also show them how to draft a long proficiency down to essentials
• Add slide to enable facilitators to paste an initial plan/idea for a schedule
• Add a slide for group to discuss their preferred working pattern - longer lunch vs longer breaks (15 min breaks and 30 min lunch, 10 min breaks and 40 min lunch, etc etc), starting/ending times.)
• Add slide before assessments to reiterate: we are assessing against the minimum bar, not comparing candidates, and that competencies are bare minimums
• Add "before the workshop day" slide with connectivity/sound test instructions for organizers, as well as a backup comms plan in case the zoom is lost.
• Add slide with note about the importance of being on video (for remote workshops)
• Add slide to take a group picture (either as a screenshot or in person)
• Add slide on multiple grades to be clear if resumes are being assessed against both, or one

Website updates

• Meet with 1-2 SMEs before the workshop to get a sense of SME awareness of hiring.
• Add content to website: in person, can get away with two days. Online, should be three days and give time for breaks given fatigue. Maximum schedule online should be 6 hours of "work" not counting breaks.
• Add a note that having an active HR specialist in the room also gives confidence.
• Add guidance about setting up workshops distinguishing between fully present people (with backups covering for them) and partially present people (who might have work or personal matters pulling them away
  • Workshops aren't effective without a core of at least 2-3 SMEs who are fully present. Partially present folks count as like half a SME.
• In remote workshops, facilitators should have a printout of the speaker notes to make sure they don't miss content, or have them pulled up with the presentation.
• ID workshop participants in advance who can screenshare and live edit in breakouts rooms instead of making someone volunteer ad hoc.
• Have time to test the zoom connection the week before the workshop, if remote.
• Add section on pushing groups to make more/faster progress.
• Add option to do written assessment before resume review to website

New Artifacts

• Set up facilitator template .doc with everyone's e-mails and attendance dates
• Create a workshop one-pagers for participants to get the week beforehand, which should be closely linked with the content in the "start to finish" example. This doc will prime participants to better track the examples and progress within the workshop.
  • Should be sent to participants before workshop.
• In guidance for SMEs attending a workshop, add a suggestion for them to set an OOO, designate a backup, and plan to do their meetings in early AM, late PM, or over a standard lunch hour.
• Add guidance that the most important time is the first morning of the workshop.
• Consider creating training materials "how to run a job analysis workshop"
• Create a "before the workshop" document/e-mail with joining instructions, length of workshop, break times, a link to smeqa.usds.gov, and a the "kickoff" overview PPT.
• Create a "speaker notes" PDF to be easily downloaded from the SMEQA site for workshops hosts.
• I think there's an issue for this, but updating the competency library.

[wslack]

@kbnammacher-usds I added content from the first workshop from this thread: https://usds.slack.com/archives/CQP4NKWQN/p1596747579090900

[stephaniefaith]

@kbnammacher-usds is this an issue you'd want to prioritize in the next few cycles before we forget? for now i made you the sole owner.

[kbnammacher-usds]

job-analysis-presentation - Sept 14 2020 DE template.pptx

Here's the most updated DE hiring template with the above changes (as checked off)

[wslack]

Pulling in from https://github.com/usds/SME-QA/issues/1477:

• We should move some content from the slides into a .doc working notebook that helps guide people doing write ups!
• Always tell people to assign a scribe
• In interview instructions, tell teams to create questions that relate to specific competencies. (one or more)
• Create guidance on how to create written prompts

[wslack]

• We are testing on competencies, not enthusiasm
• If someone comes through with these who isn't enthusiastic, would you hire them?
• Two categories per GS level