Chapter 3 Standard Operations

3.1 Team PSD

Meet the members and partners of Team Participatory Systems Dynamics at https://mtl.how

3.2 Scientific Values

Team PSD Scientific Values guide additional Participatory and Open Science principles:

Participatory Research encourages us to co-create our scientific research. Therefore…
We share decisions, which requires a high level of documentation.
We seek greater equity among partners in how collaborate, which requires responsive pivots with new stakeholder inputs.
We strive use transparent and accessible processes and platforms, and develop transparent, accessible resources.

3.3 Guiding Principles

3.3.1 Open-Source, Transparent, Reproducible

We value an open-source, transparent & reproducible workflow.

Most of our work is public-facing with the exception of any items that include Protected Health Information (PHI) or Personally Identifiable Information (PII). All of our public materials are free to download and use. We want our insights and resources to benefit the larger community.
We use R, a free open-source coding language and a specific file naming convention to ensure all of our matierals are machine & human readable (all lower case, no spaces, with dates as yyyy_mm_dd i.e. teampsd_guiding_principles_2020_01_01).
Use email for any private discussions. Host any private files in password-protected locations or in folders behind the VA firewall. When in doubt, ask an HQ member or err on the side of caution.
Make sure your work and accompanying documentation allows other team members or scientists in the field to reproduce and understand your work without further questions.

3.3.2 High Visibility

Our work has high visibility.

Keep in mind we work under the federal government of the United States for the Veterans Health Administration (VHA), the largest integrated healthcare system in the country. Everything we produce is a reflection of the VHA.
We work with a wide range of nationally-distributed partners both internal and external to VA, including very important and high-level stakeholders. Double check the role and responsibility of people you are communicating with before you do.
With these partners, we work in a participatory learning manner and iterate based on feedback from the field to ensure our work is responsive to ongoing changes.

3.3.3 Team Time

Any time saved is team time.

Ask questions early and often to prevent escalating issues down the road. Refer to existing resources (cheatsheets, checklists, etc.) as well for clarification.
Double check all work before handing it off to the next team member to reduce rework.
Think through dependencies across the team and partners and prioritize work based on the most recent information you have.
Manage workflow asynchronously (via GitHub) and only schedule meetings when absolutely necessary.

3.3.4 Communication

Use effective communication (across all types of communication including emails, GitHub, Lucid, etc.).

Assume everyone you’re communicating with is smarter than you and cares more than you and is busier than you.
Use clear and concise language, with formatting (bold, underline, bullets, etc) to emphasize the main decisions or issues making sure to include the “Who” “What” and “When”.
Always include the full context and details necessary to make an informed decision (and make sure you are up to speed on the context and details before responding).
Use complete sentences as much as possible and write in the active voice for clarity
Often the team looks back to previous records of meetings, GitHub discussions, etc. to remember and track decisions that were already made or if they missed the meetings where an issue may have been discussed. As such, we always need to keep the most accurate record possible to reduce rework and provide clarity.
Use emojis or humor (as appropriate) to help maintain a positive and collegial vibe. Turn on your webcam as much as possible for the face-to-face interaction.

3.3.5 Active Listening

We use active listening skills to ensure understanding and accurate tracking.

We work daily with team members and partners that are experts in their respective fields, and it is easy to lose track of a complex discussion.
We’ve found that reflecting-back requests and decisions in your own words has been the best method to reduce miscommunication and to keep track of all of our decisions or issues accurately. If you ever need help while scribing, always “tap” someone else on the team for assistance.
Ask for clarification and slow down if necessary.

3.3.5.1 Active listening is a stance.

Taking the stance that misunderstanding is the norm and using skills appropriate to that reality.

General skills: Reflecting content for efficiency and interpersonal rapport (i.e., avoiding rework and frustration)

“When in doubt, check it out” - It’s the listener’s job to help the speaker understand what they are getting or what they missed.
Let the speaker know when you’re falling behind. Stop them before the conversation exceeds your understanding.
Use the same precise language/vocabulary, esp. Team PSD or MTL terms with specific meanings.

“Chunking” components of what someone said (e.g., listen for the “and” when someone is speaking)

Listen for the main points.
This is a key skill to keep up with complex ideas in the moment (You don’t want to get bogged down on point 1 and miss point 2).

Reflection of feeling - listen for the feeling words and reflect them exactly.

Using the exact feeling word you heard is the safest way to ensure the person feels heard.
Don’t reflect “frustration” when someone is expressing “disappointment” or “stress.”
You can use these for non-verbals too.
If you hear a feeling, it’s best to address it in your reflection.

Synthesize two person’s ideas - or just two ideas (or more)

e.g. “on the one hand, I hear Anthony saying…” and “on the other hand, I hear Stacey saying…”
Synthesis brings the ideas together. It is not just active listening both ideas as separate ideas.

Summarize - used when it’s time to wrap up, and move on, and a lot of ideas have been said.

This is not the time to bring up new points. If you do have something to add, say it first.
It’s best to end with a summary of the key points of consensus and the key take aways of what to do next.

After you reflect, add to what the person is saying (a key to improv is “yes, and…” - don’t reject ideas, just try to understand first and then build on the ideas).

Forward momentum comes from making sure you understand the person and add your own contribution.
Listening is about being present to understand; you do not have to agree with what a person is saying.
Before you even get to agreement/disagreement, make sure you have a shared understanding first.
KEY IDEA: Really attend to make sure you understand. Do NOT think about your response when you should be listening.

3.3.5.2 Things to look out for when listening:

Know thyself: You are the only one who knows whether it’s time to multi-task, focus, scribe/document as a form of listening, or whether typing would be a distraction.

Repetition cycle: If the speaker you’re listening to keeps repeating something, your first move should be to assume that you’re missing something and ask what it is.

Your own feelings: If you’re starting to feel frustrated, it’s a key sign that you should use active listening to get back on the same page.

3.4 Project Management Principles [flowmap needed]

NOTE: Team PSD Scientific Values guide how and why we synthesize the approaches below

Team PSD integrates Waterfall principles because:

We are a research team that cannot deviate in quality, scope, timeline or cost from our thoroughly developed and vetted research plan
Therefore, there are several well-defined scientific requirements where change cannot occur - unless the PI and co-I senior scientists advise an operational definition or implementation detail is within the scope of the federally funded and registered, scientifically peer-reviewed, and VA ORD and Stanford IRB reviewed procedures.

Team PSD integrates Agile principles because:

Our research team’s method is a an implementation of participatory system dynamics in a national production environment.
These methods historically have iterated in-person locally.
Our innovation is local priorities, local data and local insights at national scale, made possible using online platforms and virtual processes.

Team PSD integrates Waterfall and Agile principles because:

We are using a funded a priori research design (Waterfall) to study an implementation science (Agile) problem.
Producing generalizable knowledge requires implementation scientists to follow and report publicly progress completing a rigorous, replicable plan (Waterfall)
However, the context is not a tightly controlled laboratory, rather we are working with operational partners to implement our innovation the dynamic context of real world healthcare operations (Agile)

Team PSD integrates Lean principles because:

Team PSD processes are assessed for continuous improvement (kaizen/muda elimination)
Team PSD communication is designed to reduce rework (muda/waste)
Team PSD encourages minimum viable products to find and synthesize team and partner expertise (just-in-time)
Team PSD platforms are designed to increase visual production controls (kanban/signboard; andon alerts)
Note: VA uses Lean as a primary approach to healthcare quality improvement.

Team PSD integrates Scrum principles because:

We need to be able keep work produced by our transdisciplinary research team on the same page with our cross-functional team.
Therefore, we use sprints (epics/milestones)so that the team priorities can be aligned/re-aligned efficiently.
We also use workgroup leads (scrum masters), workgroups (owners) and workgroup meetings to benefit from the efficiency of divvying up/delegating, while also identifying dependencies and remove blocks.
We use GitHub/ZenHub and daily huddles to assign, scope, prioritize, manage and review our capacity, requirements, estimates - this includes Project Kanbans, Issue Cards, Pipelines & Reporting.

Team PSD Pain points we are working to better address:

Need &&tight collaboration without unnecessary overhead,** so culture of participation is maintained
Need rapid iterations that prototype, check early, and synthesize inputfrom key stakeholders as efficiently as possible, so we aren’t pulled in wasteful directions or adding unnecessary rework
Need a standard set of principles and processes, so our production platforms are stable at scale.

Continuous Collaborative Iteration Cycles (e.g., “DevOps”)

SOP > Workgroup Meetings > Monthly Priorities > Requirements gathering standardization for features
Tools for team members to understand cross-functional “stakes” efficiently
“Continuous integration” to avoid “merge conflicts” - Quotes are meant to clarify that we have this problem at a communication and conceptual level, not just a code level. We need the code level next…because…
Ideally…testing, deploying and documenting would be increasingly automated.
REALLY need continuous documentation and training.

3.5 Standard Operations

3.5.1 Policy

3.5.1.1 Scope

This policy applies to all TeamPSD members involved with research under grants, R21DA042198, R01DA046651, IIRI01HX002521 commonly known as Modeling to Learn (MTL). This policy is subordinate to new or existing Veterans Administration (VA) policy. Any issues identified that are contrary to VA policy or federal laws should be brought to the attention of the Principal Investigator, Lindsey Zimmerman, (lindsey.zimmerman@va.gov).

3.5.2 Purpose

This policy details the governing principles, definitions, responsibilities and procedures for managing cards, issues, & pull requests in the Kanban production management system for the issue_tracker, feature_tracker, document_tracker, and manuscript_tracker GitHub projects for the TeamPSD. Finally, the policy will describe how MTL and overarching research experimental design and reporting with be coordinated.

3.5.2.1 Responsibilities

Principal Investigator - Provides overall direction and guidance to Workgroup Leads. Coordinates research activities and prioritizes activities within the MTL program with the HQ workgroup
Co-facilitators - Gathers field information and provides feedback to Workgroup Leads regarding the performance of a product in the teaching/learning environment. Facilitate Modeling to Learn 12 Session Program with identified clinics for the R01 and IIR grants.
Co-investigators - Oversees the science and research dependencies across the project.
Workgroup Leads - Oversees the production and maintenance of their products in terms of quality, cost and delivery, responding to the needs of the projects as defined by co-investigators and co-facilitators
1. Checks the bug_tracker, feature_tracker, document_tracker, and manuscript_tracker (as related to the workgroups) daily to identify & assign interdependencies from the labels list used as a dependency check
2. Ensures that all cards are updated in the issue_tracker, feature_tracker, document_tracker, and manuscript_tracker (as relevant to the workgroup) in include a due date once they have been triaged with HQ & other workgroup leads.
3. Trains respective workgroups in changes to policy & procedures for Kanban workflow.
4. Estimates the time required to fulfill completion of bugs (issues) & features.
5. Attends weekly Monday 8am PST / 11am EST Workgroup Leads meeting or alerts the HQ Proxy (see below) in their stead, should they not be able to attend.
Proxy - An individual who is a member that supports a specific workgroup and can participate in the absence of the Workgroup Lead to represent the interests of a workgroup. A Proxy has the decision-making authority of the Workgroup Lead they represent. The Workgroup Lead must still provide detailed and concise documentation of questions and scope on bugs and features related to their workgroup in the Workgroup Leads meeting agenda.

3.5.3 Workgroups

The table below describes all of the TeamPSD workgroups including their Workgroup Lead, Meeting Time, and Role. For each meeting, it is the responsibility of the Workgroup Lead or HQ point person to:

Set the agenda
Check RSVP’s for attendees
Clean up meeting notes for clarity
Send the follow-up email
Publish the meeting to Basecamp
Save Lucid audio in the TeamPSD folder: Meeting Agendas & Recordings (and backup and save any mtl.how/live recordings as relevant)

Workgroup (Workgroup Lead) (Meeting Time)	Role
Facilitate/EES (Jane) (Mon 10:00-11:00a Pacific, Wed 9:30-11:30a Pacific, Fri 9:30-11:30a Pacific; 3rd Friday 9:00-11:00a Pacific)	Provides MTL program resources such as learner SEE and facilitator SAY scripts, checklists, guides, EES (Employee Education Services) brochures, and post-tests; and co-facilitates the MTL program. *Rita is the HQ point person for this workgroup.
Grants (Lindsey) (Tues 10:00a-11:00a Pacific)	Develops and maintains aims, strategies, and research plan for grants
Headquarters (Lindsey) (Daily Check-In 12:30-1:00pm Pacific,Thurs 1:00-2:00pm Pacific,4th Friday 12:30-4:00pm Pacific)	Manages oversight of all workgroups, identifying interdependencies and parallel workstreams. Provides guidance on prioritization.
Manuscripts, Publications, and Conferences (Lindsey) (Mon 9:00a-12:00p Pacific)(Tues 10:00a-11:00a Pacific)	Develops and maintains manuscripts, publication schedule, authorship agreement, and conference materials
Modeling (James & Tom)	Builds models of systems that support clinician experimentation.
Qualitative (David) (Tues 8:00a-9:00a Pacific)	Codes team qhfd inputs for qualitative analysis and development of future fidelity materials. *Jessilyn is the HQ point person for this workgroup.
Quantitative (Anthony) (Thurs 10:00a-11:00a Pacific)	Provides data and analysis of data that supports other workgroups and research.
Simulation User Interface (James)	Provides an accessible, web-based user interface for practitioners to experiment using simulation.
Workgroup Leads & Support Workgroups (Stacey) (Mon 8:00a-9:00a Pacific, Thursday 8:00a-9:00a Pacific)	Triage all issues that have entered into the issue-tracker triage section and identify workgroup milestones and action items for the month. Rearrange deadlines and interdependent timelines in response to emerging issues.
VAPOR (Jennifer) (TBD)	Provides veteran perspective and guidance on Modeling to Learn initiatives

3.5.4 Definitions

Bug - An existing feature that has been developed and is not working correctly.
Issue - An important topic or problem for discussion. Issues are documented in the GitHub Issues tab by selecting “New Issue.”
Deliverable - A product of a task or group of tasks.
Expedite - To move the priority of a bug or feature to the top of a queue. Expediting is poison to a production system and should be used only in exigent circumstances.
Feature - A characteristic, attribute or topic that requires work breakdown, research, design, development and testing. Features can be tagged as “fast_track” in order to expedite its design and development. Features are documented in the GitHub Issues tab by selecting “New Feature”
Kanban - A board that contains issue cards and is used to communicate the status and priority of bugs, features, documentation, & manuscripts as they move through the production process. There are four Kanban to manage production in the MTL program: bug_tracker, feature_tracker, document_tracker, and manuscript_tracker.
Task - A cognitive or kinetic behavior that consumes time. A task or group of tasks are necessary to create an outcome or deliverable.
Labels - A tag in GitHub that is affixed by an originator or workgroup as a means to identify the task holders for a specific issue. Workgroup leads can filter by labels in the tracker to sort for their specific workgroup or other workgroups’ Kanban cards in the search bar found in the upper right-hand corner of each tracker (below). To use the filter function, use the code “label:labelname” i.e. “label:facilitate” or “label:sim_ui”. (You can use this function with other sort options as well i.e. author:lzim, etc.). In most cases, an issue should only have 1 label at any given time.

3.5.5 GitHub Labels Table

Labels shall be assigned a color, expressed in lowercase and use an underscore in lieu of a space. Below is the list of labels and their purpose:

Label	Purpose
document	expresses a dependency for at least 1 of 5 levels (describe, detail, document, disseminate, depend) of documentation to be tracked on the document_tracker. The point person for each level of documentation will be responsible for checking the issue & feature tracker for dependencies (describe: Jane & Debbie; detail: Tom & Lindsey; document: Stacey & HQ; disseminate: Lindsey, Anthony; depend: Lindsey, Jane, & Jessilyn)
facilitate	alert all of the facilitate workgroup to an issue that may affect facilitation (Lindsey, Jane, Debbie, Tom, David, Claire, Gayle, John, Matt, Jay, Theresa, Marcia)
hq	alert to HQ workgroup (Lindsey, Stacey, Rita, Jessilyn, Jennifer) to track
manuscript	expresses a dependency on manuscripts to be tracked on the manuscript_tracker
model	alert to Tom & James of a dependency on the model workgroup
pi	alert to PI/Lindsey that an action or decision is necessary
qual	alert to David, Kathryn, Swap, Lindsey, Jessilyn, & Stacey to of dependency on the qual workgroup
quant	alert to Anthony & Ash of dependency on the quant workgroup
sim_ui	alert to sim_ui workgroup lead of an issue or dependency that affects the sim_ui. The workgroup lead will evaluate sim_ui impacts and collaborate with other workgroup leads to determine an adequate resolution.
urgent	indicates a short amount of time is available for resolution and needs to be prioritized by workgroup
ees	description to come
vapor	description to come

3.5.6 Kanban Management System

3.5.6.1 bug_tracker

The issue_tracker is divided into 6 columns described below. The purpose of the issue_tracker is to provide triage and track the disposition of issues that require action by one or more workgroups. Issues labeled as “bugs” will be tracked here.

needs_triage - This column is the main intake for all issues assigned to the issue_tracker. All issues requiring a disposition will land here. When an issue lands in this section, any team lead may review it and alert other leads as to the action required.
validated_actions (ranked) - This column contains all issues that have received a “bug” disposition by a Workgroup Lead(s) and has been provided sufficient details to fix the problem. This is a rank-ordered list based on due dates indicated in the title. The rank-order of this section can be changed at any time. If an issue is determined to be a “feature,” it will be placed in the feature_kanban (see feature_kanban below).
under_development - Bugs that are currently under development are listed in this column. This section may not be reprioritized, and contents are addressed first-in-first-out by respective workgroups.
testing - Bugs that are currently being tested are listed in this column. Some issues may skip this step and go directly from under_development to done.
done - This column contains completed issues. Responsible Workgroup Leads shall communicate completion of the action to the originator in the issue thread. The originator shall review the action, indicate their satisfaction/dissatisfaction. Once the originator is satisfied, the originator should close the issue and it will automatically go in the closed column.
closed - This column contains issues closed by the originator from the done column. Any issue can be reopened as necessary.

3.5.6.2 feature_tracker

The feature_tracker is divided into 9 columns described below. The purpose of the feature_tracker is to report and maintain information regarding the analysis of dependencies and work-content, and track the progress of issues that require development. Issues labeled as “features” will be tracked here.

work_breakdown - Validated feature requirements that have received a disposition are listed here. Issues in this column are analyzed by Workgroup Leads for dependencies, effort content and milestones they may support (see Appendix 1 - issue_template). Issues will progress from this section to either the operations_to_do (ranked) or the research_to_do (ranked) sections.
operations_to_do (ranked) - Features that require research, design and development of products in one or more workgroups are listed here in order of priority by due date indicated in the title. Issues here may be reprioritized at any time.
research_to_do (ranked) - Features that support research, evaluation and documentation efforts directly related to supporting grants or higher-headquarters reporting requirements are listed there in order of priority by due date indicated in the title. Issues here may be reprioritized at any time.
under_development - Operations or research features under development are listed here. This section may not be reprioritized, and contents are addressed first-in-first-out by respective workgroups.
functional_testing - Features that are currently being tested for basic functionality (i.e. does it work?) are listed here. Some issues may skip this step and go directly from “under_development” to “done.”
done - This column contains completed issues. Responsible Workgroup Leads shall communicate completion of the action to the originator in the issue thread. The originator shall review the action, indicate their satisfaction/dissatisfaction. Once the originator is satisfied, the originator should close the issue and it will automatically go in the closed column.
closed - This column contains issues closed by the originator from the done column. Any issue can be reopened as necessary.
future_release – This column contains unresolved feature ideas that would be great to include in a future MTL release, but currently is not a pressing need.

3.5.6.3 document_tracker

The document_tracker is divided into 5 columns described below. The purpose of the document_tracker is to track documentation dependencies at 5 levels for each of the 12 sessions of Modeling to Learn. There will be a card for each session of the 12 sessions in each of the 5 Kanban columns which will be closed & reopened as interdependencies get identified.

Each column also has a “meta” card which is used to indicate interdependencies that are relevant to all/most of the 12 sessions as well as policy & workflow decisions specific to that documentation column.

describe_learners – Documentation dependencies relevant to learners, including SEE guides
detail_facilitators - Documentation dependencies relevant to facilitators, including SAY guides & one-pagers
document_team - Documentation dependencies relevant to TeamPSD, including cheatsheets
dissemintate_scientists_va - Documentation dependencies relevant to co-investigators & larger scientific audience, including progress reports, code, grants, etc.
depend_products - Documentation dependencies relevant to other MTL products, including videos

3.5.6.4 manuscript_tracker

The manuscript_tracker is divided into 10 columns described below. The purpose of the manuscript_tracker is to track progress of manuscripts through 10 major stages until ready for publication.

DO NOT post any direct manuscript content to GitHub; all drafts and related materials will be posted on the relevant OSF project. There will be a card per manuscript that moves through the tracker.

01_osf_project – OSF project is created for this manuscript with all relevant materials posted, Zotero & GitHub integrations approved, cheatsheets linked, and relevant people added
02_authorship – Initial authorship weights and task division are identified
03_write_analyze – Manuscript is written and materials for analysis are produced.
04_edit – Manuscript goes under rounds of editing and revision between writing team.
05_approve_letter – Team working on manuscript gives final approval and drafts letter to editor
06_submit_under_review – Manuscript is submitted to relevant journals and is undergoing review
07_revise_and_respond – Manuscript feedback is received from journals and team working on manuscript revises it accordingly
08_resubmit – Manuscript is resubmitted to relevant journals
09_accept – Manuscript is accepted for publishing by journals
10_publish_publicize – Manuscript is published!