Challenges

Evaluating state professional learning initiatives comprehensively, supporting comprehensive evaluation of professional learning at the local level, and increasing the shared accountability for excellence in professional learning between state and local leaders are extremely challenging endeavors. [Here is a starter list for the section.]

• SEAs often do not have the technical expertise that is needed to design rigorous studies of the effects of professional learning on adult practice and student learning.
• SEAs often do not have the staff capacity needed to systematically monitor the implementation of large initiatives. Staff members in charge of professional learning may not have time to work on evaluation in addition to their other program work, and SEAs are often not organized to include staff solely dedicated to conducting evaluations.
• Complexity.
  • The most robust professional learning can be the hardest to evaluate. If it is delivered over a long period of time in many locations, and is embedded in day-to-day work at the district, school and teacher levels, it may be hard to separate out from other improvement activities – especially when it comes to linking the effectiveness to student outcomes.
  • All of the evaluation questions — whether about design, implementation, reach and participation, perceptions or outcomes — must all be examined with the dual focus of identifying overall trends and interrogating variation across time and location. A multiplicity of factors, from geography to funding to culture to history to local capacity and others, inform the ways in which actions are approached, carried out and received, and how those actions impact practice and outcomes. Evaluations with a dual focus on overall trends and local variation provide more sophisticated information to inform efforts that must always be based on uniform high standards while carried out in widely varying contexts. However, such complex and detailed work makes that work more difficult.
• Funding ― for example, for staff or for the contract services of expert evaluators.    

Reaching significant numbers of districts, schools or educators to make a difference. In states with smaller numbers of districts and schools, the SEA can more easily engage with significant numbers of practitioners; in states with larger numbers of districts and schools, engaging directly with significant numbers of practitioners is inherently a greater challenge. RESAs, institutions of higher education and other entities can serve as valuable partners to undertake the suggested key actions in order to support statewide learning, scale and spread of best practices.