Microsoft Word Packard Teaching Case revised docx

TEACHING  GUIDELINES  AND  QUESTIONS  

These  questions  and  teaching  points  are  intended  to  be  suggestive  of  possible  discussion  questions  to  be  

used  during  the  teaching  case’s  facilitation.  This  list  is  neither  exhaustive  nor  prescriptive.  They  are  lead  

questions  that  require  probing,  elaboration,  and  discussion  during  the  teaching  case  process.  

 

Case  Teaching  Questions  

Evaluation  Points  to  Elicit  During  Questioning  

1.  Evaluation  Design:  Why  did  

evaluators  choose  a  strategic  

learning  approach  to  evaluation?  

What  were  the  factors  that  led  them  

to  conclude  that  this  was  the  best  

design  choice?  What  might  have  

been  other  alternatives?  What  

conditions  are  critical  for  a  strategic  

learning  approach  to  work?  What  

does  it  take  to  determine  a  client’s  

(i.e.,  foundation’s  or  nonprofit’s)  

readiness  and  capacity  for  a  strategic  

learning  evaluation?  

The  Packard  Foundation’s  preschool  grantmaking  strategy  included  

advocacy  and  had  a  policy  change  goal.  As  a  ten-­‐year  initiative,  it  was  clear  

at  the  start  that  the  strategy  would  evolve  in  response  to  changing  political  

and  economic  conditions  in  California.  The  evaluators  chose  a  strategic  

learning  approach  because  they  felt  it  would  be  of  great  use  to  the  

Foundation  given  these  emergent  and  changing  conditions.  They  also  felt  

that  the  Foundation’s  culture  and  overall  orientation  to  evaluation  

provided  a  promising  context  for  a  strategic  learning  approach.  The  case  

allows  for  a  discussion  about  the  conditions  under  which  less  traditional  

evaluation  approaches  like  strategic  learning  are  (or  are  not)  a  good  fit.  It  

allows  for  insights  about  the  conditions  that  are  essential  for  a  strategic  

learning  approach  to  succeed.  

2.  Users  and  Uses:  Can  an  evaluation  

focused  on  strategic  learning  and  

informing  the  decisions  of  a  specific  

group  of  individuals  be  useful  to  

multiple  audiences  at  once  (board,  

funder,  grantees)?  What  does  it  take  

to  design  and  manage  evaluations  

that  serve  the  simultaneous  purpose  

of  supporting  both  strategic  learning  

and  accountability  for  outcomes?  

Evaluations  often  have  multiple  audiences,  including  boards,  program  

officers,  and  grantees.  With  any  evaluation,  it  can  be  challenging  to  meet  

the  needs  and  intended  uses  of  all  audiences  simultaneously.  With  

strategic  learning,  where  the  focus  is  on  integrating  evaluative  information  

into  the  decision  making  process  for  a  specific  set  of  evaluation  users,  this  

can  be  a  particular  challenge.  Here,  the  evaluators  focused  on  the  Packard  

Foundation’s  program  team  as  their  primary  audience.  As  a  result,  the  

evaluation’s  findings  were  seen  as  less  useful  for  grantees.  This  choice  of  a  

primary  user  audience  also  created  challenges  when  it  came  time  to  report  

to  the  Trustees.  The  case  offers  an  opportunity  to  discuss  if  and  how  an  

evaluation  focused  on  strategic  learning  can  manage  the  differing  

information  needs  and  uses  of  different  audiences.  

3.  Evaluator  Role:  What  is  the  role  of  

the  evaluator  with  a  strategic  

learning  approach  to  evaluation?  

How  is  this  role  different  from  

traditional  evaluation?  What  does  

this  imply  for  evaluator  objectivity?  

What  should  appropriate  boundaries  

be  with  strategic  learning?  

With  a  strategic  learning  approach,  evaluators  are  embedded  and  use  a  

collaborative  and  participatory  evaluation  process.  This  approach  is  

different  from  traditional  evaluation  in  which  the  evaluator  remains  

deliberately  separate.  “Evaluators  become  part  of  a  team  whose  members  

collaborate  to  conceptualize,  design  and  test  new  approaches  in  a  long-­‐

term,  ongoing  process  of  continuous  improvement,  adaptation,  and  

intentional  change.  The  evaluator’s  primary  function  in  the  team  is  to  

elucidate  team  discussions  with  evaluative  questions,  data  and  logic,  and  to  

facilitate  data-­‐based  assessments  and  decision  making  in  the  unfolding  and  

developmental  processes  of  innovation.”

3

 This  “learning  partner”  role  helps  

evaluators  stay  on  top  of  potential  strategy  shifts  and  allows  them  to  

facilitate  reflection  and  feedback.  The  evaluators  here  were  clear  from  the  

start  that  they  wanted  to  help  Packard  succeed  in  its  goal  of  achieving  

universal  preschool  in  California.  They  were  integrated  and  part  of  the  

program  team.  This  case  allows  for  a  discussion  about  evaluator  role  

boundaries,  what  those  boundaries  are  traditionally,  and  whether  and  how  

they  should  be  interpreted  differently  with  a  strategic  learning  approach.  

                                                                                                                         

3

 Patton,  M.  Q.  (2006).  Evaluation  for  the  way  we  work.  The  Nonprofit  Quarterly,  28-­‐33.  

---