Teaching Case: Evaluation of Preschool for California’s
Children
9
(1) Leadership and Engagement (advocacy): Engage a broad coalition of organizations and
constituencies (education, business, early childhood, and the Latino community) to support
high-‐quality preschool in California. As part of this effort, Packard created Preschool
California, an advocacy hub, to coordinate the overall effort.
“Preschool needed to be more important in key constituencies than it was,” Salisbury said.
“Politically, it [didn’t have enough support]; this issue was not going to move. We needed to
identify to key constituents that were going to own it.”
(2) Research and Policy Development: Fund California-‐specific research that focused on: the
benefits of high-‐quality preschool programs for K-‐12; cost and financing; preschool teacher
training, compensation, and retention; and baseline data on California’s preschool system.
Grantees could then use this research to support policy development and advocacy.
“We were convinced that the research existed but it was not California specific enough to
persuade California audiences,” Salisbury said. “We needed a compelling research case.”
(3) Target Communities and System Building: Increase the supply and quality of preschool in
California. As part of this effort, Packard would seed local preschool systems in key,
politically important California communities that could show the difference that quality
preschool makes to children. These projects would not only serve children but would be a
way to engage business and political leaders who could see real-‐life examples of this work.
A Different Kind of Evaluation is Proposed
“Before the ink was dry” on Packard’s approval of the preschool subprogram, Salisbury met
Heather Weiss, the executive director of the Harvard Family Research Project (HFRP), which works
with foundations, nonprofits and policymakers to develop and evaluate strategies to promote the
well being of children, youth, families and their communities.
!"#$"%&'()*+*,-.#."/"-0*
!"#$%&'(#)*+#,"-./0%*-,+#
.%11"20#3-(#12"4"0*#5%3'&0+#
12*.,6""'#1"'&,+#
1"&"#%2'*+*345(26*
7"8"54)/"-0*
!"#43)*#06*#,3.*#7"2#5%3'&0+#
12*.,6""'#*813-.&"-#
9#%."0*:4//;-(<"&*+*
=6&0"/*>;(5$(-.*
!"#.%11"20#4"(*'#12*.,6""'#
12"9234.#&-#9*"1"'&/,3''+#
&41"203-0#,"44%-&/*.#
?3,
@
*3?!A
:B
*CA
@
7?
C
=*
D4#5#
:-&;*2.3''+<
3;3&'3$'*#
5%3'&0+#
12*.,6""'#
D1E@9FEGA@D*
=-,2*3.*(#12*.,6""'#
3>32*-*..#3-(#
3?*-/"-#
=-,2*3.*(#1"'&,+43)*2#
.%11"20#7"2#12*.,6""'#
@2*.,6""'#1"'&,&*.##3-(#
(*4"-.023/"-.#0630#
32*#.**-#3.#0*,6-&,3''+##
3-(#A.,3''+#;&3$'*#
3%4H5"/*A$"-#
345(<2&*J#84%#H5"#
345(2("&*3%4)4&"$#
?K9:?F,=*
3%"&2'445*L4%*:#5(L4%-(#M&*:'(5$%"-*=0%#0".6*
AF3E:9*
Teaching Case: Evaluation of Preschool for California’s
Children
10
Salisbury told Weiss about the new Packard strategy. Weiss told Salisbury that she needed to think
about how to evaluate the work.
Salisbury recalls, “When I got to know Heather, evaluation was frankly new to me. I had some
experience with evaluation in advocacy when I headed Children Now but I had not been in
philanthropy before…I had operated in a rough and tumble world where there were winners and
losers. We didn’t need evaluators to tell us whether a case won or lost. A traditional approach to
evaluation I thought would be a waste of money. It would tell us in five or ten years what you did.
Meanwhile, something has happened that affects your strategy now.”
Weiss told Salisbury about another approach to evaluation, which could give Packard “real time”
feedback on its strategy that the Foundation could use, in conjunction with other information, to
inform decisions at the strategy level and make course corrections if necessary. Called variously
“real-‐time evaluation” or “developmental evaluation,” these approaches share something in
common—an emphasis on promoting strategic learning.
Strategic learning is the use of data and insights from a variety of information-‐gathering
approaches—such
as evaluation, situation analysis, and systematic reflection—to inform decision
making about strategy. Strategic learning occurs when organizations or groups integrate data and
evaluative thinking into their work and then adapt their strategies in response to what they learn.
Strategic learning makes intelligence-‐gathering and evaluation a part of a strategy’s development
and implementation—embedding it so that it influences the process.
Evaluation focused on strategic learning is just one approach to evaluation and is not suited for
every program or every foundation, Weiss made clear. It is also an
approach that Weiss and
colleague Julia Coffman believed in, had already had some challenging experiences in testing out,
and wanted to try again—under the right circumstances.
From 1997 to 2002, the HFRP team had the opportunity to test this new approach when the W.K.
Kellogg Foundation asked them to evaluate its Devolution Initiative, which it funded to learn about
the implications of devolving major responsibilities for welfare reform and health care policy from
the federal level to the states, and to mobilize advocates to respond as the implications unfolded.
The evaluation had multiple components, which included providing timely, continuous feedback to
Kellogg about how the initiative was—or was not—working.
While Kellogg was an early pioneer in trying out the real-‐time evaluation approach, and Weiss and
Coffman and their team were able to test several of their ideas—and could point to some
successes—for a variety of reasons and circumstances the experience as a whole was difficult.
8
“As I think about [my initial conversation with Lois], I had just come off the experience with Kellogg,”
Weiss said. “We had learned a lot. This was a good opportunity for us to step back and ask, ‘what are
the conditions necessary for this to work?’”
8
See also: Sherwood, K.E. (2010). The W.K. Kellogg Foundation’s Devolution Initiative: An experiment in evaluating
strategy. In P.A. Patrizi, & M.Q. Patton (Eds.), Evaluating Strategy. New Directions for Evaluation, 128, 69-‐86.