Woman Warriors Newsletter

Reflections from the Tamarack Institutes “Evaluating Community Impacts” Workshop

Last week Megan, our graduate student from the University of Calgary doing her thesis on food security with the Women Warriors in Lloydminster (read her preliminary research findings here) and I attended Tamarack Institute’s “Evaluating Community Impact” workshop in Saskatoon, November 14th to 16th. Thanks to Rhett Sangster, Director, Reconciliation and Community Partnership, Office of the Treaty Commissioner for sharing this event via email with us and being a welcoming presence at this conference.

My purpose for attending was to learn how to better capture results from community health initiatives and to keep growing as an evaluator. Although I am taking a break from Women Warriors to finish my BA degree, I hope to return to the field of Indigenous health. Also, I have a deep interest in reconciliation and the workshop was using the reconciliation efforts, on behalf of the Saskatchewan Treaty Commissioner, as a case study. I will share what collective community impact is, and through my lens as a Metis mother and social innovator in Lloydminster, I will explore some of the barriers to implementing collective community impact in Indigenous communities.

Collective community impact is solving complex community issues through cross-sector coordination “to create mutually reinforcing community-wide strategies that yield big changes as opposed to hoping that the individual efforts of organization and services end up being more than the sum of their parts.” (Evaluating Community Impact Workbook, 2). This cross-sector alignment of government, non-profit, corporate and philanthropic interests must have five basic conditions for collective impact including a common agenda, continuous communication, shared measurement systems, mutually reinforcing activities, and a backbone organization. Together these organizations create a collective vision, implement a shared measurement framework, evaluate and capture outcomes – intended and unintended, and communicate their results.

The first way of understanding complex issues is something that most people can relate to – parenting. Complex community problems like health disparities, food security, homelessness or reconciliation, are like raising a child in that there are no best practices. While organizations and evaluators love their best practice guidelines because it provides security, we must be fluid with our strategies. As a mother of three girls with very different personalities I relate to this analogy. Tackling complex issues takes a participatory, experimental approach with perpetual planning. Social innovators, like parents are hunch testers that use statements like “What will it take to…?” or “What if…”

The adage of “parents know their children best” can be applied to complex community issues. Institutions like the government may formulate solutions and best practices based on their data, but do not have access to the unique variables in each community. We need agile approaches, especially in Indigenous communities where cultural diversity and region specific issues exist. In fact, Indigenous communities want to solve their own issues. In this CBC article, ‘These are our children’: Sexual abuse and suicide rate among Indigenous youth” Cynthia Wesley-Esquimaux, Lakehead University’s chair on Truth and Reconciliation states, “It’s not about incarceration — it’s about healing these communities and giving them the opportunity to do that work together. And that means putting the resources in the right places. It’s about getting at the root causes of a systemic problem.” This idea of addressing systemic problems is summarized in a quote from the workshop,
“Programmatic interventions help people beat the odds. Systemic interventions seek to change the odds.” Indigenous communities need systemic interventions, which are community driven.

Adequate funding and resources for capacity building in Indigenous organizations is the main barrier to Indigenous communities implementing their own collective community impact initiatives. First, Indigenous communities are missing basic infrastructure like clean drinking water, safe housing, and basic health care. On November 6, 2017 the Honourable Jane Philpott, Minister of Indigenous Services made a speech at the National Conference on Public-Private Partnerships drawing attention to these issues. She states, “Basic infrastructure needs that most Canadians take for granted are missing in far too many Indigenous communities. From reliable access to safe drinking water, to basic medical care, all-season roads, quality housing and schools, shelters, broadband connectivity, cultural and recreational spaces… What most of us consider essential for our quality of life is absent from the daily reality for thousands of Canadians.”

Furthermore, Minister Philpot recognizes in her speech that a new approach to solving complex issues in Indigenous communities is needed. She rallies the private sector, corporate businesses to partner with First Nations, and explains why we need a collective impact approach. She states, “Some of you will view a role for P3s in addressing the infrastructure gaps for Indigenous peoples as a matter of corporate social responsibility.  Others may view it from the perspective of enlightened self-interest. Either way, you’ve figured out that it’s smart to invest in Indigenous infrastructure, Indigenous communities and Indigenous people. First Nations, Inuit and Metis peoples are the youngest, fastest-growing segments of the Canadian population. Their economic and social achievements can drive Canada’s prosperous future. Last year the National Aboriginal Economic Development Board analyzed what Canada loses from the gaps in employment and economic outcomes between Indigenous peoples and the non-Indigenous population. They determined that closing that productivity gap would mean an estimated increase in GDP of $28 billion annually, about a 1.5% boost to Canada’s economy.”

A challenge for Indigenous communities utilizing collective community impact lies in the many levels of colonial systems that they must work with. In this CBC article, Indigenous care options and solutions for indigenous people, the levels of systems are demonstrated in the quote, “How much fixing can you do to a system which includes police, jail guards, lawyers, prosecutors, judges, social workers, probation and parole officers, child care workers and more cemented into a well-established industry?” Indigenous peoples are forced into colonial systems with colonial frameworks. The frameworks, methodologies and evaluation methods of collective community impact must allow for Indigenous ways of knowing – that means decolonizing and indigenizing tools and methods for interventions. (Meagan discusses decolonizing evaluative approaches below).

As mentioned earlier, one of the five basic conditions of collective community impact is a backbone organization. I assert that this organization must be healthy, and stable with strong leadership skills. Indigenous communities are at different levels of healing from the trauma of colonization and the legacy of residential schools. Not all communities may be able to shoulder the responsibility of collective community impact. For example, there are strong leaders like Osoyoos Indian Band, Chief Clarence Louie telling everyone to “‘Get a damn job’: Chief offers blunt remedy for what ails First Nations” and on the other side of the spectrum check out this post by Tammy Roberts, a Saskatchewan political blogger on “Saskatchewan’s Cote First Nation: What In the Actual F**ck Is Going On?

Moreover, how can Indigenous communities build strong leadership and healthy organizations when their children are caught in a continued cycle of poverty and trauma due to colonial policies and structures, underfunding, and institutional racism? This Vancouver Courier article, Disproportionate number of Aboriginal children in foster carestates “Most children in care are Aboriginal — 62.7 per cent — while only about nine per cent of all kids are Indigenous.” These kids in care turn into adults in prison. This CBC article, Gaps between Indigenous and non-Indigenous inmates growing, latest statistics show states “In federal penitentiaries, the Indigenous population has increased by 16.6 per cent over the past five years and almost 40 per cent since 2006. Indigenous men represent 25.2 per cent of all in-custody males, while Indigenous women represent 36.1 per cent of all females behind bars. According to Statistics Canada, five per cent of Canadians are Indigenous.”

The Indigenous communities that currently have the resources, healthy organizations and strong leadership to implement collective community impact, will find it a natural fit with their values of cooperation above competition. For Indigenous peoples, pre-colonization, cooperation was an inherent process embedded in their Indigenous governance structures. In the article, Indigenous decision making processes: what can we learn from traditional governance, George Erasmus (1975) states, “The government of the Dene before the Europeans was one of collective agreement. We did not have people, leaders sit by themselves somewhere and make decisions and come back and impose them on our people” (3).

In summary, collective community impact is the best method for Indigenous communities to bring systemic change. In order for interventions to work in Indigenous communities, the colonial frameworks and evaluation methods must be decolonized. Minister Philpott provided the economic “why” it is in Canada’s best interest to partner with Indigenous communities. I argued that some Indigenous communities are better suited than others to create these initiatives and that there is much work to be done on healing communities and providing the fundamentals of life like clean water, and safe, affordable housing.  Through multi-sector cooperation, we have the power to disrupt the cycle of poverty that results in Indigenous children being apprehended and being fed into an institutional life of foster care, and a high likelihood of prison. This disruption is the work of reconciliation and it is why all Canadians should be involved in implementing the Truth and Reconciliation Commission of Canada’s 94 Calls to Action. Reconciliation is the ultimate collective community impact initiative that has the power to transform our society.

Our workshop was held at the beautiful Delta Bessborough hotel, a historic castle located downtown Saskatoon. I’m with the facilitators of the workshop, Mark Cajab and Liz Weaver.
Rhett Sangster, Director, Reconciliation and Community Partnership, Office of the Treaty Commissioner.
My three girls after skating at the Servus Sports Centre. I feel blessed to be a parent AKA hunch tester and I believe it makes me a better person and social innovator.

Decolonizing Evaluation Approaches by Megan Sampson

When Shelley brought to my attention that Reconciliation Saskatchewan would be using their efforts at reconciliation in Saskatoon, Regina, and Lloydminster as a case study in this workshop, I decided that it would be beneficial to attend. As mentioned when I guest-authored the Women Warriors newsletter earlier this month, Indigenous women’s nutritional realities and the barriers they face in trying to manage their diets and health have direct implications when considering the TRC’s calls to action. The Tamarack Institute rightfully emphasized the complexity of the social issues which workshop attendees and the organizations they represented seek to ameliorate. I appreciated their view of successful social programs as “resilient” rather than the “sustainable”; this acknowledges that societal factors are always in flux and that one approach or solution will not remain efficient without frequent tweaks and corrections in response.

When Reconciliation Saskatchewan presented their case study to the workshop attendees, they asked us to consider a number of questions. Essentially, they were asking us to discuss the challenges of “measuring” reconciliation, and how to avoid colonial frameworks in this process. My particular table group was asked to consider Hamber and Kelly’s[i] definition of reconciliation, and to provide suggestions about how to go about evaluating cultural and attitudinal changes as they relate to reconciliatory efforts. While I was excited to find out that Reconciliation Saskatchewan is attempting to decolonize their evaluative approaches, it did occur to me that the majority the Tamarack Institute’s attendees have received training in colleges, universities, or similar institutions that favour settler/colonial hierarchies, theories, and pedagogies. While recent efforts are being made to decolonize these settings, much work has yet to be done. In a chapter dedicated to decolonizing evaluation practices, Maori scholar Alice I. Kawakami describes wariness towards “’experts’ from prominent Western institutions and academicians”[ii]. Like Vine Deloria Jr[iii]. and other influential Indigenous authors, she recognizes that these institutions encourage evaluation based on Western standards and ways of knowing. Therefore, while it was extremely exciting to be in a room full of passionate people discussing reconciliation, and while there certainly WERE Indigenous thinkers and professionals in attendance, I do hope that Reconciliation Saskatchewan continues this discussion outside of the workshop and continues to consider the input of those who may not be considered “experts”.

With those thoughts in mind, I would like to share some of the recommendations made by my group in this discussion.

  • Making evaluations longitudinal in recognition of the fact that reconciliation is a long-term process and cultural and attitudinal change does not occur overnight.
  • Prioritizing Indigenous language in the evaluation process—evaluating by Indigenous standards using Indigenous concepts.
  • Recognizing that reconciliation is not necessarily something with an “end point”.
  • Relatedly, recognizing that unlike traditional evaluative strategies based on a Western capitalist rhetoric, to the goal of this evaluation should not be to identify whether or not reconciliatory efforts are a “success” or “failure” (with that said, we do understand that some form of evaluation is necessary in order to create accountability).
  • Incorporating qualitative data and gathering testimonials over time about individuals’ quality of life and changes they have noticed.

Everyone at my table was somewhat daunted by the task at hand when asked to consider the case study set forth by Reconciliation Saskatchewan and provide our input. It was certainly humbling to realize the limitations of our “expertise”. All in all, I do believe that the case study and related activities were at the very least a great demonstration to attendees that reconciliation will require ongoing conversation and constant reflexivity. Those attempting to enact it should consistently be questioning their methods to ensure that they are not perpetuating the very colonial structures and forces reconciliation is aimed at eradicating.

Here are some interesting articles to consult if you’re interested in learning more about decolonizing evaluative strategies: Johnston-Goodstar, K. (2012). Decolonizing evaluation: The necessity of evaluation advisory groups in indigenous evaluation. New Directions for Evaluation, 136, 109-117. LaFrance, J., & Nichols, R. (2010). Reframing Evaluation: Defining an Indigenous Evaluation Framework. Canadian Journal of Program Evaluation, 3(2), 13-31.
[i] Hamber, B. & Kelly, G. (2004). A working definition of reconciliation: SEUBP & Democratic Dialogue. Published with PEACE II funding guidelines.
[ii] Kawakami, A.I., Aton, K., Cram, F., Lai, M.K., Porima, L. (2008). Improving the Practice of Evaluation through Indigenous Values and Methods: Decolonizing Evaluation Practice—Returning the Gaze from Hawai’i and Aotearoa. In N.L. Smith & P.R. Brandon (Eds), Fundamental Issues in Evaluation (pp. 219-231). Spring Street, NY: The Gailford Press.
[iii] Deloria Jr, V. (1969). Custer Died for Your Sins: An Indian Manifesto. New York, NY: Macmillan.