Abstract
With increasing rates of child and family poverty combined with inadequate social support systems, many large urban areas in the United States are facing multiple issues of isolation and violence. St. Louis is one such city. The neighborhoods are increasingly isolated by income and race. Many of the youth are inadequately educated, perceive little hope for their future, and experience violence daily. Our traditional service systems are inadequate in their response, and our tradition evaluation systems are ill equipped for measuring the flexible, collaborative systems required. This paper will review the use of participatory action research in the evaluation of two collaborative, school based Family Support Centers designed to meet the needs of youth and families in very high risk neighborhoods.
The intertwining and interaction among social service delivery, negative demographic trends, and random policy shifts in high-risk urban communities has led to conditions causing an increase in the number and scope of the problems facing high-risk children, youth, and families. Programs throughout the country are attempting to address the needs of these children, youth, and families. Programs have pulled together resources (funding and personnel) for the provision of collaborative, community-based, interdisciplinary services in an attempt to meet their needs.
Just as traditional services and educational models have been unsuccessful in serving the needs of these youth, traditional positivist research methodology has been unsuccessful in helping us understand the components necessary for empowering high risk youth, families, and neighborhoods in positive change. The ethical dilemmas of practice and the inadequacy of positivist methods of inquiry leave us without the tools necessary for developing insights into the mechanisms for major change in high risk communities. Evaluating the individual components in isolation from the community does not provide the information necessary to understand the change process and successful components. Qualitative naturalistic inquiry provides the framework necessary for the contextualized study of multiple perspectives.
Our efforts, as an interdisciplinary team, to evaluate a collaborative providing services for at-risk families at school sites illustrate both the challenge and the richness of multiple perspectives. Our research team brings together the perspectives of education, social work, and public policy. Two schools, organized as local community centers, provide the sites. The principals, community education coordinators, faculty, students, staff, parents, and agency personnel contribute multiple perspectives. The richness of the data arising from the interdisciplinary qualitative evaluation helps clarify multiple levels of complexity. This paper describes the development and early evaluation of two interdisciplinary Family Support Centers. The evaluation plan and initial analysis are reviewed with a discussion of the implications.
Background
There is a growing awareness of the social problems impacting children and families in the United States (U.S.) pushing professionals across disciplines to share expertise in meeting the increasing needs. (Franklin & Streeter, 1995). This shift in both need and awareness pushes disciplines to define the needs of the children, youth, families, and neighborhoods from a more holistic perspective. The ecological perspective, which builds off systems theory, recognizes the impact of the environment on the individual and family (Germain & Gitterman, 1995) laying the framework for multi-level assessment and intervention.
As groups are beginning to examine the needs of high risk children and families, school linked, collaborative services are recognized as critical (Caplan & Gal, 1996; Franklin & Streeter; 1995; Lawson & Hooper-Briar, 1994). Franklin and Street discuss the importance of educators and social workers pulling together in meeting the needs of our children and youth. Lawson and Hooper-Briar, in their report on expanding partnerships, point to the significance of the transition. As we place social and human service systems in school settings, we are challenging traditional definitions of the disciplines. The need is to rethink how we put the systems together and how those systems can collaborate and integrate services to not only make the delivery of services more efficient but also more effective and responsive.
While dramatic changes which impact our schools, agencies and universities are beginning to occur, many of the collaborations are evolving in isolation trying to learn without models (Lawson & Hooper-Briar, 1994). Lawson and Hooper-Briar challenge universities to increase their involvement in collaboration and service integration through community partnerships. Universities have a responsibility to share their expertise, educate students for collaborative practice, and generate new knowledge through research and scholarship.
Potential Contribution
As programs throughout the country are attempting to address the needs of children, youth, and families from a very different perspective by pulling together resources (funding and personnel) for the provision of collaborative, community-based, interdisciplinary services for very high-risk children, youth, and families, they are also struggling to find evaluative methodology which fit the context. Narrative and other qualitative data are necessary in expanding our understanding of the components promoting development and precipitating change.
The qualitative approach to research through listening produces many insights to understanding by using a different set of methods required in the human and social sciences (Patton, 1990). The qualitative paradigm allows for an understanding of themes, patterns, and meanings within context (Goldstein, 1994), recognizing the importance of "many kinds of knowers: researchers, practitioners, clients" (Hartman, 1994, p. 461). It adds depth and detail to areas of inquiry for which quantitative methods have no precise measurement allowing for the generation of new insights (Patton).
Qualitative, naturalistic inquiry allows for the study of the specific contexts within which each program develops. Programs evolve differently within different contexts. Understanding those differences along with the similarities allows for an understanding of which components are transferable across contexts. Community based, collaborative programs must be studied in their natural setting. The use of qualitative naturalistic inquiry employing the methods of participatory action research as described in this study provides a model for evaluating and impacting the change process. This model provides a structure for hearing multiple voices in the ongoing process of development and feedback.
The Context: Children and the City
Nationally, poverty among our children is increasing rapidly. In 1989, the national statistic for children living in poverty was 19 percent. By 1993, it rose to 22.7% percent (Children's Defense Fund [CDF], 1995). The rate of increase is highest in cities, particularly among families of color. Across all ethnic groups, the highest rates of poverty are among single-parent families (Schmitz, 1995). The Children's Defense Fund (Sherman, 1994) reminds us that children in poverty are significantly handicapped educationally at an enormous expense to society as a whole. Poverty and violence increase the isolation of the community and the children who then lack exposure to diversity among people and income groups as well as experiences in life (Garbarino, Dubrow, Kostelny, & Pardo, 1992; Halpern, 1995). Many of these children never leave their immediate neighborhoods and, as a result, end up without a sense of future options and goals.
Youth in the City of Saint Louis face a particularly high risk. In an annual rating of ten outcome measures for all 115 Missouri counties, the City of St. Louis ranked last. They have the highest rate of child poverty (39.3%), highest school dropout rate (23.2%), and highest rate of low birthweight infants (11.8%) (Citizens for Missouri's Children, 1995a). Crime and violence have also been increasing nationally among our youth in a never-ending cycle. The number of children killed by gunfire as well as the number of youth arrested for murder has increased dramatically in recent years (CDF, 1995). In Missouri, homicide was the leading cause of death for 15 to 19 year old males in 1993 (Citizen's for Missouri's Children, 1995b). As children experience the trauma of ongoing violence in their homes and neighborhoods, they begin to suffer from post-traumatic stress leading to a decreased sense of empathy, so that the cycle then increases as it repeats itself (Garbarino et al., 1992).
Saint Louis Public Schools struggle to design and implement programs which will help the success rate of children and youth in schools by supporting the healthy development of families and communities. As a part of this effort, Saint Louis Public Schools instituted the development of Community Education Centers in 15 city schools during the 1994/95 academic year. The Community Education Centers (CECs) are designed to decrease the increasing isolation in these neighborhoods, integrate the schools into the community, maximize the availability of the resources in the neighborhood however limited, and provide an environment for attracting new resources. As a piece of that effort, the CECs have facilitated increased involvement of individuals and organizations invested in city neighborhoods. A number of organizations spanning the private nonprofit service community as well as business and industry have come together to support the development of strength based family support centers within the CECs. Saint Louis University has been one of the organizations investing heavily.
The Centers
St. Louis City advocates, business leaders, private nonprofit organizations, governmental agencies, public school staff, university representatives, and community members including parents and youth are developing collaborative family support programs in neighborhood sites. The goal is to produce sustainable family support centers based on the needs expressed by the neighborhood. Services are developed through community commitment so that they will remain affordable and accessible on an ongoing basis. The process of program development and grant writing has involved members of the community in an effort to design programs that meet locally identified needs.
The Family Support Centers involve a plan for coordinated service delivery with linkage to community resources. The Support Centers include a) family focused/family friendly services, b) strength based programming, c) culturally appropriate services, d) interagency collaboration, and e) community driven development. A two-pronged strategy is employed to improve family functioning, school achievement, community development, and school family connections. At each site, a Family Counselor and a Neighborhood Outreach Worker is hired to: (a) link families with the school, the Community Education Center, developing services, and existing social services; (b) provide counseling and services to improve parenting/family interaction skills and family management; (c) coordinate the provision of family support services; and (d) assess family needs. Community teams are working to expand the range of supplemental services available at each site to support classroom success of children and youth.
Two neighborhoods with multiple risk factors for families with children were chosen for the first centers. Both neighborhoods have high poverty and unemployment rates as well as high rates of crime. The neighborhoods also have high rates of youth dropping out of school and high levels of unemployment among youth. The population in these neighborhoods is dropping, and few housing units are owner occupied. One of the neighborhoods was chosen because of extreme racial isolation (93% African-American; 6.7% Euro-American) and the other chosen because the ethnic composition is more diverse (43.2% African-American, 54.5% Euro-American, 1.5% Asian, and 1.2% other). Ninety-eight percent of the students in both areas qualify for the free school lunch program.
The components at the two centers started in January 1995 are based on assessment of neighborhood residents and school personnel. The sites identified the following areas as primary program needs: a) health/mental health, b) learning support, c) economic well-being/job skills, d) self-improvement, e) weekend programming, and f) basic service (housing, child care, etc.) linkage. The initial components of the centers include parent outreach, home based family counseling, parent training, training of school personnel, youth tutoring, and skill development for children, youth, and adults. Programming is being developed that involves adult education/employment and primary health care.
The Evaluation Plan
Faculty from the School of Social Service and the Departments of Education and Public Policy at Saint Louis University are combining perspectives and skills to design and implement an interdisciplinary evaluation of the Family Support Centers. Our evaluation of the developing centers utilizes action research methodology by engaging a steering group of community, school, and agency participants as partners in planning and implementing the evaluation. The very emphasis on the "developing" centers underscores a fundamental assumption of qualitative research that social life is an ongoing process of construction. Regular interviews and focus groups with teachers, staff, parents, and students contribute to describing the multiple perspectives constructing the Family Support Centers. One member of the research team has participated since the inception of the program as a key member of the project team. This adds a participant-observer approach to the methodology increasing the evaluative perspectives.
As an interdisciplinary team, our own disciplines and research interests add still other perspectives. These provide different lenses from which to view the unfolding program. They also provide the opportunity for exchanging views with fresh ways of seeing the unfolding process of development. Engaging the metaphor of a kaleidoscope to discuss this interdisciplinary evaluation of the collaborative focuses attention on the complexity of the program as well as the evaluation itself. The metaphor raises awareness of multiple frames and shades of meaning. The frames are created by differing stakeholder values and perspectives inviting different questions. The metaphor appropriately highlights the value-laden, political context of social policy development and program implementation. This approach recognizes and compares fundamental yet tacit belief systems involved in both the program and the evaluation.
The evaluation involves the assessment of the multiple components of the programs and their impact on the individual children, youth, families, school/agency, and neighborhood. The communities want to understand which components (a) assist youth in a successful school career, (b) help strengthen the parent-child relationship as well as the parent-school relationship, (c) provide the basis for a positive teacher-parent relationship, and (d) ultimately improve the overall safety and health of the neighborhood. The evaluation focuses on understanding the components that are the most helpful in facilitating change.
Methodology
The methods of action research are employed involving the people in the programs and community in evaluating both outcomes and process. Action research incorporates methods that allow an interweaving of the inquiry into the change process with ongoing feedback impacting the quality of program development (Patton, 1990). The lead investigator participates on the administrative team participating in program development and monitoring while providing ongoing evaluative feedback into the development and data gathering process. Other members of the evaluative team were nonparticipant observers and data gatherers/analysts. Focus groups and semi-structured interviews add additional data from multiple voices. The depth and rigorousness of methods allow for multiple layers of understanding.
Planning and Collecting Data
During the first quarter of program development, baseline data was gathered on the child/youth, program, family, and neighborhood levels through focus groups, semi-structured interviews, and collection of data on the youth and families provided with intensive services. The evaluation consists of two sets of components: (a) change in child and family systems and (b) the process and impact of collaboration and program development. This analysis focuses on the assessment of teacher, staff, and administrative perception of the children and families, program, community, and neighborhood.
Community data involves collecting information on the process of the development of the collaboration itself. This data is being gathered through observation of collaborative meetings and interactions, semi-structured interviews with key informants, and focus groups. Participation in focus groups flows from constituency membership. Each focus group is composed of individuals from six separate populations: program staff, teachers, child and youth program participants, families with a program participant member, other youth and families using the CEC, and community members. The focus groups utilize a semi-structured protocol developed over the course of the study. These data serve as the basis for interpreting the subjective experience of the program and its participants while documenting lessons learned. Semi-structured interviews were conducted with identified program staff and collaborative team members. Later a purposive sample of youth and families will be identified by program staff for interviews on their perception of shifts in program availability.
Data Analysis
Data is being analyzed for themes which will be discussed with participants for further analysis and interpretation. Early themes were identified and then coded using a combination of the methods described by Bogdan & Biklen (1992), Miles & Huberman (1984), and Strauss & Corbin (1990). The results and findings in this paper reflect analysis of observational data, semi-structured interviews with the administrative team, and focus groups involving teachers, staff, and administrators. Focus groups with children and families are currently underway.
A Kaleidoscope: Results and Findings
The data collected are like the various pieces of colored glass that create differing patterns when framed differently. The kaleidoscope invites thinking beyond the linear dimension and allows multiple possibilities. Qualitative data from interviews, focus groups, minutes, reports, and the grant itself reveal the frames the participants themselves use to make sense of the program. Feedback from focus groups, observation, and interviews capture stakeholder perceptions of the program at a point in time while also providing useful information to managers as they work to shape the program. Such groups can also engage participants in developing the evaluation.
In the interpretive frame we can examine how the various stakeholders perceive and experience the program and capture how program processes emerge. Shifting frames and adding another dimension, the evaluator pays attention to the program's operation, providing useful information to program managers. In still another shift, the evaluator engages stakeholders as participants in critical action research.
Multiple Frames
Perceptions come together from multiple sources--teachers, staff, administrators, volunteers, evaluators, and other stakeholders. Interviews, focus groups, and participant observation provide the basis for our data. These perceptions form multiple frames. Three major areas surfaced repeatedly -- role confusion, access to information, funding, and attitudes/perceptions.
What's My Role?
Evaluators, school staff, and agency staff could be heard expressing confusion about roles reflecting the coming together of CEC, agencies, University, and neighborhood into the newness of a community based Family Support Center. Staff, administrators, and evaluators were frequently heard to repeat comments such as:
"I wasn't sure how to do that."
"I wasn't sure what I would do there until . . ."
"You're too involved to evaluate"
The role of participant observer in action research can be particularly confusing. The roles of evaluator, coordinator, and educator can blend together leaving one wondering if they are providing empowerment, education, or evaluation.
The Grant: Financial and Perceptual Seeding
Teachers knew that there was $100,000 to be given to the schools for services, and they agreed with the need for services for their students. Many were upset, however, that they had no input in deciding if they wanted to be involved in the program or in hiring the staff. There seemed to be confusion about the history of the grant. They didn't understand how the schools were chosen or who the providers were to be. One teacher said "I'm not sure of what it is. I'm suppose to refer some kids for counseling."
A concern of key stakeholders interviewed is the reimbursement of the funds. Some of the people interviewed are concerned about rumors that people had been hired but not yet paid. There was agreement that this should change. Several had heard that there was improvement in the funding. They did, however, understand the grant is designed to decrease risk factors at the two schools through interventions involving counseling, tutors, and parent involvement.
Many Voices, Multiple Perceptions
The Children
There was much diversity in the descriptions of the children by administrators, teacher, and staff. One person talked of the wide range of kids. They described the children as predominantly black, from diverse socio-economic status with some from stable homes.
They were described as super, eager, and creative. They were said to have high aspirations. One teacher remarked, "They adjust well to change. They have many different teachers each year." On the other hand, they were described as unmotivated and challenging with no sense of "I can do." A lot of concern was expressed. "They come to school sick with lice, measles." "They don't care." People talked of the children's behavior as being influenced by crime.
The concerns run deep. The "kids have low skills; I just need to go back and start all over again." "Our kids are so far behind. We would need rooms of four and five to bring them up to level." Informants talked about the children's inappropriate behavior, deep emotional problems, unruly behavior, and short attention span.
Some talked about the children coming from homes and neighborhoods where the values were different from the schools. They also pointed out conditions faced by the children -- latchkey kids, poor, transient, high free and reduced lunch participation. One person talked about the children wanting to be held. While many of the children need this, the teachers and staff do not feel they have time for all the hugs. There seemed to be frustration that the children -- don't talk in complete sentences, aren't readers, lack self esteem, don't know how to play, talk too much, don't bring supplies, and are immature. They talked about the children who will do their homework and not turn it in.
Some of the comments cross over between frustrations with the parents and concerns about the children. "I think some are on drugs and abuse alcohol. Some are crack babies." "They are so easy to get angry." "I can't take them on field trips because I can't control them."
The Parents
During focus groups with teachers, administrators, and staff, the parents were viewed negatively by some. They talked about problematic relationships between the parents and the school. They have "uncooperative attitudes . . . I don't know what to do." "They take stands against the school." "There is much animosity. Most parents are uncooperative." "Parents encourage their children to dropout and get GEDs in order to help them take care of siblings." "They don't show up for conferences or Individual Education Plans." A music teacher said, "In 37 years, I have never had a parent come to see me."
Others talked about more general concerns. "They spend little time with their children." The "kids seem to be raising themselves. They wash, cook, watch siblings." "Many of the parents cannot help when asked. . .I have a problem getting current telephone numbers." "They send them to school to get them out of the house."
Some of the comments were more descriptive and/or positive but frequently with a negative twist. "They [the parents] come to school immaculately dressed and groomed." "Most keep shots up to date." "Young parents. . " "Most parents are not working but still are not involved with their children."
The Schools and Positive Signs
One of the schools was described as the 6th richest in the city with some home ownership. Job satisfaction was discussed. "It is a good place to work. Administration and staff are close, warm, cooperative, and supportive." "There are some stressful days but here it is pretty relaxed. I really enjoy working here."
Frustrations were again expressed. "I took six or seven hours to write IEPs for each of my students. Most parents do not come for the conferences." Many teachers are frustrated with expectations to raise test scores without parents help. Parents will sometimes agree to go on field trips and not show up.
Some talked about creative solutions for students and parent involvement. One teacher talked about pairing students as helpers/tutors. "I have other students call their peers to remind them of homework and to invite them to school." There was talk of the children learning to work together "I see some growth. Every now and then, if one student bumps into another one, they might say excuse me. They are courteous." When a child falls out of a chair, the entire class says, "Are you all right?" or a kid will say, "I'm still in book one" and the class will not make fun of him. They encourage each other. This is an improvement from so much fighting, insensitivity, and hostility.
A male teacher has learned how important he is as a role model for many fatherless boys and girls. "So many children cling to me... I find great commitment and interest in helping children."
The Development of Programs
Family counseling and parent outreach workers from a local nonprofit agency are in place in each school, working with teachers and school administrators in setting up a system responsive to the neighborhood. The CEC coordinator at each school works with the principal to develop social, recreational, support, and training programs that meet the needs of the children, youth, and families in the neighborhood. During the summer remedial education programs, community trips, family camping, youth training programs, video discussion, and recreation/social skill programs were developed and funded. This fall, the schools continued community trips, video discussion, and recreation/social skill programs. Tutoring programs using a neighborhood coordinator and University students were implemented. Teen dancing, holiday dinners and parties, and Kwanza events were funded. Adult education and training programs were started. Planning and discussion continue.
The major problems observed by the evaluation team have been territorial, administrative, and bureaucratic. Some members of the team have felt threatened and attempted to grab control. Ongoing discussions involving members of the evaluation team facilitated shifts that have provided continued health in developing programs meeting the needs of the community rather than the professionals. Accessing state and federal funds channeled through the school bureaucracy has been the major impediment requiring a significant time commitment by the principals, agency administrators, CEC coordinators, and participant observer. The vast majority of the funds for the first year were not released until the last possible day. To address these difficulties the supervision of the service integration coordinator shifted to the administrative team, and monthly coordination meetings were established.
Discussion and Conclusion
The use of participatory action research is providing a background for successful program development and meaningful evaluation of context and process. The involvement of the lead investigator on the administrative team with ongoing evaluative feedback into program development has resulted in shifts in program and process designed to better meet the needs of the children, youth, families, and neighborhood. The interviews and focus groups provide valuable insight while investing participants in the development and implementation of the Family Support Centers. Youth and family interviews currently underway will further increase the depth of knowledge.
Although our evaluation is only in its beginning stages, we have captured a sense of the basic environment and the coming together of multiple systems and individuals to build creative programs for children, youth, and families. The teachers express frustration. They also discuss commitment to the school and concern for the children and families. Administrators are working creatively in reconfiguring programs, services, and spaces. Ongoing feedback has allowed quick response to bureaucratic difficulties, system barriers, and attitudinal limitations. The Family Support Centers exhibit flexibility in shifting resources into programs that meet the immediate need while maintaining an ongoing thread of counseling and outreach.
Reference List
Bogdan, R. C. & Biklen, S. K. (1992). Qualitative research for education: An introduction to theory and methods. Boston: Allyn and Bacon.
Caplan, J. & Gal, A. (1996). The school: The crossroads for collaboration. Journal of Children and Poverty, 2 (1), 55-91.
Children's Defense Fund. (1995). The state of America's children yearbook. Washington, DC: Author.
Citizens for Missouri's Children. (1995a). Kids count Missouri. St. Louis, MO: Author.
Citizens for Missouri's Children. (1995b). No safe place: A report on children and violence. St. Louis, MO: Author.
Franklin, C. & Streeter, C. L. (1995). School reform: Linking public schools with human services. Social Work, 40, 773-782.
Garbarino, J., Dubrow, N., Kostelny, K., & Pardo, C. (1992). Children in danger: Coping with the consequences of community violence. San Francisco, CA: Jossey-Bass Publishers.
Germain, C. B. & Gitterman, A. (1995) Ecological Perspective. In Encyclopedia of Social Work (19th ed) (pp. 816-824). Washington, DC: NASW Press.
Goldstein, H. (1994). Ethnography, critical inquiry and social work practice. In E. Sherman & W. J. Reid (Eds.), Qualitative research in social work (pp. 42-51). New York: Columbia University Press.
Halpern, R. (1995). Children on the edge: An essay review. Social Service Review, 69, 131-150.
Hartman, A. (1994). Setting the theme: Many ways of knowing. In E. Sherman & W. J. Reid (Eds.), Qualitative research in social work (pp. 42-51). New York: Columbia University Press.
Lawson, H. A. & Hooper-Briar, K. (1994). Expanding partnerships: Involving colleges and universities in interprofessional collaboration and service integration. Oxford, OH: The Danforth Foundation and The Institute for Educational Renewal at Miami University.
Miles, M. B. & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage Publications.
Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage Publications.
Schmitz, C. L. (1995). Reframing the dialogue on female-headed single-parent families. Affilia, 10, 426-441.
Sherman, Arloc. (1994). Wasting America's future: The Children's Defense Fund report on the costs of child poverty. Boston, MA: Beacon Press.
Strauss, A. & Corbin, J. (1990). Basics of qualitative research. Newbury Park, CA: Sage Publications.
Go back to QUIG 1997 Home page
