Sondra K. Patrick, James J. Fletcher, and Sara Looney
Introduction
Our presentation at the 1996 Conference on Qualitative
Research in Education accomplished two goals. First, it demonstrated the
benefits of incorporating video inserts into a slide show presentation.
Second, it examined the relative utility of three different ways of reporting
findings for a qualitative study - written report, slide show presentation,
and video. In the course of preparing the reports we were also able to
assess the role of technology in the reporting process.
Miles & Huberman (1994) and Dey (1993) argue that the method through
which a researcher reports research data plays a critical role in how the
findings are perceived and used by an audience. In preparing the initial
report of our qualitative study, we were very aware of our audience. Because
we knew their time was limited and that our findings could be of value
to them in the institution's decision making process, we carefully thought
about our reporting strategy. Our first report consisted of a 23-minute
video tape supplemented by a 5-page written text. Later, we turned to presentation
software capable of handling video inserts to maximize the impact of our
contextual data in a slide show presentation. Through our exploration of
these three methods of reporting research results, we found that technology,
though facilitating the analysis and reporting process, imposed constraints
that often limited the usefulness of segments of data. The following discussion
will describe the project's background, data collection method, findings,
reporting strategies, and what we learned about the impact of technology
on research reporting.
Background
Like many institutions, George Mason University (GMU) regularly collects
information about its students through large scale surveys. These quantitative
survey instruments provide university administrators with a general picture
of students' experiences. Frequently, however, they do not provide the
richness and depth of information needed to make effective administrative
decisions that impact today's demographically diverse student populations.
To gain a more holistic picture of our student population, we incorporated
qualitative research methods into our institutional research process.
Our research study began in 1992 when the President of our University funded
a research study designed to identify the characteristics of students who
have a satisfactory academic experience at GMU. Because we already had
student data from large quantitative studies, we decided to capitalize
on the strengths of qualitative research to provide us with an in-depth
description of our students' experiences in their own voices. Through a
series of focus group interviews, students told us about their academic
and student life experiences at GMU. They voiced their concerns and brought
to light key issues facing students today. The components of our research
design are explained below.
Data Collection
From January to August 1993, we conducted 12 focus group sessions with
63 George Mason University students. These one-hour interview sessions
were held at various locations on campus and involved a representative
sample of our diverse student population, including representation across
different categories of student involvement such as campus athletes, student
leaders, commuters, and campus residents. Student participants were obtained
by public announcements for volunteers in classrooms, at student organizational
meetings, and through university offices such as the Minority Student Services
Office, Student Housing Office, University Scholar's Office, and Athletic
Services Office.
Each one-hour focus group session contained 2 to 14 students and was held
in a conference room where the students and moderator sat around a large
table. As an icebreaker, each session began with informal introductions
and a brief description of the research project. Basically, the same 12
questions were asked to each group except for minor variations for returning
adult students and campus athletes. The moderator was free to probe as
students responded with new and interesting avenues of related discussion.
In addition, each session was videotaped and audiotaped for later transcription
and analysis. [We did request written permission from each interviewee
at the beginning of each session before taping.]
Our data sample consisted of 63 students (54% Males, 46% Females, 65.3% Whites, 15.8% African Americans, 17.4% Asians, and 1.5% Hispanics). Our data source was the entire student population (N=20,829) of George Mason University in Fairfax, Virginia. We tried to accurately represent the diversity of our student population in organizing the focus groups. African American and Asian students actually constituted a somewhat larger proportion of our student sample than their numbers at the university. Also, our sample reverses the actual gender status at GMU. In 1992, our entire student population consisted of 45.7% males and 54.3% females. Of this group, 5.6% were African Americans, 9.8% Asians, and 3.7% Hispanics.
Findings
Through these 63 student voices, we moved a step closer to understanding
the needs and expectations of our diverse student population. For the purposes
of our study we defined 'successful experience' as one in which the student
expressed satisfaction with his/her student life and had no intention of
transferring to another institution. Our findings revealed a complex picture
of two types of students who are most likely to have a successful experience
at our institution. First, among traditional age students seeking a traditional
college experience, the most successful students were those who used their
own initiative and became actively involved in campus activities early
in the Freshman year. Second, most non-traditional age students said that
since they identify with off campus communities, they look to the University
only for their academic needs.
In addition to characteristics of success, other themes important to our
ability to serve our students emerged. For example, although the majority
of students indicated satisfaction with their academic experience, some
pointed to the need for broader cultural representation in classroom curricula
and textbook selection. Similarly, many students who expressed satisfaction
with their student life experiences also expressed a need for more community-building
activities that unite our diverse student population on both academic and
social levels. One African American student brought a 'world' literature
textbook to the focus group interview and stated emphatically that 'his
world' was not represented in it. We also found that retention and involvement
in campus life were closely intertwined. While some students indicated
a reason to be dissatisfied with some aspects of their experience at GMU,
the main point uncovered by our research was unaffected, namely, that students
who got involved from the start of their years at the university were less
likely to transfer and more inclined to have a positive college experience.
Although many of our findings confirmed what we already knew about our
students, it was the first time that the confirmation came directly from
the students themselves in their own voices. We felt that this was an important
contribution of our study to our academic community. When we thought about
ways to report our findings, our goal was to find a reporting strategy
that both
retained the integrity of the contextual data (student voices) and maximized
audience impact.
Ways of Reporting
We explored three ways of reporting our findings: Written report, slide
show with or without video inserts, and video. As noted by Elliot Eisner
in his Keynote Address, there are many other ways of reporting; however,
these are the three ways most widely used on our campus. The use of video
as a research reporting tool is one of the most recent ways of reporting
research findings on our campus, and it began with our project. As stated
in the introduction, our findings were first reported on a 23-minute VHS
video tape accompanied by a written executive summary. Later, we produced
a slide show presentation using PowerPoint presentation software with video
inserts.
Technology definitely had a positive impact on our reporting strategy.
Through the video format, we were able to quickly and effectively communicate
a wide range of issues to a large audience. Since our initial target audience
was over-committed university administrators and faculty, the efficiency
of this reporting format was a critical benefit.
Although technology positively enhanced the reporting process for us, it
also posed some important concerns. The concerns primarily focused on the
ways in which technology imposed restrictions on data collection, data
analysis, data usage, and data reporting. The following discussion will
examine what we found to be the advantages and disadvantages of each of
these three ways of reporting and the impact each had on the message.
Written Report. Obviously, the written report is the most common
way of reporting information. We first wrote an executive summary to accompany
the video then later wrote a longer report. We found two basic advantages
to reporting via written text. First, the written text maximized the use
of our transcribed data, allowing us to use as many excerpts as necessary
to get our message across. Second, it increased the usefulness of the data
since we had no technical constraints except our degree of word processing
proficiency. Our report could be as long as we wanted, and there were no
restrictions on the number of themes or topics we could cover. In this
sense, we found the written report to be the most information laden of
the three ways of reporting. Conversely, the main disadvantage of the written
report centered on the degree of impact. After careful consideration of
our target audience, we knew that a long, written text would not be read
by many members of our audience. Today more than ever, we are experiencing
information overload in almost every aspect of our lives. Duplicating a
long, written text (30-50 pages) then distributing it widely via campus
or electronic mail knowing that few would actually read it seemed very
inefficient and ineffective. Since we wanted to share our findings with
a large portion of our target audience, (i.e., we wanted to maximize the
impact of our findings) we turned to technology.
Slide Show Presentation. Reporting research findings
using bulleted text with or without video inserts can be an effective means
of reaching a larger audience. We identified three advantages to this reporting
method. First, information can be organized and disseminated quickly through
a bullet format. Second, major points and key issues can easily and quickly
be highlighted. Third, since most presentation software packages allow
the user to insert video clips, we were able to incorporate the advantages
of this reporting procedure with our contextual data. Not only could our
audience quickly understand the key elements of our study; they could also
hear the voices and see the faces of our students. The interplay between
text and video in slide show presentations has the potential to disseminate
information quickly without totally losing the contextual features of qualitative
research methods. It is, however, a fairly recent method of reporting that
carries with it some important disadvantages. Although a slide show format
is an effective communicator of information, bullets alone lose the richness
of both the written text and the contextual data. Inserting video clips
into a slide show presentation can help address this issue. However, video
clips need large amounts of computer memory, limiting the number that can
be used. We found that we could insert only four 30-second video clips
into our presentation. These two minutes of video used approximately 140
megabytes of memory in addition to the slide presentation software. For
our presentation we used a 486 PC with an 850 MB hard drive running at
75mhz. A slower computer would probably not be able to synchronize a video
clip's sound and image. Obviously faster computers with larger memory capacities
would allow for more video inserts thereby increasing the use of the contextual
data.
Video Presentation. Initially our findings were reported in a 23-minute
non-linear video format. We chose this format with our target audience
in mind. We knew that time could prevent many members of our audience from
reading a long written report. One of the major advantages, then, to the
video format is that a researcher can maximize the impact of research findings
by reaching a larger audience more quickly. Another important advantage
is that the video enabled us to connect the text to a speaker. This connection
gave life and importance to the data. It was no longer just another institutional
research study, it was rich insight into the lives of our students. Their
voices were heard rather than interpreted through the words of a university
administrator or faculty member. This way of reporting can have a powerful
impact on an audience.
Video reporting is not, however, without its disadvantages. Videos are
extremely time consuming to create. In addition to transcribing and analyzing
large amounts of contextual data, they require additional time to edit.
We had over 20 hours of video tape. Many hours were spent reviewing video
clips before we were able to reduce the 20 hours to 23 minutes without
compromising or interfering with the data-to-message relationship. Video
reporting also requires specialized knowledge. We had to learn to video
edit and we had to rely heavily on our technical support people. The art
of video editing takes many years of experience to master. For most of
us, the availability of technical support is critical. The most salient
disadvantage to video reporting is that technology often increases the
potential for altering the message. Many of our students made important
points during the focus group interviews but for technical reasons such
as background noise, poor positioning of the camera, or poor audio quality,
we were unable to use their video clips in the final video report. To minimize
this disadvantage, our video was framed by a spoken text that incorporated
many of these potentially lost points.
In comparing the reporting formats, we discovered an inverse ratio between
the affect on the audience and the amount of data communicated. The chart
below summarizes the differences we found. The word 'impact' in the chart
below refers to the reporting format's affect on the audience and its ability
to elicit response.
| Video | Slide Presentation | Written Report |
| High------------------ | --------Impact-------- | -------------------Low |
| Low------------------- | -----Information----- | ------------------High |
Lessons Learned
There are five important lessons that we want to share:
1.Careful initial planning of any research study remains critical. Although
we decided early in the project development stages that we wanted to use
video to report our findings, we lacked the experience necessary to fully
understand the consequences of this decision for the project design. To
use video as a reporting strategy, it is necessary to think ahead about
the availability of appropriate computer and audiovisual equipment, technical
support/expertise, research subjects, transcribers, and appropriate space
for videotaping interviews.
2.We used focus groups as the format for data collection. Each focus group
session must be carefully planned, giving careful consideration to the
setting and location of the focus group since the video's message will
include the focus group environment. We also recommend using the same room
for all focus group interviews, paying particular attention to arrangement
of chairs, pictures on walls, and outside noises such as ringing telephones
or hall traffic. Setting up student focus groups can be a very time consuming
process. We found it necessary to call students frequently and remind them
about an upcoming session, emphasizing the importance of their contribution
to the research study. Even then, many did not show. Although we did not
pay students for their participation, we did provide refreshments. Finally,
we recommend that all interviewees sign a consent form at the beginning
of the interview session giving permission for their image and comments
to appear in future video reports.
3.As with all research studies, the reporting format must be tailored to
the intended audience. Our immediate audience was our executive administration.
Sensitive to the demands on their time, we felt that it was inappropriate
and ineffective to produce a text of more than 5 pages or a video of more
than 30 minutes.
4.The video taping process is also one that requires careful attention
to detail. Work closely with the videographer. He/she needs to understand
the nature of the research project and how the video tapes might be used
in the future. We started our study with only one microphone positioned
in the middle of a conference table and soon discovered that we needed
more to increase the audio quality. We would recommend viewing video tapes
early in the study so that changes could be made to enhance audio and video
quality if needed. This step will maximize the usefulness of the data for
future video reporting.
5.Our final lesson learned relates to the cost/benefit ratio. Using some
forms of technology to report research findings can be both costly and
time consuming. Our direct and indirect costs included the purchase of
presentation software (Microsoft PowerPoint), the need for a 486 laptop
computer with an 850 MB hard disk to support video inserts and external
speakers, and the use of the Video Cube ($50,000) for digitizing all VHS
tapes. In addition, we needed the services of a videographer ($20/hour)
and the expertise of our technical support people. Our expenditures in
time consisted of hours spent transcribing and analyzing tapes, learning
new software, learning video editing procedures, and editing/producing
both the video report and slide show presentation.
Conclusion
As we explored the uses of technology in reporting our findings, we
felt that we were on an expedition into unknown territory. At every turn,
we discovered something new and unexpected that caused us to continually
reflect on the relationship between the data, the message, and the medium.
From the start, we knew that there would be limitations to any reporting
strategy. We also knew that the 'information age' has imposed its own constraints
on time, making it more difficult to find time to wade through lengthy
documents. Today, disseminating information quickly and accurately is almost
synonymous with usefulness, even though we recognize the importance of
long, written documents to providing thorough, detailed understanding of
an issue or concept. We certainly don't want to minimize their contribution
to advancing knowledge. We do, however, want to encourage an openness to
other ways of disseminating what we know. We found technology to offer
two important advantages. First, it supports the kind of flexibility, speed,
and utility necessary to deal with the rapidly changing, complex issues
facing faculty and university administrators today. Second, it has the
potential to maximize the usefulness of vast amounts of contextual data
for decision making purposes by making it possible to disseminate information
quickly and concisely without completely losing the depth and richness
of language.
Although our research lead us into new territory, we are encouraged by
it because we see many ways in which our experience with this project will
enhance future research studies at our university. We hope that our presentation
and this report will help others who are exploring new ways of using technology
in the educational research process. Clearly the integration of technology
and qualitative research has important implications for making more informed
decisions in an increasingly more complex society.
References
Dey, I. (1993). Qualitative data analysis. London: Routledge.
Miles, M. G., & Huberman, A. M. (1994). Qualitative
data analysis. Thousand Oaks, CA: Sage.
Go back to QUIG 1997 Home page
![]()