David Greenfield
    Framing the Research Problem    Force Field Analysis    Ubiquitous Technology: The Cell Phone  
    Literature Review    Cycle One Report    Elevator Pitch    Cycle Two Report: Data Analysis  
   Final Report    Blogs    Related Work    About David Greenfield    Contact  •  Home    
 

Cycle 2 Data Analysis
PDF Version

At the end of cycle two, I realized that rather than create a collaborative project, I needed to focus on understanding the components that go into creating a successful collaboration. Because I have been working as an independent, the challenge that presented itself to me was from where and how I would obtain my data. After receiving feedback and advice from my instructors and peers, I chose to do individual wrap-up conversations with the two project partners (from the museum and from the school) as well as to conduct interviews with museum professionals that I planned to meet at the Museums on the Web conference that I attended in April 2007.

I was able to have an informative conversation with the museum director, but because of scheduling issues have been unable to meet with the school director (although we will be meeting during the summer). I had several intersecting and wide ranging conversations with colleagues attending MOW, but decided that a more thorough follow-up survey was needed.

The museum director (she is the director of one department) was quite frank about her experiences involving collaborative projects. She said that in general, she found it very difficult to collaborate with schools because of scheduling problems with teachers and administrators. She also said that she found many teachers to be overworked and under-assisted to be able to commit to collaborations in general and specifically to innovative projects. Her experience with technology is that the digital divide is very pronounced in LA city schools, making it virtually impossible to use. I found one comment that she made to be particularly curious. She spoke to me at length about the importance of institutional commitment and identifying the right partners for any collaboration. She told me that in reality, her department was not specifically suited for the type of innovative technology-based collaboration that I had proposed, and perhaps the museum itself was not suited. I found this very interesting since I my original conversation was with the executive director of the museum who had steered me to the department director.

The conversations that I had at the Museums on the Web conference were interesting, stimulating, fun but pretty general in nature. As a whole, attendees of this conference are techno-friendly, and are open and interested in collaborations and innovation. The main concerns of the people that I spoke with are a perennial lack of funding and a shortage of personnel.

As interesting and fun as the conference was, I was still not satisfied with the information that received there, so I decided to create a short survey of eight questions about the collaborative process:

  • Have you had experience in working with museums/school partnerships and collaborations?
  • If yes, how may projects have you worked on?
  • What was your role in these projects?
  • What is the average amount of time (days, weeks, months) that you have devoted to developing a collaborative project (from inception to launch)?
  • On a scale of 1-5 (one being the least, 5 being the most), have these experiences been successful and have they met their goals?
  • What are the 5 primary skill sets required in the creation of these collaborations?
  • What are the 5 main milestones designated in the process?
  • Who initiated the collaborative projects that you worked on? Were they internally driven or did they come from the partner (or other) institution?

The survey was sent via email to 18 individuals with whom I had conversations with about collaborations.  I received 7 replies. Although not enough to be called a “comprehensive” survey, I did gain an understanding of several of the important attributes needed to develop a successful collaborative project (see raw data chart below).

The responders are all professionally active in museum education.  Three are actual museum educators, one is a professor of museum studies, one is a new media designer who specializes in the design of educational exhibitions, one is a researcher in education specializing in museums and one is doctoral student in the business of museums.  Of the seven, only one person has experience in the classroom, but two have graduate degrees in education. Six of the respondents identified their positions as project managers/leaders or designer/developer; one role was identified as evaluator/researcher, one as teacher/trainer to participating teachers. The average quantity of collaborative projects that each individual worked on is 5.5 (the range went from 2 to 12) and the average amount of time spent was 5 months (the range went from 21 days to 1.5 years).  

The responses that I received to questions about needed skill sets needed and procedural tasks are varied in nomenclature, but similar in tone. I identified the eight most important skill sets as:

  • Project management
  • Organization
  • Scheduling
  • Flexibility
  • Patience.
  • Communication skills.
  • Resourcefulness.
  • Content awareness.

Additionally, there are 5 procedural milestones and goals that need to be clearly identified and defined.   A successful project (where goals are met integrated into a collaborative project

  1. Information needs assessment (discovery).
  2. Development of shared vision and goals.
  3. Identifying key stakeholders for project commitment and buy-in.
  4. Obtain funding.
  5. Iterative design process with time scheduled for multiple assessments and refinements

Six of the seven responded to the question about who initiated the collaborations:  5 said that they were internally driven and all seven expressed a high degree of satisfaction in working on museum-school collaborations (4-5 on a scale of 5).

As I reviewed the responses, I became aware of important information that I was not obtaining, such as:

  • The ratio between the quantities of individual collaborations done over an identified time span (for example 3 collaborations completed over a 4 year period).
  • If there was a period for reflection included in the process.
  • If the collaborations included technology
  • Defining a general attitude to technology
  • Size of the institution
  • Quantity of people involved.

I think that as informative as this survey is, it really functions more as an introduction to the subject of developing institutional collaborations.  Most of the responses that I received only spurred my curiosity for additional and needed data. Additionally, I was overly sensitive to open-ended questions to not influence the responders. These types of questions provided the responders more opportunities to express their meanings as well as gave me options for interpreting and inferring meaning. But it also made it more difficult for me to code the responses for a comprehensive understanding of the process.

The quality and quantity of data that I did not obtain has shown me that I need to develop more sophisticated and comprehensive surveys to be able to better understand complex processes and concepts. To accomplish this, I need to make the development of surveys a more iterative process in both the development of the survey questions and the inclusion of a wider range of subjects (for example, this survey did not include several potential key players, such as classroom teachers and school administrators). I can also improve the quality of my surveys by making the development of them a more collaborative process. By integrating input from peers, colleagues and test subjects, I can develop surveys that will provide me with a more detailed data set and will not leave me wondering about the questions not asked.

In conclusion, I was not particularly surprised with the responses that I received. Although not a comprehensive survey, the responses did deflect much of what I have heard about and learned as a museum professional. Additionally, the responses do reinforce many of restraining forces that I identified in my Force Field Analysis. Resources management (money, time and personnel) and institutional commitment continue to be serious issues that hinder, if not prevent institutional partnerships and collaborations. Yet, even though these problems exist, they are not insurmountable. I would venture to say that 90% of the people that I spoke with regarding my specific project and collaboration in general are interested in developing a practical application of the idea.  I continue to believe in the viability of this project, but I have learned that success depended not only on my enthusiasm and knowledge, but on my ability to: 1) clearly identify serious partners who will be able to commit necessary resources as we well; 2) to adapt to the time frames and schedules of partner institutions and 3) to continue to search for creative and flexible solutions to issues as they arise. These three elements will help insure the success of using a mobile phone to bring together museums and schools to create an innovative and collaborative learning experience.


Raw Survey Responses

To help insure confidentiality, names of individuals, institutions and specific geographic locations have been omitted.

The top row identifies the titles and type of affiliated institution of survey participants.
The left column is the survey questions.

 

Head of Online Education, Art Museum

Assistant Professor, College of Information

Senior Research Associate, Learning Research Firm

Director of Education, Science Institute

CEO and Director, New Media company USA

PhD. Student, University of South Australia
International Graduate School of Business

Museum Educator,
Cultural Heritage Museum

1) Have you had experience in working with museums/school partnerships and collaborations?

Yes

Yes

Yes

Yes, The Institute has had numerous school partnerships over the years.

At the moment, the signature partnership is our new SLA which opened in September 2006. Right now, the first group of freshman are finishing their first year. The School District of XXX approached The Institute about partnering to develop a small high school dedicated to science leadership. The school will ultimately have about 500 students. It's a magnet school...students apply and are considered based on recommendations and expressed interest in science and technology (not necessarily on academic achievement.)

Yes

Yes, in a variety of forms, with direct collaborations, or mediated through a larger education agency.

Yes

2) If yes, how may projects have you worked on?

1 overall project with 12 schools to date, hopefully 15 schools by June 07

Depends on how you define museum/school partnerships. :-)  While working as a museum professional, I worked on at least half a dozen different museum/school collaborations, mostly online educational outreach type projects.  While working as a university professor, I have supervised two or three collaborations with local museums where university students worked with museum professionals (primarily on digitization projects).

Hard to remember, but perhaps    5-6.

Hmmm...it's hard to say how many. Over my 13 years, I can think of about 7 projects with which I've been involved, although to varying degrees.

Major one was Zoo project that one Best of the Web a few years ago. Did quite a bit of development for school and family connections there, including printout PDF activities.  I tend to do more of my “StoryKiosk” work which is based on a model of visitor storytelling, and when possible, I try to prepare a teacher resource for that (2 page printout activity).  I’m working with Productions on an installation at the National Archives in DC devoted exclusively to use by school groups, where groups have an experience in the archives and then prepare a media report that they all watch on big screen.

3 major programs involving a large number of individual schools

2

3) If yes, how may projects have you worked on?

Project leader/evaluator

With the museum/school projects, I was responsible for designing and developing the online tools used by the students in the projects.
With the university/museum projects, I supervised university students working with the museums.

 I was the researcher and/or evaluator so: convening focus groups, interviewing students and teachers, advising the museum, reviewing curriculums, testing curriculum and games, analyzing responses, working with the museum to improve.

I am somewhat involved with the SLA, although less so now than I was in the beginning. I participated in the planning for the school and, during the first few months, worked directly with the students one day a week.

During the mid-1990s, I was the director for a project that connected The Institute with a K-8 public school for about 4 years during which we developed and delivered professional development programs and projects that helped teacher learn how to use the Internet to support science inquiry. It was a whole-school change model funded by the National Science Foundation.

From 1998-2003, I co-directed a project, also funded by NSF, called the
Keystone Science KSN, which formed an alliance of schools and science teachers throughout Eastern XXX. We delivered professional development events and online curricular companions.

I've also been involved with the NSF Math Science Partnership project in the School District of XXX for the past four years. Before that,
I was involved with the Urban Systemic Initiative and Partnership
projects. I wasn't the director of any of these, but I played a part in
all of them, helping teachers use technology effectively in science
education.

I'm also responsible for the Educator Advisory Committee at The Institute which is a group of 25 teachers who meet with me 3 times a year to act as a focus group for us in our planning of exhibits and programs.

The Zoo was a 2-year project, National Archives an 8-month project. Storykiosk installations tend to be about 4-6 months from start to finish.
 

Project initiator, project manager, project supervisor

 In one project I trained teachers how to make an in-school and online exhibition with their students to explore their students' Jewish heritage and family histories.  The teachers provided us with feedback on the website that we are building for this purpose. 
In the second project I acted in a support role to facilitate the creation of an interfaith exhibition between one Jewish and one Muslim school. 

4) What is the average amount of time (days, weeks, and months) that you have devoted to developing a collaborative project (from inception to launch)?

Project planning and introductory workshop: approx 21 days over c. 6 months
Project implementation: 15 days over 10 months

Varies tremendously by project, especially since most projects continue to evolve after launch.
I'd say on average development takes around 3-4 months, but there's a large standard deviation!

 It varies widely depending on my role, but a median time of 5 weeks per year.

 That varies, of course, but I would say it averages to about half of my
time.

  

Highly variable, the larger ones took months, as there was usually grant funding process to establish the process.

By the end of the first project I mentioned I will have spent 1 1/2 years on the project.  I spent about 3 days on the second project.

5) On a scale of 1-5 (one being the least, 5 being the most), have these experiences been successful and have they met their goals?

4

I'd say most experiences have been a 4 out of 5. We've been very lucky with our collaborations.

4

Yes, all of the projects have been successful and have met their goals.
Certainly, some have been more successful than others, but we've never
really had a bad experience. (At least not during my 13 years there.)

But, it may be a smidge early to declare the SLA partnership as a full
success yet. It's going well, but there have certainly been bumps.

Awareness of constraints on teacher time
Ability to develop imaginative plans that students are excited about
Awareness of media capabilities, both high end and low end (PDFs are great and often underutilized)
Availability of time to make them work and to follow up

I would say 4/5 I think they often tend to meet predetermined goals only partially and often achieve results not expected which are equally or more valuable to the collaborators.  Most collaborations seem to involve a lot of mutual learning between schools and museums, which means original expectations and goals often change in the course of the project.

4

6) What are the 5 primary skill sets required in the creation of these collaborations?

• Project conceptualization and planning
• Scheduling and logistics
• Teaching with objects in the museum and with images on a web site
• Designing an evaluation to address specific outcomes
• Evaluating and creating a report about the projects and use of the web site in each school

It's hard to identify five specific skills, but I'd say the most important skills are not in the use of technology itself, but in the
• ability to assess information needs,
• to work with the users of the system, and
• to coordinate educational activities among all the different participants, from content creators, to teachers, to students, to museum professionals.

Satisfying the needs of all the project participants is far more difficult that developing the actual technological systems.

Organizational skills
Management skills
Flexibility and willingness to try out ideas, openness to change
Patience
Observation/Ability to reflect

1. Pragmatism. It's very important to be pragmatic, and realistic.
Understand that a partnership with the museum can be a win-win, but the
schools may not be able to turn on a time.

2. Have good content. If you don't deliver high quality programs that
teachers value, nothing else will matter.

3. Personalities matter. Veteran teachers have little tolerance for amateurs.

4. Nurture the seeds. Partnerships take time to grow strong roots. They
need to be nurtured...and not always with more and more programs.
Sometimes just a fun outing is what's needed.

5. Institutional support, of course, is important. Without high-

 

Patience and persistence: there are often many institutional barriers that make collaborations difficult both on the school and museum side.
 
Resourcefulness: As chronically under funded institutions, collaborations between museums and skills are always drawing on the willingness of individuals to do more than they are paid for, to find ways to do things 'on the cheap', and to be creative in bringing in resources.
 
Awareness of what a community needs: All school communities are not the same.  It is important to adjust any program to the particular skills and needs of the school community involved.
 
Ability to make something sustainable:  Too many projects are done as one offs. What schools need is something that is sustainable, that creates a chain of mutual value that will motivate all participant to continue.
 
Obviously communication skills and the ability to get stakeholder buy in are essential too.

Patience, listening skills, communication skills, flexibility, planning/managerial skills 

7) What are the 5 main milestones designated in the process?

• Determining the effectiveness of trying to model our teaching methodologies in the design and functionality of the web site
• Observing how students use the web site and respond to the images without the intervention of a museum educator
• Discovering how teachers and students use the web site in unexpected and imaginative ways
• Finding out which of the site’s resources and features are actually most useful for classroom teaching and learning
• Using what we learned from these collaborations to inform future developments on the web site

Again, this varies from project to project, but any project needs to include

• information needs assessment,
• an iterative design process with extensive user testing and redesign,
• pilot testing of the beta version of the project, and
• formative and summative evaluations during and after each project.
• There has to be a constant focus on evaluation and testing throughout the project life cycle.

Development of concept and team
Obtainment of funding
Design Phase
Initial Implementation
Refinement

Milestones vary. For our partnerships, milestones are frequently
associated with the terms of a grant. Like, for example, certain
objectives need to be met at the end of each year.

Generally speaking, milestones tend to be event-based. Like, if we're
working on projects together, culminating events provide an opportunity to
showcase what we've done. Those celebratory moments are important, and
often serve as turningpoints for the partnership.

Last semester, the kids in the SLA created presentations about topics in
the exhibits in the museum. Then, in December, each group presented to the
whole school and museum community. That event was a real
milestone...something we were working toward and then achieved together.

I know that's not really 5 milestones, but I can't really think of any
others.

 

  • Shared vision and agreement on desired outcomes for each party.
  • Identification of stakeholders and key players
  • Establishment of goals and project plan
  • Buy in at the CEO level in both organizations
  • Funding commitment

As I was not the primary leader on the second project I will only discuss the first.
1.  Discover the needs of stakeholders
2. Develop the website
3. Train Teachers to use the curriculum and the website
4. Refine the website and worksheets
5. Develop video to further support teacher training

8) Who initiated the collaborative projects that you worked on? Were they internally driven or did they come from the partner (or other) institution?

We (the XXX Ed dept) initiated the collaborative projects

That's an interesting question.

With the museum/school collaborations I worked on, often a school would approach us first, and we would develop a project together, but then we might approach other schools with similar ideas later on.

With the university/museum collaborations, it's mostly been museums contacting us to assist them with their projects.

About 85% from the other partner organizations.

Most of the projects have been internally driven, although frequently with the external partner being a co-conspirator. But, generally speaking, I'm usually involved with projects where my own institution is considered the lead. Often, that has to do with funding structures. As a non-profit with an established name, history, and reputation, we often have access to funding opportunities that our partners might not. (The School District pops to mind as one example.)

Even when we act as "the lead," though, we always involve our partners from the very earliest stages as true, equal partners. Without that collaborative engagement from the get-go, the odds of ultimate success for the project decrease significantly.

We were contacted about the l Zoo project, but suggested particular activities to them. (They knew they wanted to connect with classrooms, but didn’t have a specific plan).
 
The dialogue was a good thing – they more we spoke on the subject, the more they signed up for! :-)

 

We initiated both the project and the collaboration