Survey Savvy

By Cooper-Hewitt National Design Museum, December 6, 2007

Grade Level

Middle School

Subject Area

Language Arts
Mathematics
Social Studies

Lesson Time

Two to three fifty-minute class periods

Introduction

Cooper Hewitt Smithsonian Design Museum’s exhibition Design for the Other 90% explores a growing movement among designers to design low-cost solutions for the 90% of the world’s population who have little or no access to most of the products and services many of us take for granted. In this lesson, students will learn about some of the innovations included in the exhibit and create a survey to gather information about opinions of these innovations.

National Standards

Common Core Literacy for Other Subjects

Strand Reading for History/Social Studies Grades 6-8

RH.6-8.3. Identify key steps in a text’s description of a process related to history/social studies (e.g., how a bill becomes law, how interest rates are raised or lowered).

RH.6-8.4. Determine the meaning of words and phrases as they are used in a text, including vocabulary specific to domains related to history/social studies.

Strand Writing for History/Social Studies, Science, and Technical Subjects Grades 6-8 WHST.6-8.1. Write arguments focused on discipline-specific content. WHST.6-8.7. Conduct short research projects to answer a question (including a self-generated question), drawing on several sources and generating additional related, focused questions that allow for multiple avenues of exploration. WHST.6-8.8. Gather relevant information from multiple print and digital sources, using search terms effectively; assess the credibility and accuracy of each source; and quote or paraphrase the data and conclusions of others while avoiding plagiarism and following a standard format for citation.

Common Core English Language Arts

Strand Speaking and Listening Grade 6-8

SL.6-8.1. Engage effectively in a range of collaborative discussions (one-on-one, in groups, and teacherled) with diverse partners on grade 6 topics, texts, and issues, building on others’ ideas and expressing their own clearly. SL.6-8.2. Interpret information presented in diverse media and formats (e.g., visually, quantitatively, orally) and explain how it contributes to a topic, text, or issue under study. SL.6-8.5. Include multimedia components (e.g., graphics, images, music, sound) and visual displays in presentations to clarify information.

Common Core Mathematics 6-8

Cluster Use random sampling to draw inferences about a population.

Grade 7

7.SP.1. Understand that statistics can be used to gain information about a population by examining a sample of the population; generalizations about a population from a sample are valid only if the sample is representative of that population. Understand that random sampling tends to produce representative samples and support valid inferences.

Mathematics

Level III. Standard 1. Uses a variety of strategies in the problem-solving process Benchmark 4. Formulates a problem, determines information required to solve the problem, chooses methods for obtaining this information, and sets limits for acceptable solutions Level III. Standard 6. Understands and applies basic and advanced concepts of statistics and data analysis Benchmark 5. Uses data and statistical measures for a variety of purposes (e.g., formulating hypotheses, making predictions, testing conjectures)

Benchmark 10. Understands basic concepts about how samples are chosen (e.g., random samples, bias in sampling procedures, limited samples, sampling error)

Language Arts - Writing

Level III. Standard 1. Uses the general skills and strategies of the writing process Benchmark 1. Prewriting: Uses a variety of prewriting strategies (e.g., makes outlines, uses published pieces as writing models, constructs critical standards, brainstorms, builds background knowledge) Benchmark 11. Writes compositions that address problems/solutions (e.g., identifies and defines a problem in a way appropriate to the intended audience, describes at least one solution, presents logical and well-supported reasons)

Language Arts - Speaking and Listening

Level III. Standard 8. Uses listening and speaking strategies for different purposes Benchmark 1. Plays a variety of roles in group discussions (e.g., active listener, discussion leader, facilitator) Benchmark 6. Makes oral presentations to the class (e.g., uses notes and outlines; uses organizational pattern that includes preview, introduction, body, transitions, conclusion; uses a clear point of view; uses evidence and arguments to support opinions; uses visual media)

Objectives

Students will:

learn about the benefits of several new innovations
participate in small-group and large-group discussion
conduct internet research
examine, evaluate, and analyze the elements of survey design
design an original survey with group members
create a class presentation

Resources

Cooper-Hewitt, National Design Museum: Design for the Other 90% http://www.designother90.org/solutions/?exhibition=12
Lifestraw: http://www.designother90.org/solution/lifestraw/
Solar Dish Kitchen: http://www.designother90.org/solution/solar-dish-kitchen/
Q-Drum: http://www.designother90.org/solution/q-drum/
http://www.statpac.com/surveys/question-qualities.htm
http://www.stats.gla.ac.uk/steps/glossary/sampling.html#srs
http://www.nss.gov.au/nss/home.nsf/SurveyDesignDoc/F9D27A66738B57ADCA2571AB00247B4F?OpenDocument
http://stattrek.com/survey-research/survey-bias.aspx
“Survey Savvy” handout (attached)

Materials

Computer with internet access

Procedures

Building Background Generating Ideas

The purpose of this activity is to provide students with an opportunity to explore varied aspects of design innovation.

1. As a class, visit Cooper-Hewitt’s Design for the Other 90% website. Invite students to read the introduction aloud. 2. Show the students the following three inventions from the Design for the Other 90% Web site:

Lifestraw: http://www.designother90.org/solution/lifestraw/
Solar Dish Kitchen: http://www.designother90.org/solution/solar-dish-kitchen/
Q-Drum: http://www.designother90.org/solution/q-drum/

After the students view each invention, ask them the following questions:

What does the invention do?
What issue does this invention address?
Who does it help?
Do you think this is an important innovation? Why or why not?
Who created it?

After viewing each innovation, ask students to discuss what the inventions have in common. 2. Divide the class into small groups. Ask each group to visit the Design for the Other 90% website and choose one additional innovation to research. Have students answer the questions above about the innovation they’ve chosen and present their findings to the class. Teacher Note: You may also choose to assign this as homework.

Steps for Learning Activity One: Survey Parameters

The purpose of this activity is to help students understand the parameters involved in setting up a survey.

1. The class should now have information about several innovations presented in the Design for the Other 90% exhibit. Write the name of each innovation on the board. Tell the students that they will be designing a survey based on these innovations. 2. Present students with the following prompt: The Design for the Other 90% website includes these creative, innovative solutions for some of the world’s problems. Who votes for this innovation? (Point to one of the innovations on the board.) After the students respond, ask them how they came to their decision. Discuss the importance of identifying an attribute on which to vote. Revise your question in one or more of the following ways and repeat the class vote.

Which of these innovations do you think would help the greatest number of people?
Which of these innovations do you think costs the least to implement?
Which of these innovations do you think is the most beneficial to the earth?
Which of these innovations do you think would be the least successful?

Ask your students to discuss how the design of the questions can influence the results of a survey. As a class, review additional information on designing survey questions at http://www.statpac.com/surveys/question-qualities.htm. 3. Ask the students to look around the room at the people who are casting votes in the class survey. Ask the following questions:

How many people voted?
What do the voters have in common?
How are they different?
What is the age-range of the voters?
Are there more males than females voting?
Are all the voters from the same town?
Is it possible to pick one or two students who can accurately represent the whole class?

Discuss the term "sample," and how the characteristics of a sample can influence the results of a survey. As a class, discuss common types of sampling after reading the information presented here: http://www.stats.gla.ac.uk/steps/glossary/sampling.html#srs. 4. Discuss the method used to conduct your class survey. Tell the students that they were a captive audience who listened to your presentation about several innovations from the Design for the Other 90% website, and then had time to research on their own. As participants, they were fairly well-educated on the topic. Discuss the following questions:

How do you think the results of the votes would have differed if you had only been given the name of each innovation, and shown a picture?
How do you think the results of the votes would have differed if you had been given five minutes to visit the Design for the Other 90%, and then been asked to vote?
How do you think the results of the votes would have differed if you had been given the information during lunchtime in the cafeteria and been asked to vote on your way out?
How do you think the results of the votes would have differed if you had been sent home with the information and told to bring back your vote whenever you were ready?

Discuss how the format in which a survey is presented and the method of data collection can influence the results of the survey. As a class, visit the following website that contains additional information on research methods: http://www.nss.gov.au/nss/home.nsf/SurveyDesignDoc/F9D27A66738B57ADCA2571AB00247B4F?OpenDocument. 5. Draw the students’ attention back to the six innovations listed on the board. Ask the students how the voting might have been influenced in the following situations:

You had introduced one of the innovations using personal information: for example, “This is one of the best ideas I have ever heard.”
Your descriptions of the innovations were confusing and incomplete.
You miscounted the number of votes for one or more of the innovations.
You included the vote of a student who arrived late and missed the introductory information on the innovations.
The vote was taken on the day of a big exam that many of the students in the class were worried about.

Discuss how bias and error can influence the results of a survey. As a class, visit the following website that contains additional information on bias in survey results at http://stattrek.com/survey-research/survey-bias.aspx.

Activity Two: Designing a Survey

The purpose of this activity is for students to work collaboratively to create a survey. 1. Divide the class into small groups. Give each group a copy of the “Survey Savvy” handout. Ask the students to use the information they’ve learned to create a survey based on the Design for the Other 90% innovations. 2. Allow each group to present its survey designs to the class. When the presentations are complete, as a class, vote on the best survey design. Lead a class discussion using the following questions as guidelines:

Was the question clear enough, i.e., what did you mean by “best”?
Was the group that voted large enough to yield a useful result?
Was the group that voted representative of the school as a whole? Could the results be generalized to the school population?
Was the method used for voting adequate?
Did the voters have enough information to make a valid choice?
Did some students vote for the survey designed by their friends?
Did some vote for their own design regardless of its quality?
Were there any errors in counting votes?

Collecting and Analyzing Data

1. Give each student a copy of the winning survey form and give them time to conduct the survey among their peers, their teachers, their parents, etc. Once the survey is complete, lead a discussion on issues that arose during its implementation. Talk about the design of the questions, the appropriateness of the method used, and any bias that might have developed.

2. Compile the results of the survey and use these ideas as a starting point for analysis:

What was the sample like? Were there more male than female participants, or more teachers than parents? What was the average age of the participants? How did the characteristics of the sample influence the results?
Analyze the data. Did the teachers’ votes differ significantly from those of the students? Did one of the innovations receive a large number of votes compared to the others? What might have contributed to these results?

Assessment

Create a class rubric with your students that will help them understand the effectiveness of their work. Use the following guidelines to help create the rubric.

Rate how well you understood the concept behind the innovations on the Design for the other 90% website.
Rate the quality of your question design.
Rate the quality of your sample population.
Rate the appropriateness of your research method.
Rate your understanding of bias in survey results.
Rate your participation in the creation of the survey.
Rate the quality of your presentation.
Rate the quality of your group’s collaborative efforts.

Enrichment Extension Activities

Differentiation for Elementary School:

Younger students may need help designing their surveys. Have an adult volunteer, teaching assistant, or teacher's aid help each group develop their survey, or take turns working with each group yourself.

Differentiation for High School:

Have each group implement their own survey prior to voting for the most effective survey. The students can then compare their results with other groups to decide which survey was most effective and why.
Students can take what they learned about survey design and create and implement another survey on a subject matter of their choice.