Our Results
What does evidence-based mean?
Once a program is developed, we need to know if it works. Measuring effectiveness is a critical part of what we do. And yet, some things in life require rigorous studies to prove their effectiveness, while others have a lower burden of proof. For example we want to be 100% sure a drug is safe before we give it to our child, but we don’t need a study to show us that it’s worthwhile to take dinner to a new mother. The challenge for organizations is matching the assessment to the question being asked.
The gold standard for assessing early childhood program effectiveness is to say it is “evidence-based.” This is a very specific term that means the program has been assessed using a randomized control study, a methodology that comes from drug trials. The results of an intervention group are compared with the results of the control group before and after the intervention. Conducting a true evidence-based trial requires significant time, money and rigorous research that meets strict academic, legal, and ethical standards.
The results of rigorous, repeatable research that meets strict academic, legal, and ethical standards can ultimately become evidence. This evidence-bearing research is where Talaris finds inspiration for products and programs.
For a detailed discussion of evidenced-based programs, see this article (PDF) from the University of Wisconsin.
Our Results
Technically speaking, Talaris products and programs are not “evidence-based.” Rather, they are a) developed based on research and b) evaluated using a variety of study methodologies. Using survey tools and focus groups, Talaris materials have been evaluated by outside organizations to determine how they were used, whether or not they were helpful to professionals, and whether or not parents felt that the materials increased their knowledge of early development or confidence in interacting with their child after having received Talaris / Parenting Counts kits or materials. Over the years, we have conducted multiple studies of our effectiveness. Overall, these studies show that Talaris materials and the kits have been highly rated by parents and professionals for increasing both knowledge and confidence. Please see below for details on key studies.
Who: Child Health and Behavioral Development Center at Seattle Children’s Research Institute
What: The Talaris Institute has teamed up with Dr. Dimitri Christakis, Director of the Child Health and Behavioral Development Center at Seattle Children’s Research Institute, in his NIHCD sponsored early parenting intervention study. The goal of the study is to help parents support their child’s cognitive and emotional development.
When: 2012–2015
Method: Between 500 and 700 18 to 23-year-old first-time mothers in the Seattle area will be followed for 42 months beginning at the birth of their first child. In addition to other sources of parenting information and support, a group of these mothers will receive a selection of our Parenting Counts materials.
Results: This study will help us find out what parents like most about our Parenting Counts DVDs and brochures, as well as which of our materials are most useful to them.
Who: Health and Human Sciences (HHS) Extension, Purdue University
What: Health and Human Sciences (HHS) Extension educators presented eleven Parenting Counts lessons (short videos paired with parent handouts and group discussions) to parents and caregivers of young children in 5 counties of Indiana.
When: 2010–2011
Method: Pre- and post-surveys were administered to measure changes in knowledge and behavior before and after exposure to one of the Parenting Counts lessons.
Results:
Results analyze parents’ and caregivers’ change in knowledge and self-reported behavior before and after participation in a one hour parenting class. Topics presented included: understanding appropriate responses to newborns based on their communication cues, appropriate understanding and response to preschoolers’ emotions and feelings, and learning healthy ways to handle stress in order to help children develop healthy emotions.
242 parents and caregivers participated in the study. Since participants were assigned to one of 11 parenting lessons and not all post-tests were collected, sample sizes for each lesson were very low (with <15 respondents on average). Given these small sample sizes, most results are not statistically significant.
There were three lessons, however, that have somewhat larger sample sizes since they were used across multiple counties. These lessons included: Book of Jenny (topic: communication, n=18), Morning Madness (topic: stress, n=41) and Rough Day (topic: recognizing emotions, n=64).
For these lessons, examples of questions for which parents and caregivers showed a statistically significant increase in knowledge include:
- Babies experience stress in a similar way to adults. (true)
- It takes trial and error to learn to respond to baby’s cues. (true)
- A baby will be spoiled if you respond too quickly or too often. (false)
- If children do not learn healthy ways to calm down or react to stress, they may struggle to handle everyday stress as they get older. (true)
Examples of questions for which parents and caregivers showed a statistically significant positive change in behavior include:
- I comfort my baby as quickly as I can when he or she cries.
- I help my baby learn to manage stress by comforting and responding to my child’s “cues.”
However, given the small sample size, test results and conclusion are not representative. A more reliable conclusion could be reached with a larger sample size.
Who: State of Minnesota (Department of Education, Departments of Health and Human Services)
What: Shared "Life with Baby" and "Life With Toddler" parent guides (DVDs and booklets) with high risk families as part of a series of 4 home visits
When: 2008-2009 school year
Method: Post-survey measured home visitor and parent satisfaction with Parenting Counts materials.
Results:
"Overall, home visitors and parents felt that the Life with Baby and Life with Toddler DVDs were superb in providing child development information to families, especially first-time parents."
Percent of parents who agreed or strongly agreed that the DVDs:
- Provided useful and practical tips – 91%
- Strengthened their understanding of the importance of their role in their child’s development – 91%
- Were an extremely valuable resource – 93%
Percent of home visitors who agreed or strongly agreed that the DVDs:
- Were interesting and fun to watch – 100%
- Helped structure their session – 86%
- Facilitated discussion about child development – 73%
- Stimulated a conversation about community resources – 67%
Who: Harvard Project Zero
What: Local YMCA used Parenting Counts Materials to add a social-emotional education component to a program called Learning Games; compared results with other center using Play and Learn program that did not include social-emotional materials.
When: 2009-2010
Method: Pre- and Post Survey measured various child, caregiver, and child-caregiver interaction outcomes.
Results:
Participants using the program that included Parenting Counts (Talaris) materials reported greater positive change across all survey questions associated with the outcomes:
- Improved social-emotional development in children
- Increased reports of positive interactions between child & caregivers
Who: Washington State University
What: Evaluation of Life With Kits (Baby, Toddler, Preschooler) at various locations in Washington.
When: May 2008-April 2009
Method: Pre-test, Post-test, control group. Measured change in parenting attitudes, knowledge, and behavior before and after a series of home visits.
Results:
Parenting Attitudes
Brain development begins at birth, and therefore children are impacted by their relationships, emotions, and experiences during their first year of life.
Skills in reading and language development are shaped by parent-child interactions starting in infancy.
Future success in school and in relationships with others is built on healthy early social emotional development.
Parents are instrumental in shaping their child’s emotional and social skills.
In the group that viewed the Parenting Counts DVD’s as part of their home visits, parenting attitudes were significantly changed in these areas:
Parent Knowledge
Parents who received the toddler and pre-schooler kits demonstrated a statistically significant increase in knowledge. Parents in the control group did not.
Parenting Behavior
- Regardless of the type of kit, almost 1/3 of parents said that the kit increased the amount of time they play, interact, and communicate with their child.
- Parents of pre-schoolers read to their child more frequently after receiving the pre-school kit.
- Overall, almost 20% of parents who received the kits started or changed their child’s bedtime routine.
- 35% of parents said that their confidence had increased somewhat since using the "Life With…" kit, while 9% said their confidence had increased greatly.