Findings from ReCAPP's Online User Survey 2004
Lori A. Rolleri, MSW, MPH
ReCAPP Project Director and Editor-in-Chief
This survey summary contains the following sections:
Over the past five years, ReCAPP's project team has relied on several sources to help determine the content and organization of the web site. During ReCAPP's preparation phase for initial launch in 1999, an advisory team of adolescent reproductive health experts was assembled to help conceptualize the site and determine a preliminary schedule of topics to be presented on ReCAPP. As the web site expanded its reach, more and more feedback was received from ReCAPP's users in the United States and abroad. This feedback was used to improve ReCAPP's navigational structure and prioritize content.
In addition to the feedback we receive from ReCAPP users, feedback from individuals participating in ETR's nationwide trainings on evidence-based effective programs and practices has also helped to guide ReCAPP's content. The ReCAPP project team meets regularly to discuss recent interactions ETR staff have had with practitioners in the field, as well as the emerging research on adolescent reproductive health presented in journals, web sites and conferences, newly published educational materials (e.g., curricula), and the effect of changing politics and policies on our work. However, the best profile of ReCAPP's users, including their professional needs and the ways they use ReCAPP, comes from the data we collect from online user surveys.
ReCAPP has developed and launched two online user surveys in its history. The first survey was launched in January 2002 and yielded 191 responses. The second survey was launched from September 1 to October 19, 2004 and yielded 322 responses. This paper will summarize the results of this most recent survey and will serve several purposes:
- it will be shared with ReCAPP's 6,000+ users;
- it will be used to develop future grant proposals to fund ReCAPP; and
- it will guide the ReCAPP project team in developing future editions of ReCAPP.
ReCAPP posted a 22-question user survey on its web site from September 1 to October 19, 2004. The survey included 18 closed-ended questions (i.e., multiple choice) and six open-ended questions. Once a final draft of the survey was completed, it was prepared for online presentation using SurveyMonkey1 — an online survey administration and analysis tool.
The availability of the survey was announced on recappnotes — ReCAPP's email list which at the time of the survey was comprised of 3,100 members. As an incentive, the announcement email about the survey informed potential survey respondents that at the close of the survey, 10 respondents would be randomly selected for a $50 gift certificate to ETR's Publishing Catalogue.2 After 49 days, the survey was closed and 322 individuals had completed the survey from the United States and abroad.
A summary of the findings from these completed surveys follows.
The 22 survey questions were divided into four sections. Each of the four sections is briefly described below. The section titles used in the survey will also be used in this paper as a way of organizing a summary of our findings.
The Work You Do: The seven questions in this section were aimed at getting a picture of the type of practitioner using ReCAPP, where he/she works, and his/her years of experience.
The People You Serve: The five questions in this section helped the ReCAPP project team get a better picture of the people being served by survey respondents.
How You are Using ReCAPP: The six questions in this section informed ReCAPP's project team on the most popular columns (sections of the web site) and how the information found on ReCAPP is being utilized.
Some Miscellaneous Questions: The four questions in this section allowed the ReCAPP project team to ask respondents about spirituality and its relationship to adolescent reproductive health (a specific interest of a potential funder), and web accessibility. It also gave respondents an opportunity to make final comments/suggestions.
Note: Some respondents chose more than one category in response to a given question and therefore the percentages reported for each question do not always add up to 100%. For simplicity purposes, only the top few response categories for each question will be reported here. Percentages were rounded off to the nearest whole number.
Section One: The Work You Do
Occupation (see slide #2)
The majority of survey respondents identified as health or sexuality educators (55%), followed by program administrators/managers (26%), health care providers (e.g., nurse, nurse practitioner, physician, physician assistant) (20%), counselor/social worker (13%) and academic/researcher (13%).
Other response categories included: advocate, elementary school teacher, middle school/junior high school teacher, high school teacher, parent and "other." Only 5.2% of respondents identified as any kind of school teacher. This was a disappointing finding given that ReCAPP has always intended to reach this audience.
Explanations for this lower than expected representation may be that perhaps some school teachers who teach health education identified as a "health educator" instead of school teacher, and/or some school teachers may have a difficult time accessing the web because of limited school budgets or time required to stand up and teach classes. In any case, future ReCAPP marketing efforts should take a closer look at ways of bringing ReCAPP to the attention of school teachers.
Work Setting (see slide #3)
The most common work setting of survey respondents was government organizations (27%). Government organizations can be on the local, state or national level and can include public health departments, departments of education/instruction, federal entities like the Centers for Disease Control, etc. In retrospect, this response category would have been more informative if broken down into at least a few subcategories.
After government organizations, respondents reported working in health clinics or Planned Parenthoods (25%), community-based organizations (21%), high schools (13%) and colleges/universities (11%). Given that such a small percentage of respondents identified as school teachers, those respondents selecting high school as their work setting must be working in capacities other than teacher such as a school nurse or counselor.
Years of Experience (see slide #4)
Survey respondents appear to be seasoned professionals with almost a third (32%) reporting having more than 10 years experience of working in "adolescent pregnancy prevention or related field." Practitioners who are relatively new to the field (two years or less) represented 22% of the survey respondents.
Location (see slides #5, #6)
The overwhelming majority of respondents reported working in the North American region (89%). However, modest representation was reported from Central America, South American, the Caribbean, Europe, Africa and Asia. Although no respondents reported working in Australia, Australian users have made a presence on ReCAPP's online discussion forums in the past. Respondents work largely in urban settings (49%), but also in rural (29%), suburban (21%) and very secluded (2%) settings.
Respondents from every state were represented in the survey except: Alaska, Idaho, Kansas, Louisiana, Mississippi, North Dakota and South Dakota. States with the highest representation in the survey included: California, New York, Minnesota, Michigan, Texas, Georgia, Washington, Maryland and North Carolina. Many of the states in the "top nine" are states with large populations (e.g., California, New York and Texas) and it would make sense that they be represented in this group.
Other states in the "top nine" like Minnesota, Georgia and North Carolina have strong adolescent pregnancy prevention state coalitions whose members/associates are likely to be attracted to ReCAPP. Several states with very low or no representation in our survey like Louisiana, Mississippi, Alabama and Arizona also have high (above the national average) adolescent pregnancy rates. Future ReCAPP marketing efforts may want to prioritize practitioners in these states who would likely benefit from ReCAPP services.
Barriers and Challenges (see slide #7)
Survey respondents talked about a wide range of barriers and challenges facing them in their work. Responses to this open-ended question were hand-coded for themes. The most frequently occurring theme found in the responses to this question relate to "financial constraints" (57%) and included issues like dependency on grants, funding available only for abstinence-only education, reduction in donations due to weak economy, and a decrease in funding for prevention efforts on the state level.
Second to financial constraints was a theme centering around politics (35%). Specifically, respondents talked about "the Church's" influence on public health policy and programs, turf issues with school boards and districts, abstinence-only-until-marriage education, denial about adolescent sexual behavior, basing policy on values not science/research, government bureaucracy, and lack of political will.
After politics, respondents talked about the challenge of working with populations from different cultures (22%). For example, one person said that in one culture, there is "acceptance, and in some cases, even encouragement of teen pregnancy." Others talked about some cultures not "believing" in any type of family life education. This belief can cause conflict amongst parents and teens who are from different generations and/or have different levels of acculturation.
Some mentioned the need for cultural competency training and materials to better serve Hispanic, African American, Asian and special education youth. Other major themes found in the responses to this question include: "staffing problems," "apathy in the community," "working with parents," and "reaching youth in rural areas."
Section Two: The People You Serve
Sex of Clients
The majority of the clients served by respondents are female (63%).
Age of Clients
Respondents serve clients of various ages with the majority being 15 to 17 years old (34%), followed by clients who are 20 years old or older (30%) (may include parents or professionals), 12 to 14 years old (25%), 18 to 19 years old (20%) and 11 years old or younger (13%).
Ethnicity of Clients
In retrospect, the quality of this question was poor, and it would be difficult to give an accurate estimate of the percentage of Hispanic, African American, Native American, Pacific Islander and White individuals served by survey respondents. We are able to say that 90% of the respondents serve some white individuals (on average 47%), 82% serve some Black/African Americans (on average 31%), 81% serve some Hispanic/ Latino/Chicanos (on average 22%), 59% serve some Asians (on average 13%), 50% serve some Native Americans (on average 40%), 38% serve some Pacific Islanders (on average 30%), and 37% serve some individuals who do not fit into these categories.
Sexual Orientation of Clients
Of the 239 clients who responded to this question, 43 (18%) reported that the sexual orientation of their clients was unknown, and 28 (12%) reported that 0% of their clients were gay, lesbian, bisexual or transgendered. Of the remaining responses, the average percentage of GLBT clients served was 13% with responses ranging from 1% to 100%.
Difficult to Reach Populations (see slide #8)
There were several populations that survey respondents reported wanting to reach but have difficulty in doing so. These populations included: Hispanic populations (12%), males (8%), parents (8%), youth in rural areas (6%) and GLBT youth (6%).
Section Three: How Are You Using ReCAPP
How ReCAPP Information is Used (see slide #9)
The information found on the ReCAPP web site is used by survey respondents in a variety of ways. Almost 55% of respondents said they use ReCAPP to teach youth classes/workshops. Respondents also use ReCAPP for professional development (49%), program planning and evaluation (41%), training other professionals (36%), research (31%), and one-on-one education/counseling.
Most Useful Columns (see slide #10)
Respondents were asked to indicate the three most useful ReCAPP columns. The "Skills for Educators" column was most popular with 59% of respondents choosing it. After the "Skills for Educators" column, "Current Research" (48%), "Evidence-based Programs (39%), "Skills for Youth" (36%) and "Learning Activity" (36%) columns followed in popularity. The columns that were selected least include the "Calendar" column (3.4%) and the "Forum Summary" column (2.3%). A significant amount of time and resources are spent on summarizing the proceedings from each online forum discussion. Given its low rank in this question, the ReCAPP project team may want to consider using these resources for other work that is more likely to be used/appreciated by users.
Future Topics (see slide #11)
Respondents were asked to list three topics that they would like to see ReCAPP explore in the future. Answers to this open-ended question were hand coded for themes. The most frequently appearing theme found in the responses to this question relates to "working with parents" (16%). Specifically, there were requests about how to engage/recruit parents, parent education activities, parent-child connectedness, and teen parenting. After "working with parents," "culture" was found as a prevalent theme (14%). Within this theme, we found requests for tips on working more effectively with African American, Hispanic, American Indian, recently immigrated youth, and GLBT youth. In addition, requests for working around racism issues were found. The issue of "violence" was the third most prevalent theme found in response to this question. Issues such as rape, sexual coercion and its connection to pregnancy, effects of domestic violence on youth, dating violence and suicide were mentioned in responses. Other major themes found in response to this question include HIV prevention education, working with males, addressing the influence of the Internet on teen sexual behavior, and alcohol and drug use.
Time Using ReCAPP (see slide #12)
Respondents to this survey include those who are relatively new to the site and those who have been using it for years. The majority of the respondents reported using ReCAPP for 1-2 years (40%), while 26% reported using it for less than a year, 34% reported using it 3-4 years and 11% reported using it for more than 4 years.
Frequency of Using ReCAPP (see slide #13)
The majority of the individuals (43%) responding to this survey use ReCAPP "often — a few times a quarter", 39% of respondents use ReCAPP "occasionally — a few times a year," and 7% use it "rarely — a few times ever."
Sharing ReCAPP Information with Others
On a monthly basis, the respondents to this survey forward or share ReCAPP information on average (mean) with 11 colleagues. It is important to note that the range for the responses to this question spanned from 0 to 600. The median number of colleagues with whom respondents share information was only three. So although recappnotes (ReCAPP's mailing list) is currently at 3,200 members, we can safely say that ReCAPP information reaches more than this number on a monthly basis.
Section Four: Some Miscellaneous Questions
Web Accessibility (see slide #14)
The overwhelming majority of survey respondents reported that the web is "very accessible — I can get on the web easily at work, pretty much whenever I want" (89%). Seven percent of respondents reported that the web was "somewhat accessible — I can get on the web at work, but it takes significant effort" and 2% reported that that the web was "not accessible — It is difficult for me to obtain web access at work."
Spirituality (see slide #15)
At the time of writing and administering this survey, the ReCAPP project team was developing a proposal to a foundation with an interest in spirituality and its connection to adolescent reproductive health. We took the opportunity presented by this user survey to find out what our users might want to learn with respect to this topic. Of the 322 people responding to the survey, 184 responded to this open-ended question. Responses to this question were hand coded for themes.
The most frequently appearing theme (18%) found in the responses to this question relates to an "overview of major religions and how they interpret sexuality." The second most prevalent theme (14%) related to "engaging clergy" in sexuality education. Interestingly, 11% of respondents reported "not being interested in the topic" or something close to this wording (third most prevalent theme).
Tied for the fourth most common theme were "helping youth define spirituality for themselves" (10%) and "research on spirituality's connection to health outcomes" (10%). Other major themes included "separation of church and state," "definition of religion and spirituality," "how to be non-denominational," "how to bring up topic of spirituality to an agency," and "abstinence."
Survey respondents had many positive things to say about ReCAPP commenting on the relevance of its content, timeliness, ease-of-use/navigation and ease of understanding. Below find a sampling of some of these comments.
"We are being successful and your web site has helped tremendously. I thank you for being a great resource which I can rely on for accurate, up-to-date information."
"I find the online forums very helpful — It's great to hear from people around the country and the world."
"Your work makes our work easier."
"I am about to complete a Master of Health Promotion degree and have been using materials from this web site to complement my reading and practice."
"Thanks for the great ideas and training you've provided me with over the years. I also really appreciate all of the help understanding the BDI Logic Models as well as the easy to understand and useful curricula ideas! I greatly appreciate the expertise of the folks on the other end of the internet connection at ReCAPP."
"I love the layout, design, it is very easy to navigate and find just what you need. Again — THANKS!"
"It's easy to access and use and provides a valuable service."
"I do believe that is a very informative and practical based web site: this makes it unique!"
"We are working with emancipating foster care youth, primarily female. I use information from the web site to provide information on preventing pregnancy as well STDs as these individuals often have issues with sexuality …"
"It is my greatest resource. I have all new employees in adolescent health go through your web site for training and orientation. If anyone asks where I get my information, I refer them to your web site"
"I find it very helpful and recommend it to my nurse colleagues and teachers. Actually, I depend on ReCAPP when I prepare a presentation."
The ReCAPP project team along with other senior staff at ETR Associates have pursued multiple sources of funding for the web site with little success. Over the past year, over 20 foundations with history of giving to adolescent reproductive health projects were contacted with letters of inquiry and follow-up phone calls. Although several foundations expressed an enthusiastic interest in ReCAPP, shrinking endowments due to the depressed economy have left foundations with their hands tied. ReCAPP will continue to explore funding opportunities with foundations and will also explore ways of funding ReCAPP through existing and new larger scale projects related to adolescent reproductive health.
The findings from this survey give the project team direction especially in regard to marketing efforts and content development. Clearly, school teachers are a population of professionals that we can do a better job at reaching. An investigation into the journals, newsletters, web sites and listservs used by school teachers should be identified and promotional pieces on ReCAPP and its services (e.g., online courses, online discussion forums) should be developed and placed in these resources.
We have a clearer understanding of what columns (i.e., "Skills for Educators," "Current Research," "Evidence-based Programs") are most useful to our users, and we will concentrate our efforts on developing these columns. We also have some solid suggestions for future topics (i.e., working with parents, culture, violence) to be explored on ReCAPP. The findings from this survey will also prove quite valuable as we prepare future grant proposals.
Finally, the ReCAPP project team expresses it sincere appreciation to the 322 people who responded to this survey. Their input will help us to more effectively serve our users in the future.
For more information about ReCAPP, please contact Lori Rolleri at firstname.lastname@example.org.
>> Link to Graphic Presentation of Some Findings (a Flash document) >>
For more information about Survey Monkey, visit www.surveymonkey.com/
2 For more information about ETR's Publishing Division, visit ETR Publishing