Climate Assessment Survey FAQ

 Frequently Asked Questions

1.  Why are we conducting this assessment?

   The survey will provide valuable baseline data for developing and implementing a strategic plan for how the university can be more effective in supporting diversity among faculty, staff, and students.

2. Has this survey received Institutional Review Board (IRB) approval?

   Yes, the UMass Boston Institutional Review Board has reviewed and approved the study.

3. Who will be conducting the survey?

   The survey will be conducted by Dr. Susan Rankin, of Rankin & Associates Consulting.

   The Office of Diversity and Inclusion along with the Climate Study Working Group (CSWG) conducted a national search to select a qualified, experienced consultant to assist with the process.

   Rankin & Associates has been selected as the consultant for this project.  Professor Susan “Sue” Rankin is a faculty member in Education Policy Studies and College Student Affairs at The Pennsylvania State University and a Senior Research Associate at the Center for the Study of Higher Education.  She has extensive experience in institutional climate assessment and institutional climate transformation based on data-driven action and strategic planning. Dr. Rankin (with R. Reason, 2008) developed and utilizes the Transformational Tapestry model, a “comprehensive, five-phase, and strategic model of assessment, planning, and intervention. The model is designed to assist campus communities in conducting inclusive assessments of their institutional climate to better understand the challenges facing their respective communities”(Rankin & Reason, 2008)

   click for more info

4. How were the questions developed?

   The Office of Diversity and Inclusion along with the Climate Survey Working Group (CSWG), a cross-section of UMB staff, students, and faculty met regularly with Dr. Rankin to define and choose questions appropriate for our campus.

   The CSWG selected survey questions from the consultant’s tested collection and added questions from surveys in which UMass Boston has participated in order to conduct longitudinal analysis.

   click for more info

5. Why do some questions require additional response choices?

   In a campus climate research survey, it is important for participants to “see” themselves while responding to questions. Along these lines, survey respondents will see a long list of possible choices for many demographic questions.

6. What is the response rate goal, and what limitations exist if that goal is not met?

   A 30% or higher response rate is adequate in order for results to be used with confidence.

   Participation in the survey is completely voluntary, and participants do not have to answer any question except the first positioning question (staff, faculty, student), and can skip questions about which they are uncomfortable.

   A 30% or higher response rate is adequate and results can then be used with confidence.  However, if a total response rate of 30% is not reached, individual populations with response rates at 30% or higher are able to be reported.  For example, if all faculty respond at 20%, generalizations among all faculty will not be reliable.  However, male faculty of color may respond at 40%, which will allow for some generalized themes to emerge for male faculty of color.  Weight may be applied to the analysis based on the variables for which population data are available.  However, the university only collects population data on position (faculty, staff, students), sex, race/ethnicity, and international status so weighting is limited.  In addition, the consultant will also conduct response bias and response fatigue analysis.

   click for more info

7. How is a respondent’s confidentiality protected?

   Confidentiality is highly protected in multiple ways: the survey collects no data that can identify individuals, and the computer-based survey does not record the IP address of the respondent’s computer.

   Confidentiality is vital to the success of campus climate research, particularly as sensitive and personal topics are discussed.  While the survey cannot guarantee complete confidentiality, the consultant and campus data coordinators will take multiple measures to enhance individual confidentiality. Respondent authentication (user sign-on) is utilized only to verify eligibility and will not be associated with survey responses. The consultant will include a notification statement in the introductory section of the survey in order to notify respondents about privacy and data uses.

   In the event of any publication or presentation resulting from the assessment, no personally identifiable information will be shared.  Confidentiality in participating will be maintained to the highest degree permitted by the technology used (e.g., IP addresses will be stripped when the survey is submitted).  No guarantees can be made regarding the interception of data sent via the Internet by any third parties; however, to avoid interception of data, the survey is run on a firewalled web server with forced 256-bit SSL security. In addition, the consultant and university will not report any group data for groups of fewer than 5 individuals, because those “small cell sizes” may be small enough to compromise confidentiality.  Instead, the consultant and university will combine the groups or take other measures to eliminate any potential for demographic information to be identifiable. Additionally, any comments submitted in response to the survey will be separated at the time of submission to the consultant so that they are not attributed to any individual demographic characteristics.  Identifiable information submitted in qualitative comments will be redacted and the university will only receive these redacted comments.

   Data may be used for secondary research purposes, but will be subject to Institutional Review Board (IRB) approval. Researchers that want to use the data must submit an application to the Office of Diversity & Inclusion outlining the scope of the research project and enter into an agreement to guarantee safeguarding of original data confidentiality and the use of an appropriate security protocol.

   click for more info

8. Why is this a population survey and not a sample survey?

   The survey will be administered to all students, faculty and staff at UMass Boston, as a way of maximizing response rates so that we can be inclusive, capturing as many of our campus’ microclimates and subgroups as possible.

   Climate exists in micro-climates, so it is important to maximize participation as well as increase opportunities to reach minority populations. Because the goal of the project is inclusiveness, the sampling technique is not used.

   click for more info

9. What protections are in place for storage of sensitive data, including for future secondary use?

   UMass Boston has worked with Dr. Rankin to develop a research data security description and protocol, with data from online participants submitted to a secure server she hosts in Pennsylvania, and a rigorous approval process required for anyone to gain access to the aggregate data for further research. The introductory section of the survey makes clear that participation in the survey is completely voluntary, respondents need only answer questions they choose, all data use is subject to Institutional Review Board (IRB) approval, and respondent authentication (user sign-on) is used only to verify eligibility but not to code survey responses.

   UMass Boston has worked with the consultant to develop a research data security description and protocol, which includes specific information on data encryption, the handling of personally identifiable information, physical security, and a protocol for handling unlikely breaches of data security.

   The data from online participants will be submitted to a secure server hosted by the consultant.  The survey is run on a firewalled web server with forced 256-bit SSL security and is stored on a SQL database which can only be accessed locally. The server itself may only be accessed using encrypted SSH connections originating from the local network.  Rankin & Associates, Consulting project coordinator, Dr. Susan Rankin will have access to the raw data along with several R&A data analysts. All R&A analysts have CITI (Human Subjects) training and approval and have worked on similar projects for other institutions.  The web server runs with the SE-Linux security extensions (that were developed by the NSA). The server is also in RAID to highly reduce the chance of any data loss due to hardware failure. The server performs a nightly security audit from data acquired via the system logs and notifies the administrators.

   The consultant has conducted over 100 institutional surveys and maintains an aggregate merged database.  The data from the UMass Boston project will be merged with all other existing climate data stored indefinitely on the consultant’s secure server.  No institutional identifiers are included in the full merged data set held by the consultant.  The raw unit-level data with institutional identifiers is kept on the server for 6 months and then destroyed.  The paper and pencil surveys are returned to the consultant directly and kept in a locked file drawer in a locked office.  The consultant destroys the paper and pencil responses after they are merged with the online data.  The consultant will notify Juan Nunez of any breach or suspected breach of data security of the consultant’s server.

   click for more info

10. What will be included in the final report?

    The report will include an executive summary and a narrative report of the findings summarizing main themes, based on statistical and qualitative analysis of all responses.

    The report will include: an executive summary; a report narrative of the findings based on cross tabulations selected by the consultant; frequencies, percentages, means and standard deviations of quantitative data; and content analysis of the textual data.  The reports provide high-level summaries of the findings and will identify themes found in the data.  Generalizations for populations are limited to those groups or subgroups with response rates of at least 30%.  The CSWG will review the draft report and provide feedback to the consultant prior to public release.

    click for more info