Features of IDEA Feedback Systems
The IDEA Feedback for Administrators System is designed for online administration. This online system contains the following features:
- Administrator self-assessment of administrative roles and personal characteristics (note that the self-assessment can be omitted to accommodate local policies).
- Rater subgroup results can be provided as an option. This allows an administrator to reflect on any possible differences that specific constituencies may have.
- Email delivery of the Information Form (i.e., self-assessment) to be completed by the administrator being rated.
- Email delivery of the surveys directly to the eligible raters (faculty, professional and support staff, etc.).
- Campus-specified start and end dates of the survey’s online availability.
- Automatic email reminders delivered to those who have not responded. The institution determines the frequency of reminders.
- Ability to preview surveys, which encourages reflection and thought prior to completion.
- One-time restriction on survey submission to ensure confidentiality and authentication of responses.
- Opportunity for administrator to access the Information Form (i.e., self-assessment) multiple times to provide adequate opportunity for self-reflection.
- Confidential submission of surveys directly to IDEA in Manhattan, Kansas (see Statement of Confidentiality in Section V).
- The summary report of results is emailed to the primary contact within 10 working days of the survey end date.
- If multiple administrators are using the Feedback System simultaneously, each set of reports is returned separately to further ensure confidentiality.
- Option to add a maximum of 20 additional questions for survey customization.
Campus Environment for Rating Administrator Performance
Special Notes: Surveys will only be administered as part of an institution’s official feedback process or with the administrator’s consent or at the administrator’s initiation. This must be verified on the Feedback for Administrators Survey Request Form. A minimum of five respondents is required for each Administrator’s Survey
Setting the Stage
IDEA believes it is important to conduct regular and meaningful ratings of administrators at all levels. The IDEA Feedback for Administrators System is designed to be one component in the systematic rating of higher education leaders (e.g., deans, presidents, vice presidents, institutional officers over admissions, student affairs, institutional research and development, and business).
Just as faculty benefit from regular and valid ratings, the IDEA Feedback for Administrators System should provide beneficial information when conducted at regular intervals, such as every two to three years. The timing of the ratings may be dependent upon its purpose. The results may be used to provide formative feedback to the administrator (e.g., areas where the administrator might focus his or her improvement efforts) and should occur earlier in the administrator’s appointment to allow time for improvement. If the rating is to be used for summative feedback (e.g., decisions to retain the administrator), the timing of the process is usually later in the administrator’s appointment.
While IDEA Feedback for Administrators can be one source of evidence in an administrator’s feedback process, other sources should also be included. Examples of other information might include administrative responsibilities, a statement of administrative philosophy, administrative accomplishments, and administrative awards (refer to Seldin and Higgerson, 2002, for additional ideas).
Questions to Consider
There are several questions to consider prior to beginning any of the IDEA Feedback processes. The important questions relate to purpose, timing, number of administrators being rated simultaneously, and number of raters asked to participate.
- How will the results be used – summative or formative feedback?
- Who will receive the report? If the information is to be used by an administrator to improve his or her performance, copies of the report might only be seen by the administrator being rated. If the information is to be used in personnel decisions, then the immediate supervisor of the administrator might also review the report. It is important to make these decisions and communicate the purpose to both the administrator and the raters.
- When will surveys be administered? While campus policies may determine the timing of ratings, it is wise to consider factors that can impact the response rate of the surveys. Such factors might include potential workloads (e.g., registration) and holiday schedules (e.g., Spring break) of the raters. It is strongly recommended that individuals not be asked to rate more than three administrators in one semester. If individuals are asked to complete more than three surveys, the response rates could be lower which, in turn, will decrease reliability of the results.
- Who will be asked to provide ratings? Consideration should be given to the purpose of the ratings (formative or summative), the extent to which individuals have enough experience with the administrator to provide good input, and the culture of your institution. When possible, there should be at least 10 raters for any IDEA Feedback system that is undertaken. The reliability of any survey is questionable when there are fewer than 10 raters responding, and extreme caution should be used when interpreting results. In cases of fewer than 10 raters (e.g., small departments), it is important to rely more heavily on other sources of information as part of the rating process. A minimum of five respondents is required. Reports cannot be generated with fewer than five. In addition, results can be interpreted with more confidence when the response rate is high. For example, you can have more confidence in the interpretation of results when eight surveys represent a 100 percent response rate than if those eight only represent 50 percent of those surveyed.
- Are there important performance expectations that are not addressed in the IDEA instruments? No single survey form can anticipate all the needs of all administrators. The IDEA systems offer the option of adding a maximum of 20 additional questions. These can consist of either open-ended comments, questions using from two to five response options (strongly agree, agree, etc.), or a combination.
Preparing the Campus Environment
It is essential that the administrator, or those requesting the process, prepare the campus prior to the online delivery of the surveys.
The following areas should be addressed in communications to those who are being rated (the administrator) and those who have asked to complete the survey (faculty, staff, etc.).
The purpose of the rating - Will the feedback be used for improvement, decision-making purposes, or both? What additional sources of information will be collected as part of the overall evaluation process?
A description of the survey process - Email delivery of survey link from info@IDEAedu.org. Email reminders until completion of the instrument
Timing of the survey process (start and end dates) - Who will receive a copy of the reports?
Confidentiality of the system (see Section V. Statement of Confidentiality) - This discussion can occur through a personal correspondence from the administrator or coordinator of the process (e.g., either an email or a letter can be sent to all raters), public forums where raters will have access to information (e.g., institution web sites), through announcements at meetings where all of the raters will be in attendance, or a combination of the above.
The following is a sample letter of introduction that may be edited for use on your own campus. The important thing is to be sure that all raters are aware of the process that is occurring, how the information will be used, and that they will be receiving an email requesting their participation.
Introductory Letter Template
Every [insert timing] our campus solicits systematic feedback about our administrators. As part of my [or insert name of administrator] overall rating, I am requesting your input.
Next week you will receive an email from IDEA (info@IDEAedu.org) requesting your participation in a survey to provide feedback to me [or insert name of administrator]. The email will provide you with specific directions.
The Impressions of Administrators survey will be available for you to complete until [insert survey end date]. You can expect email reminders from IDEA until you have completed the survey. The results will be aggregated and provided to me, and [insert who will see the results] in one report. The information will be used primarily to insert purpose, for example: guide future goals and to improve my overall effectiveness as [insert job title]. OR: The information will be used as part of my overall rating to determine continuance in this position.
Since confidentiality is always a concern with this type of process, I have attached a Statement of Confidentiality provided by IDEA. If you have additional questions about the logistics of the process, please contact [insert campus contact name].
As always I value your input and feedback and strongly encourage you to take this opportunity to participate in the survey. The results will be most meaningful if everyone provides input.
Sincerely, [insert name]
Logistics and Checklist
After all pre-implementation discussions and plans have been made, the IDEA Feedback System for Administrators Survey Request Form should be completed and submitted. To obtain the link to the form, email info@IDEAedu.org.
Before you submit the online Survey Request Form, you will need to have the following decisions made and information available. You must submit all information at one time! You will not be able to save part of the Request Form and finish it later. You may add a row for each administrator to be rated at this time. You should submit separate request forms for surveys to be conducted with different start and end dates.
Step 1: Set Survey Dates
Each administrator will be asked to complete an Information Form (i.e., self-assessment). Please specify the start and end dates for the survey. By default, reminders will be sent every three days until the Information Form is complete. You can change the frequency with which reminders are emailed to non-respondents.
Impressions of Administrators Form
Please specify the start and end dates for the rater survey. Rater surveys can be administered at the same time as the Information Form. By default, reminders will be sent every 3 days to non-respondents. You can also change the frequency with which reminders are emailed to non-respondents.
Step 2: Enter Contact Information
The Primary Contact is the person responsible for initiating the IDEA Feedback process, and who will address general questions and make needed decisions during the process. The Primary Contact must also verify that the survey process is being conducted as part of the formal/required institutional process or that the administrator being rated has consented to or is initiating the process. In addition, the Primary Contact will be the recipient of the report, transcription of comments, and summary of (optional) additional question responses, which will be prepared and emailed within 10 working days following the survey administration end date indicated on the Survey Request Form.
Step 3: Enter Billing Contact Information
The Billing Address is the person to whom an invoice will be sent following the conclusion of the survey.
Step 4: Add Administrator(s) Information
Enter the Administrator’s Name (first and last), Job Title and Unit, and Email address.
- Multiple administrators can be included in one request if surveys will be administered using the same start and end dates. Just “Add Another.” If different start and end dates are needed, please submit a separate request for each set of dates.
- By default, the administrator will be asked to complete the Administrator Information Form (i.e., self-assessment), resulting in a gap analysis that can support deeper reflection. Contact IDEA if you wish to omit the self-assessment.
Step 5 Add Rater Email Addresses
There are a number of options to add the rater email addresses:
- Type one email address per line into the specified text area
- Paste a list of email addresses into the text area (one per line)
- Upload a text file containing a list of email addresses (one per line)
NOTE: We strongly advise that your campus test the email addresses prior to sending them to IDEA. This should assist you in sending addresses without errors and only sending addresses of eligible raters. There is a fee for correcting undeliverable emails (see the Feedback System for Administrators—Fee Schedule).
Sending the introductory announcement to raters via email is a good way to test the email addresses before providing the addresses to IDEA. Emails provided to IDEA will be used solely for the purpose of survey administration.
Step 6: Add Additional Questions (optional)
A maximum of 20 additional questions may be added. These can be either open-ended (comments) or fixed-response questions with 2-5 response options (e.g., 1=Strongly Disagree, 2=Disagree, 3=Agree, 4=Strongly Agree). “Cannot Judge” can be selected as an additional response option.
Note: We strongly advise that you carefully draft and organize the additional questions completing the web-based Survey Request Form. Using the Additional Questions area of the Survey Request Form, you must enter the questions in the order you want them to appear in the survey. If you need assistance with this step, please contact IDEA.
Tip: If you have multiple fixed-response questions using the same response options and instructions, you can add them as a Question Group so that you only need to enter the response options once.
Each Question Group must have a scale and that will appear on the survey. If multiple response scales will be used, add an additional fixed-response question group for each response scale. Copy/paste or type the question in the Question Text box.
If using open-ended questions:
Do NOT add item numbers to your fixed-response or open-ended questions. These questions will be automatically inserted near the end of IDEA’s standard survey questions.
Step 7: Define Rater Sub-groups (optional)
Survey results can be presented for 2-4 rater subgroups you select from the list below. The groups will depend on the specific position of each administrator. Note that there must be a minimum of five responses in a designated subgroup to receive subgroup results. If you do not want subgroup results, please select “omit this question.” If omitted, respondent results will only be shown in the aggregate for the entire group.
- Faculty member (or this can be two groups: Adjunct faculty and Regular faculty)
- Administrator’s staff
- Professional colleague/associate
- Department head/chair
- Governing board member
- Community member
- I choose to omit my response to this question (If Rater Sub-groups are used, this option will always be included for raters to “opt out” of this question)
Step 8: Review and Submit
The Primary Contact indicates that he or she has the authority to request and administer the IDEA Feedback for Administrators Survey(s) with the consent of the administrator(s) listed and Submits the form. Clicking the Submit button provides an opportunity to review the information submitted.
Using the Report
The IDEA Feedback System for Administrators Report has been updated to include new interactive features, a “print report” function, and most importantly, new data and information including a new Executive Summary.
Learn more about the report's navigation and interactive features.
Statement of Confidentiality
One of the critical considerations in the development and administration of IDEA Feedback Systems is ensuring the confidentiality of individual responses. All data are submitted directly to a secure and protected IDEA database.
The system does track who has responded to the survey so reminders can be sent to those who have not. However, when responses are downloaded for processing, no identifying information (email address, name, etc.) is linked to the data. As a result, the responses of specific individuals cannot be identified in our data.
Only select IDEA staff members have access to the online system; individuals from your campus do not have access to the system or data.
The IDEA Feedback systems do allow for respondents to make comments to open-ended questions. Responses to these questions are provided verbatim to the administrator. The survey instructions advise that individuals should avoid comments that could reveal their identities.
Reports are emailed to the person identified as the campus contact on the initial request form. To provide further levels of confidentiality, demographic subgroup results are only reported if there are at least five individuals responding from that category.
Samples of the reports that show how the information is summarized can be found here.
Abby, C. L. (1995). The president’s journal: Issues and ideals in the community college. Bolton, MA: Anker.
Bogue, E. G. (1994). Leadership by design: Strengthening integrity in higher education. San Francisco, CA: Jossey-Bass.
Cooper, T. L. (1998). The responsible administrator: An approach to ethics for the administrative role. San Francisco, CA: Jossey-Bass.
Croteau, J.D., & Smith, Z.A. (2012). Making the case for leadership: Profiles of chief advancement officers in higher education. Lanham, Maryland: Rowman & Littlefield Publishers, Inc.
Higgerson, M.L., & Joyce, T.A. (2007). Effective leadership communication: A guide for department chairs and deans for managing difficult situations and people. Bolton, MA: Anker.
Kouzes, J. M., & Posner, B. Z. (1998). Encouraging the heart: A leader’s guide to rewarding and recognizing others. San Francisco, CA: Jossey-Bass.
Kouzes, J. M., & Posner, B. Z. (2000). The five practices of exemplary leadership: When leaders are at their best. San Francisco, CA: Jossey-Bass.
McDade, S. (1989). On assuming a college or university presidency: Lessons and advice from the field. Washington DC: American Association of Higher Education.
Pierce, S.R. (2012). On being presidential: A guide for college and university leaders. San Francisco, CA: Jossey-Bass.
Seldin, P. (1988). Evaluating and developing administrative performance: A practical guide for academic leaders. San Francisco, CA: Jossey-Bass.
Seldin, P., & Higgerson, M. L. (2002). The administrative portfolio: A practical guide to improved administrative performance and personnel decisions. Bolton, MA: Anker.
Vineyard, E. E. (1993). The pragmatic presidency: Effective leadership in the two-year college. Bolton, MA: Anker.
Wergin, J.F. (Ed.) (2007). Leadership in place: How academic professionals can find their leadership voice. Bolton, MA: Anker.
Wilcox, J. R., & Ebbs, S. L. (1992). The leadership compass: Values and ethics in higher education. San Francisco, CA: Jossey-Bass.
Wolverton, M., Bower, B.L., & Hyle, A.E. (2009). Women at the top: What women university and college presidents say about effective leadership. Sterling, Virgina: Stylus.