FEASIBILITY STANDARDS

The Foundational Principles of Feasibility Standards

The concept of Feasibility Standards is a fundamental pillar within the field of program evaluation, specifically codified by the Joint Committee on Standards for Educational Evaluation (JCSEE). These standards are designed to ensure that an evaluation is conducted in a manner that is realistic, prudent, diplomatic, and frugal. In the complex landscape of psychological and educational assessments, the theoretical elegance of a research design often clashes with the pragmatic realities of the field. Therefore, feasibility standards serve as a vital bridge between abstract methodology and successful execution, ensuring that resources are not wasted on evaluations that are too disruptive or too costly to provide meaningful insights. By adhering to these benchmarks, evaluators can navigate the intricate balance between technical rigor and the practical constraints of real-world environments.

The importance of feasibility cannot be overstated, as an evaluation that is technically sound but practically impossible to implement is of no value to stakeholders. These standards require evaluators to anticipate potential obstacles long before the first data point is collected. This proactive approach involves a comprehensive assessment of the organizational climate, the availability of personnel, and the technical infrastructure required to support the evaluation activities. By prioritizing feasibility, the evaluator demonstrates a commitment to the sustainability of the project and the well-being of the participants involved. This ensures that the evaluation findings are not only accurate but also actionable within the specific context of the program being studied.

Furthermore, the integration of feasibility standards into the evaluation process fosters a culture of transparency and accountability. When evaluators acknowledge the limitations of their reach and the constraints of the environment, they build trust with stakeholders who might otherwise be skeptical of the evaluation’s utility. This transparency allows for a more collaborative approach to problem-solving, as stakeholders and evaluators work together to identify the most efficient and least intrusive ways to gather data. Ultimately, the goal of feasibility standards is to maximize the return on investment for the evaluation, ensuring that every dollar spent and every hour dedicated to the process contributes to a deeper understanding of the program’s effectiveness and areas for improvement.

In the broader context of the Program Evaluation Standards, feasibility is one of four major categories, alongside utility, propriety, and accuracy. While accuracy focuses on the truthfulness of the findings and utility focuses on their usefulness, feasibility focuses on the “how” of the evaluation. It addresses the logistics and the human element of the research process. Without feasibility, the other standards may remain unreachable, as a lack of practical planning can lead to ethical breaches, data inaccuracies, and a total loss of utility. Thus, feasibility acts as the logistical backbone that supports the entire weight of the evaluative inquiry, making it indispensable for professional practitioners in psychology and social sciences.

The Practical Procedures Standard (F1)

The Practical Procedures standard, often designated as F1, mandates that evaluation procedures should be practical and keep disruption to a minimum while needed information is being gathered. This requirement is particularly critical in psychological settings where the presence of an evaluator might alter the behavior of participants—a phenomenon known as the Hawthorne Effect—or interfere with the therapeutic or educational processes under study. A well-designed evaluation must integrate seamlessly into the existing workflow of the organization, ensuring that data collection methods, such as surveys, interviews, or observations, do not place an undue burden on the staff or the subjects. This involves careful timing, the use of non-intrusive instruments, and a clear understanding of the daily operational cycles within the environment being evaluated.

To achieve high standards of practicality, evaluators must engage in meticulous logistical planning. This includes selecting data collection tools that are user-friendly and time-efficient, as well as scheduling activities during periods that do not conflict with essential program operations. For instance, in a school setting, an evaluator must avoid high-stakes testing periods or major school events that would make data collection difficult or stressful for students and teachers. By demonstrating respect for the participants’ time and responsibilities, the evaluator increases the likelihood of high-quality participation and reliable data. Practicality also extends to the back-end of the evaluation, such as the ease of data entry and the accessibility of the analysis software used by the team.

Another aspect of the Practical Procedures standard involves the minimization of administrative burden. Evaluators should strive to use existing data sources whenever possible, such as administrative records or previously collected assessment scores, to reduce the need for new data collection efforts. When new data is required, the instruments used should be as brief as possible while still maintaining validity and reliability. This approach not only respects the participants but also reduces the likelihood of “survey fatigue,” which can lead to low response rates and biased results. A practical evaluator is essentially a minimalist, seeking the most direct and least complicated path to answering the evaluation questions without sacrificing the integrity of the study.

In conclusion, the F1 standard emphasizes that the viability of an evaluation is often determined by the small details of its execution. If a procedure is too cumbersome, it will likely be ignored or implemented poorly, leading to missing data and compromised results. By focusing on practical procedures, evaluators ensure that their work remains grounded in reality. This standard encourages a form of “methodological humility,” where the evaluator recognizes that their needs must be balanced against the ongoing needs of the program and its participants. Successful implementation of F1 ensures that the evaluation is a helpful addition to the program rather than a disruptive intrusion.

Navigating Political Viability (F2)

The Political Viability standard (F2) requires that evaluations be planned and conducted with anticipation of the different positions of various interest groups. This is essential so that their cooperation may be obtained and so that possible attempts by any of these groups to curtail evaluation operations or to bias or misapply the results can be averted or counteracted. In any organizational setting, there are power dynamics and competing interests that can significantly influence the success of an evaluation. Evaluators must possess a high degree of political acumen to identify these stakeholders, understand their motivations, and navigate the potential minefields of organizational politics without compromising their professional independence or the integrity of the evaluation findings.

Political viability often hinges on the ability to secure stakeholder buy-in from the outset of the project. This involves identifying key influencers within the organization and involving them in the planning stages of the evaluation. By giving stakeholders a voice in defining the evaluation questions and the criteria for success, evaluators can foster a sense of ownership and reduce resistance to the process. However, this must be balanced with the need for objectivity. The evaluator must remain vigilant against attempts by powerful stakeholders to steer the evaluation toward a pre-determined conclusion or to suppress unfavorable results. Navigating these pressures requires strong negotiation skills and a clear ethical framework, often guided by the American Evaluation Association (AEA) Guiding Principles.

Moreover, the political viability of an evaluation is closely linked to how the results are communicated and disseminated. Evaluators must be strategic in their reporting, ensuring that the findings are presented in a way that is constructive rather than punitive. This does not mean softening the truth, but rather framing the results in a context that emphasizes program improvement and organizational learning. By highlighting both strengths and weaknesses, and by providing actionable recommendations, the evaluator can help stakeholders see the value of the evaluation as a tool for positive change. This reduces the likelihood that the results will be ignored or used as a political weapon within the organization, thereby protecting the long-term viability of the evaluative process.

In practice, ensuring political viability also means being prepared for conflict resolution. Disagreements are almost inevitable in any evaluation that addresses sensitive issues or resource allocation. The evaluator must act as a neutral facilitator, helping stakeholders navigate their differences and find common ground. This may involve mediating disputes over data interpretation or managing expectations regarding the scope of the evaluation. By maintaining a professional and diplomatic demeanor, the evaluator can preserve the integrity of the process and ensure that the evaluation remains a credible source of information for all parties involved. Ultimately, political viability is about building a sustainable environment where evaluation is seen as a necessary and beneficial part of organizational life.

Economic Efficiency and Cost-Effectiveness (F3)

The Cost-Effectiveness standard (F3) dictates that an evaluation should be efficient and produce information of sufficient value to justify the resources expended. In an era of limited budgets and increasing demands for accountability, evaluators must be exceptionally mindful of the financial and human costs associated with their work. This standard is not merely about finding the cheapest way to conduct an evaluation; rather, it is about maximizing the value of the information gathered relative to the investment made. A cost-effective evaluation is one that provides high-quality, actionable data that leads to improved program outcomes, thereby justifying the expenditure of time, money, and effort required to produce it.

To adhere to the F3 standard, evaluators must conduct a thorough budgetary analysis during the planning phase. This involves estimating the costs of personnel, materials, travel, data collection, and analysis, as well as the indirect costs associated with the participants’ and staff’s time. It also requires the evaluator to consider the opportunity costs—what other activities could have been funded with the resources dedicated to the evaluation? By being transparent about these costs and discussing them openly with stakeholders, evaluators can ensure that the scope of the evaluation is aligned with the available resources. This prevents situations where an evaluation is started but cannot be completed due to a lack of funding, which is a significant violation of feasibility standards.

Improving economic efficiency often involves the use of innovative methodologies and technologies. For example, the use of online survey platforms can significantly reduce the costs of data collection and entry compared to traditional paper-and-pencil methods. Similarly, the application of secondary data analysis can provide valuable insights at a fraction of the cost of new data collection. However, the evaluator must ensure that these cost-saving measures do not compromise the accuracy or utility of the evaluation. The goal is to find the “sweet spot” where the evaluation is rigorous enough to be credible but lean enough to be sustainable. This requires a deep understanding of evaluation design and the ability to make strategic trade-offs between different methodological approaches.

Finally, the concept of cost-effectiveness extends to the long-term impact of the evaluation. An evaluation that identifies significant program inefficiencies or leads to the discontinuation of an ineffective intervention can save the organization substantial amounts of money in the long run. In this sense, a well-conducted evaluation is an investment rather than an expense. Evaluators should strive to document the ways in which their work has contributed to financial savings or improved resource allocation, as this provides powerful evidence of the evaluation’s cost-effectiveness. By demonstrating a clear link between evaluation activities and organizational benefits, evaluators can secure ongoing support for their work and reinforce the importance of feasibility as a core professional standard.

The Interplay Between Feasibility and Accuracy

While Feasibility Standards focus on the practical implementation of an evaluation, they are intrinsically linked to the Accuracy Standards. There is a common misconception that feasibility and accuracy are in opposition—that one must be sacrificed for the other. In reality, a lack of feasibility often leads directly to inaccuracies. For example, if a data collection procedure is too complex or time-consuming (a failure of feasibility), participants may provide rushed or incomplete responses, leading to measurement error and biased results. Therefore, ensuring that an evaluation is feasible is a prerequisite for ensuring that it is accurate. The evaluator’s task is to design a study that is both practically viable and methodologically sound.

One way that feasibility supports accuracy is through the minimization of attrition and non-response bias. In longitudinal psychological studies, maintaining high levels of participant engagement is essential for the validity of the findings. If the evaluation procedures are too burdensome or culturally insensitive, participants are more likely to drop out of the study, which can severely compromise the representativeness of the sample. By prioritizing practical and respectful procedures, evaluators can maintain higher retention rates, thereby enhancing the accuracy and generalizability of their conclusions. In this context, feasibility acts as a safeguard for the statistical integrity of the evaluation design.

Furthermore, the Political Viability standard plays a crucial role in ensuring the accuracy of an evaluation. When stakeholders feel threatened by an evaluation, they may intentionally withhold information or provide misleading data. Conversely, when an evaluator successfully navigates the political landscape and builds trust with stakeholders, they are more likely to gain access to the “real” story behind the program’s operations. This access to high-quality, honest information is essential for accurate analysis and reporting. Thus, the diplomatic and interpersonal skills required by the feasibility standards are directly instrumental in achieving the technical goals of the accuracy standards.

Ultimately, the relationship between feasibility and accuracy is symbiotic. An accurate evaluation that cannot be conducted is useless, and a feasible evaluation that produces inaccurate results is misleading. Professional evaluators must constantly weigh these two sets of standards against each other, making informed decisions that maximize the overall quality of the evaluation. This may involve choosing a slightly less sophisticated statistical model that can be easily explained to stakeholders, or opting for a smaller but more manageable sample size to ensure data quality. By acknowledging the interdependence of these standards, evaluators can produce work that is both rigorous and relevant to the needs of the program.

Methodological Flexibility and Adaptive Design

In the real-world application of Feasibility Standards, evaluators must embrace methodological flexibility. Unlike laboratory research, where variables can be strictly controlled, program evaluation takes place in dynamic and often unpredictable environments. A design that seems feasible on paper may encounter unforeseen obstacles once implemented, such as sudden budget cuts, staff turnover, or changes in program leadership. To maintain feasibility, the evaluator must be prepared to adapt their approach without compromising the core objectives of the study. This requires a mindset of continuous improvement and a willingness to pivot when necessary to keep the evaluation on track.

Adaptive design in evaluation involves building feedback loops into the process so that feasibility can be assessed in real-time. For example, an evaluator might conduct a pilot study of a new survey instrument to identify any practical issues before rolling it out to the full sample. If the pilot reveals that the survey is too long or that some questions are being misinterpreted, the evaluator can make adjustments immediately. This iterative approach ensures that the evaluation remains grounded in the practical realities of the field and that potential problems are addressed before they can derail the entire project. It also demonstrates a commitment to responsiveness and professional excellence.

Flexibility also extends to the selection of methods. If a planned randomized controlled trial (RCT) becomes unfeasible due to ethical concerns or logistical barriers, the evaluator must be prepared to transition to a quasi-experimental or case study design. While this may involve a trade-off in terms of internal validity, it ensures that the evaluation can still provide valuable information that would otherwise be lost. The key is to be transparent about these changes and to document the rationale for the shift in methodology. By being flexible, the evaluator ensures that the evaluation remains a viable and useful tool for the organization, even in the face of significant challenges.

Moreover, methodological flexibility is essential for ensuring cultural feasibility. In psychological and educational research, what is considered practical and appropriate can vary significantly across different cultural and linguistic groups. An evaluator who is rigid in their approach may find that their methods are culturally insensitive or irrelevant to the participants, leading to poor data quality and ethical concerns. By being flexible and working closely with local stakeholders, evaluators can adapt their procedures to be culturally congruent, thereby enhancing both the feasibility and the accuracy of the evaluation. This approach not only fulfills the F1 and F2 standards but also aligns with the broader goals of equity and inclusion in evaluation practice.

Challenges in Implementing Feasibility Standards

Despite their importance, the implementation of Feasibility Standards is often fraught with challenges. One of the primary obstacles is the tension between rigor and reality. Academic training often emphasizes the importance of experimental control and sophisticated statistical techniques, which can lead evaluators to design studies that are overly ambitious for the context in which they are operating. When these evaluators transition from the university to the field, they may struggle to reconcile their methodological ideals with the practical constraints of program budgets, staff time, and organizational politics. Overcoming this challenge requires a shift in professional identity from “researcher” to “evaluator,” with an increased focus on the pragmatic needs of stakeholders.

Another significant challenge is the lack of resources dedicated specifically to the evaluation process. In many organizations, evaluation is seen as an “add-on” rather than a core function, and it is often the first thing to be cut when budgets are tight. This creates a perpetual struggle for feasibility, as evaluators are forced to do more with less. Under-resourced evaluations are prone to a host of problems, including small sample sizes, inadequate data collection tools, and insufficient time for thorough analysis. To address this, evaluators must become skilled advocates for their work, demonstrating the value of feasibility and the risks associated with cutting corners in the evaluation process.

The dynamic nature of political environments also poses a constant threat to feasibility. Even a well-planned and politically viable evaluation can be derailed by a change in leadership or a shift in organizational priorities. An evaluator may spend months building trust with a program director, only to have that director replaced by someone who is hostile to evaluation. Navigating these transitions requires resilience and the ability to rebuild relationships quickly. It also highlights the importance of institutionalizing the evaluation process so that it is not dependent on the support of a single individual. By embedding evaluation into the organization’s policies and procedures, evaluators can create a more stable and feasible environment for their work.

Finally, the technological divide can present a barrier to feasibility, particularly in evaluations that involve underserved populations. While digital tools can enhance efficiency, they are only feasible if the participants have access to the necessary hardware and internet connectivity. An evaluator who relies exclusively on online surveys may inadvertently exclude the very people the program is intended to serve, leading to biased results and ethical concerns. To overcome this, evaluators must maintain a diverse toolkit of data collection methods and be prepared to provide technical support or alternative formats to ensure that everyone can participate. This commitment to inclusivity is a vital component of the modern application of feasibility standards.

The Role of Professional Competency in Feasibility

The successful application of Feasibility Standards depends heavily on the professional competencies of the evaluator. Beyond their technical skills in statistics and research design, evaluators must possess strong interpersonal, organizational, and managerial skills. They must be effective communicators who can explain complex concepts to non-technical audiences and negotiate with stakeholders who have competing interests. They must also be skilled project managers who can keep the evaluation on schedule and within budget while managing multiple tasks and competing priorities. These “soft skills” are often just as important as technical expertise in ensuring the success of an evaluation.

One of the key competencies related to feasibility is reflective practice. This involves the evaluator constantly monitoring their own work and the environment in which they are operating to identify potential feasibility issues. A reflective practitioner is quick to notice when a procedure is not working as intended and is proactive in seeking solutions. They are also aware of their own biases and how these might influence their judgment regarding the feasibility of a particular design. By engaging in regular self-assessment and seeking feedback from peers and stakeholders, evaluators can refine their approach and improve the overall quality and viability of their work.

Building trust and rapport with participants and stakeholders is another critical competency. Feasibility is often a matter of human cooperation; if people do not trust the evaluator or the process, they are unlikely to provide the information needed for the study. Evaluators must be able to demonstrate empathy, respect, and cultural sensitivity in all their interactions. This involves listening to the concerns of stakeholders, being transparent about the evaluation’s goals and limitations, and following through on commitments. When participants feel that their voices are heard and their contributions are valued, they are more likely to support the evaluation, making it more feasible and successful.

In addition, evaluators must be knowledgeable about the legal and ethical frameworks that govern their work. Feasibility is not just about what is possible, but also about what is permissible. This includes understanding privacy laws, human subjects protection regulations (such as IRB requirements), and professional codes of conduct. An evaluation that violates these standards is not feasible in the long run, as it can lead to legal liability and damage to the evaluator’s professional reputation. By staying informed about these issues and integrating ethical considerations into every stage of the planning process, evaluators ensure that their work is both viable and responsible. This holistic approach to competency is what defines a truly professional evaluator in the field of psychology and beyond.

Future Directions in Feasibility Standards

As the field of program evaluation continues to evolve, the application of Feasibility Standards is being shaped by new trends and technologies. One of the most significant developments is the increasing use of Big Data and automated analytics. These tools have the potential to make evaluation more feasible by streamlining data collection and analysis, allowing for real-time monitoring of program outcomes. However, they also raise new feasibility challenges, such as the need for specialized technical expertise and the management of large, complex datasets. Evaluators of the future will need to be proficient in these technologies while still maintaining the human-centered focus of the feasibility standards.

Another emerging trend is the globalization of evaluation standards. As more organizations operate across international borders, there is a growing need for feasibility standards that are applicable in diverse socioeconomic and cultural contexts. What is feasible in a high-resource setting like the United States may be completely unworkable in a low-resource setting in the developing world. The next generation of feasibility standards will likely place a greater emphasis on contextual adaptation and the use of locally-driven evaluation models. This will require evaluators to be even more flexible and culturally competent, working in close partnership with local communities to design evaluations that are truly viable and meaningful.

The rise of virtual and remote evaluation is also changing the landscape of feasibility. The COVID-19 pandemic accelerated the adoption of remote data collection methods, such as video interviews and digital observations. While these methods offer significant advantages in terms of cost and efficiency, they also present new challenges related to digital equity and the quality of human interaction. Future feasibility standards will need to address these issues, providing guidance on how to conduct high-quality evaluations in virtual environments while ensuring that the process remains accessible and respectful to all participants. This will involve a careful balancing of the benefits of technology with the ongoing need for personal connection and ethical integrity.

In conclusion, the Feasibility Standards will remain a vital part of the evaluator’s toolkit for the foreseeable future. While the specific methods and technologies used to conduct evaluations will continue to change, the underlying principles of practicality, political viability, and cost-effectiveness will remain constant. By adhering to these standards, evaluators can ensure that their work is not only scientifically rigorous but also practically useful and ethically sound. The ongoing commitment to feasibility is what allows the field of program evaluation to fulfill its promise of providing the evidence needed to improve programs, policies, and the lives of the people they serve. As we look to the future, the integration of feasibility with other core standards will continue to define the highest level of professional practice in psychology and the social sciences.