AI Decision-Making in K-12 Education: Ethical Implications and Educational Concerns

The Growing Use of AI in K-12 Education

Artificial intelligence (AI) and data-driven systems are becoming increasingly integrated into K-12 education. School districts now use AI to assist with student placement, intervention strategies, academic monitoring, behavioral tracking, and resource allocation. While these systems are often promoted as efficient and objective, growing concerns exist regarding bias, transparency, accountability, privacy, and overreliance on quantitative data when making decisions that directly affect students’ educational experiences and futures. 

AI and Student Placement Decisions

In K-12 settings, AI systems are increasingly being used to support decisions regarding special education evaluations, Individualized Education Programs (IEPs), Response to Intervention (RTI), Multi-Tiered Systems of Support (MTSS), basic skills placement, gifted and talented identification, academic intervention groups, and behavioral monitoring. These systems often analyze standardized test scores, attendance records, assignment completion, behavioral data, demographic information, and socioeconomic indicators to identify patterns and generate recommendations.

However, many educators and researchers warn that these systems can unintentionally reinforce existing inequities because the data being analyzed are often tied to broader systemic issues such as poverty, unequal access to resources, language barriers, disability identification disparities, and historical bias within educational systems. 

Ethical Concerns Surrounding IEPs and Special Education

The ethical implications become especially significant when AI systems are involved in high-stakes decisions concerning IEPs, special education eligibility, accommodations, and placement into remedial or intervention-based programs. These decisions directly affect students’ legal protections, educational opportunities, classroom environments, and long-term academic trajectories.

Federal laws such as Family Educational Rights and Privacy Act (FERPA)¹ and Individuals with Disabilities Education Act (IDEA)² require individualized consideration and professional judgment when determining educational services for students with disabilities. Critics argue that overreliance on AI-generated recommendations risk reducing highly individualized student needs into simplified data categories that fail to capture the full picture of a student’s growth, potential, and lived experiences.

The Limitations of Data-Driven Systems

AI systems struggle to account for non-quantifiable aspects of learning and development. Characteristics such as resilience, motivation, emotional well-being, creativity, effort, family circumstances, and learning styles cannot be fully measured through algorithms alone.

A student experiencing trauma, inconsistent access to technology, language acquisition challenges, or temporary hardships may be incorrectly flagged as “at risk” by a system that lacks the contextual understanding educators gain through direct interaction and observation. 

The Impact of Educational Labeling

Another concern involves how AI-generated labels may shape educator perceptions and expectations. Labels such as “at risk,” “below benchmark,” or “behavioral concern” can unintentionally influence how students are viewed and supported within educational settings.

Research has consistently shown that educational labeling and lowered expectations can negatively impact student confidence, participation, opportunity, and academic identity. When these labels are generated through AI systems without transparency or strong human oversight, bias can become embedded under the appearance of objectivity.

Student Data Collection and Consent Concerns

Without clear and informed consent, the collection and analysis of student behavioral data raises serious ethical concerns regarding privacy, transparency, trust, and student autonomy. Even when the stated goal is “student success,” students and families may feel monitored rather than supported when they are unaware of how their information is being collected, analyzed, stored, or used to make educational decisions.

Data such as login activity, attendance patterns, classroom behavior, assignment completion, online participation, disciplinary records, and engagement metrics can create detailed behavioral profiles of students without their full understanding or consent.

When schools rely heavily on these systems, students may begin to feel surveilled instead of encouraged, which can negatively affect participation, trust, and the overall learning environment. Additionally, AI-generated “risk” labels or behavioral indicators may unintentionally influence how educators, counselors, and administrators perceive students before meaningful interaction occurs.

Students who are experiencing external challenges such as poverty, trauma, disability-related needs, mental health struggles, family responsibilities, or limited access to technology may be inaccurately categorized by the system, reinforcing existing inequities rather than addressing them.

Transparency, Privacy, and Family Trust

The lack of transparency surrounding behavioral data collection can weaken relationships between schools and families. Parents and guardians may not fully understand what information is being gathered, how long the data is stored, who has access to them, or how those data influence placement, intervention, disciplinary, or academic decisions.

This becomes especially concerning in K-12 education, where students are minors and legal protections such as FERPA and IDEA require careful handling of educational records and individualized decision-making.

Educational leaders therefore have an ethical responsibility to ensure that informed consent, transparency, human oversight, and data privacy protections remain central whenever AI systems are used to analyze student behavior. While data can help identify students who may need additional support, educational decisions should never rely solely on algorithmic outputs without meaningful human review, contextual understanding, and opportunities for families and students to question or appeal to those determinations.

The Importance of Human Oversight

Across K-12 education, a recurring concern is that AI systems are not inherently neutral. Because algorithms are trained using historical and institutional data, they can replicate and reinforce existing inequities already present within educational systems.

Organizations such as UNESCO continue to emphasize that human oversight must remain central in AI-supported educational decision-making. While AI can assist with identifying patterns and organizing information, it cannot fully account for human growth, context, lived experiences, relationships, or potential.

Educational leaders therefore face increasing pressure to implement ethical safeguards, maintain accountability, protect privacy, and ensure that AI functions as a supportive educational tool rather than a replacement for human judgment and individualized decision-making. 

Side Note: Professional Perspective and Experience

As an eLearning instructional designer, I utilize AI systems regularly throughout my workflow for tasks such as brainstorming, organizing content, supporting accessibility efforts, assisting with course development, and improving efficiency in online learning environments. My work involves collaborating with subject matter experts, designing complex college-level eLearning courses, developing instructional materials, ensuring accessibility compliance, and integrating learning technologies into online education.

I also have training and experience in instructional design methodologies, accessibility standards such as Web Content Accessibility Guidelines (WCAG) and Universal Design for Learning (UDL), Quality Matters (QM) standards, Learning Management System (LMS) platforms, educational technologies, project management, and instructional design models such as Analysis, Design, Development, Implementation, and Evaluation (ADDIE) and Backward Design, all of which directly influence how AI tools are evaluated and implemented within course design.

Because of my professional responsibilities, training, and doctoral-level research, I have seen firsthand that while AI can be an extremely valuable support tool, the output still requires substantial human insight, oversight, and auditing to ensure the information generated is accurate, accessible, pedagogically appropriate, and aligned with course outcomes and institutional standards.

In practice, AI can improve productivity and streamline certain processes, but it cannot replace professional judgment, instructional expertise, or quality assurance. AI-generated materials still need to be reviewed by instructional designers, faculty, and subject matter experts to verify accuracy, identify misinformation, ensure accessibility compliance, and determine whether the content truly meets the needs of diverse learners.

---

ANNOTATIONS

¹ FERPA stands for the Family Educational Rights and Privacy Act. It is a federal law that protects the privacy of student education records and gives parents and eligible students certain rights regarding access to educational information.

² IDEA stands for the Individuals with Disabilities Education Act. It is a federal law that ensures students with disabilities are provided with a free appropriate public education (FAPE) tailored to their individual needs through services such as Individualized Education Programs (IEPs).

---

About the Author

Carly Kutsup is an experienced educator, instructional designer, curriculum developer, and educational leader with nearly two decades of experience spanning K-12 education, career and technical education (CTE), online learning, instructional technology, and higher education as well as course design.

At the high school level, Ms. Kutsup taught Television Production, Broadcast Journalism, Videography, Digital Photography, and Studio Production. She developed and managed fully operational media and television production programs while aligning curriculum with New Jersey state standards, Bloom’s Taxonomy, and multiple instructional design frameworks. Ms. Kutsup also taught internationally through VIPKid and participated in an educational exchange program through a New Jersey high school, further expanding her experience working with diverse learners and multicultural educational environments.

At the higher education level, Ms. Kutsup taught Video Location Production, Digital Imagery, and Student Success Seminar courses. She continues to teach continuing education courses at Bergen Community College, where she remains actively involved in adult learning and workforce-focused education.

Throughout her career, Ms. Kutsup has written and developed curriculum for three different school districts, designing instructional materials and coursework that support diverse learners, state standards, accessibility requirements, and career readiness initiatives. Her curriculum work has focused on creating engaging, standards-aligned learning experiences while integrating instructional technology, project-based learning, and real-world applications into both traditional and online educational environments.

Currently working in eLearning and instructional design, Ms. Kutsup specializes in the development of accessible and engaging online learning experiences for higher education. She collaborates closely with faculty and subject matter experts to design complex college-level courses while integrating instructional technologies, accessibility standards, and learning management systems into course development. Her expertise includes accessibility compliance, Universal Design for Learning (UDL), Web Content Accessibility Guidelines (WCAG), Quality Matters (QM) standards, Learning Management System (LMS) integration, course quality assurance, project management, and instructional design methodologies such as the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) model, Backward Design, and learner-centered course development practices. She also has experience working with emerging educational technologies and the ethical implementation of artificial intelligence in education.

Ms. Kutsup is currently pursuing a Doctor of Education (Ed.D.) degree in Leadership and Innovation through Purdue Global, which is part of the broader Purdue University system. Her doctoral studies focus on educational leadership, ethics, accountability, innovation, instructional design, accessibility, and the evolving role of technology and AI within education. Through her doctoral work, Ms. Kutsup continues to research how leadership decisions, educational systems, instructional technologies, and data-driven practices impact students, educators, accessibility, and learning outcomes across both K-12 and higher education environments.

In addition to her instructional design and educational leadership experience, Ms. Kutsup has extensive experience working with accessibility initiatives, faculty training, course development policies, and online learning support systems. She has also worked on curriculum development and eLearning initiatives for institutions including community colleges throughout New Jersey.

Drawing from both classroom and instructional design experience, Ms. Kutsup advocates for ethical, student-centered approaches to educational technology. She believes that while AI and data-driven systems can serve as valuable tools within education, human insight, professional judgment, transparency, and accountability must remain central to all educational decision-making processes.