Training in this program is open to all graduate students whose native first language is not English and those who are currently serving as TAs or will hold teaching assistant duties during Fall 2009 semester. Selection to to the program is based on the immediacy of the TA assignment and evidence of need, as determined by a 30-minute ITAP Language Assessment (ILA).
Course Number/ID:
EDUC 5780
Course Length (number of weeks):
15
Course Delivery Mode:
Hybrid/Blended (some face-to-face and some online interactions)
Average Number of Enrolled Students:
Between 30 and 60 students
Course Level:
College/University
Course Contributors:
Theresa Pettit, Faculty/Project Director
Clare van den Blink, Project Director
Kimberly Kenyon, Faculty/Project Director
Stew Marklel, Faculty
Stephanie Hanson, Faculty
Jody Gabler, Faculty
Nina Loney, Faculty
Pamela Pollock, Faculty
Eric Machan Howd, Admin/Inst.Design
Course Development:
The EDUC5780 course is designed to include a range of student activities and assignments to meet the educational objectives. Students record audio assignments each week, and video recording of teaching exercises. A key challenge for the course is how to easily submit this large collection of assignments, and have a way for faculty to provide feedback for each assignment. The course is designed with a range of assessment strategies from face-to-face feedback with instructors, online assessment of submitted audio & video assignments, and peer review. Students receive weekly feedback on their assignments and progress in meeting the course goals.
Another requirement of the course is to have students share their video with other students for peer review. Since the program consists of a series of courses, student assignments are retained and reviewed across several courses and several semesters. To meet these challenges, the Sakai OSP tool was selected. The portfolio features were critical to collecting and assessing student work. It keeps a record of student progress, and whether students are meeting objectives. Student artifacts include a variety of digital media files: audio files, MP3, WMA, video, Adobe pdfs and MS Office files. Using Sakai’s portfolio CLE allows students to use an electronic portfolio to collect artifacts and assists instructors in the creation of student-centered tasks. It provides participants with a portfolio of work that allows both instructor and students to monitor learner progress.
Since OSP is part of the Sakai learning system, the courses were able to seamlessly post assignments that were then submitted through the portfolio tool. A portfolio tool alone was not sufficient in meeting the needs of the courses.
A preliminary evaluation of International Teaching Assistant (ITA) students who participated in a pilot study affirmed that ITA students were using technology to enhance their language development, and that they believed that technology is an effective tool for improving language. Learner engagement and motivation are two important factors in an ITAs success. Offering ITAs creative ways to engage with digital media was at the forefront of the creation of the ITAP learning system. A small pilot study about the use of Sakai, found that Sakai CLE was a more robust system than other technologies. As a result it was implemented for courses in the program. A post pilot study evaluation tool was then used to assess the current pilot program. The focus of the evaluation was to examine the impact of this instructional design module on student's perception of their language gains. A method for comparing the oral proficiency levels of ITAs was built into the final analysis. Qualitative data was compared with quantitative findings to better inform CTE-ITAP's future initiatives.
The first phase in developing this course (and those across the program) in Sakai OSP was to define the learning outcomes and assessments within the curriculum in order to determine the design of the portfolio matrix. Much of this work had already been accomplished in the design of the courses themselves, so the focus of building the matrices was to settle on a consistent design of all four matrices, across all four courses in the program. Once that was determined, the last phase of course development was to adjust the courses and matrices within Sakai, according to the pedagogical principles and requirements of the program.
Course Delivery:
EDUC5780 is the first course that students in the ITAP must complete in the four-course program. EDUC5780, as well as the other three courses in the program, are delivered as ‘hybrid’, or ‘blended’, courses through Sakai’s portfolio CLE.
Incoming TAs whose first language is not English, and who are tapped to hold TA duties in an upcoming semester, are scheduled for a screening interview (ILA). Subsequently, they are informed of the program's purpose and assessment process. The screening of English language skills provides a critical juncture for the program. The level of spoken English, as well as listening comprehension skills, need to meet established criteria to assume TA responsibilities. The program assumes that if an ITA passes the initial language screening, then the ITA’s communication skills will be “successful” in the classroom. If an ITA passes the screening, then she/he can proceed as a TA.
If a student is required to take classes wirth the ITAP, he or she enrolls in EDUC 5780. The student is required to maintain of portfolio of all work: recordings of audio journals, oral and written reflective assignments, and video recordings of teaching practices/sessions. All of these artifacts are submitted to the Sakai portfolio matrix and commented upon by the instructor; the student also reflects upon their artifacts/learning throughout the course via the CLE and in small-group, face-face consultations.
At the end of the EDUC 5780 course, the ITA's level of language proficiency is re-assessed by an ITAP instructor. When ITAs have been determined to have “passed”, meaning they have successfully been able to communicate an Advanced Low level on the ILA, they are no longer required to participate in the program. If the ILA result is below the Advanced Low level, the ITA will be required to enroll in EDUC 5790. If ITAs reach the Intermediate High on the ILA (one level below Advanced Low), they can hold TA duties concurrent with EDUC 5790.
The method of course delivery is not only strongly ‘learning-centered’, but also heavily ‘assessment-centered’. The process of submitting ‘artifacts’ to a portfolio matrix, receiving feedback/assessment on those artifacts, supports research that shows that assessment-centered learning environments offer many opportunities for feedback and revision, and tie directly into the student’s learning goals. The strength of ITADP’s innovative design and use of the Sakai OSP supports that claim.
The final outcome of the program assumes that successful communication skills in a classroom will lead to an improvement in undergraduate instruction.
Communication & Collaboration Self-Assessment:
Excellent
Communication & Collaboration Evidence:
Communication and collaboration in this course are one of the most important aspects of the innovation. Students have a lot of communication/collaboration with their instructors regarding the evidence that they submit to their portfolio matrix. Students regularly meet with instructors, both in-class and through small-group conferences, to reflect upon their learning and their evidence. The frequency of adding learning artifacts to the portfolio and receiving feedback on those artifacts instills a large amount of assessment, regular assessment, in this course, so learner input and reflection are highly valued in the pedagogy behind the course design. Community building takes place outside and around the course, through the ITADP program itself. The power of this use of the Sakai portfolio CLE lies in the constant reflection, self-assessment and instructor assessment that students engage in on a regular basis.
Learning Material Self-Assessment:
Excellent
Learning Material Evidence:
Our self-assessment of the learning material in this course (andprogram) has found that students clearly understand the components,structure and roles of the Sakai portfolio CLE.
Students in the program reported that course interactions were easy,and that the assignments for completing their portfolio matrix wereclear and concise. The ‘hybrid’ model of delivering the learning(in-class and Sakai portfolio CLE) also gave the student moreopportunities to reflect upon their evidence and learning throughoutthe semester.
Learning Outcomes & Assessment Self-Assessment:
Excellent
Learning Outcomes & Assessment Evidence:
The overall goal of this project and, in part , the goal for this course, was to develop and implement a working prototype of a “Portfolio Learning System” for the International Teaching Assistant Development Program (ITADP).
Our self-assessment reiterated that the mission of this program is to enable ITAs to enter the Cornell classroom as confident instructors who can communicate their domain expertise in an interactive, student-led environment. Curriculum in four courses addresses instructional strategies as well as general language proficiency. In addition to coursework, students attend weekly small group tutorial sessions during which instructors work with participants to create an individualized plan for language development. The prototype was successfully created using Sakai OSP.
We found that the new portfolios enable ITADP students to create, store, and manage audio and video recordings for their assignments in all four courses ITA courses. If students are enrolled for four semesters, even with different instructors along the way, they can maintain the same portfolio throughout the entire period. As portfolios grow in size and quality, students gain evidence to document their progress in spoken English. The same goes for documentation of their teaching abilities. The system also enables instructors and program administrators to review submitted materials. Students have the ability to hone their critical thinking/judgment and problem solving skills through multiple means. Also, students are able to receive individual feedback on their learning progress through multiple assessment opportunities in the course. These assessments provide feedback to help students reach target levels with consideration of individual student's personal styles and needs.
At present, all sections and all courses with ITADP are using the online space. It has been the experience of program administrators and instructors that, compared to the “old way” of handling audio files through conventional e-mail, Sakai has “distinct advantages” and is more streamlined. No longer does the program have to concern itself with managing a large number of files, which now reside on an external server. In addition, shared artifacts eliminate the complications of losing files from individuals computers. The impact on the program’s workflow is great in that this course, and others in the program, reduce a lot of the administrative overhead so that the focus of the teaching and learning can be more about the ‘learning’ (teaching, student interaction, reflection, etc.) and less about the logistics of storing files, evidence, etc.
Course Look & Feel, Web Usability Self-Assessment:
Effective
Course Look & Feel, Web Usability Evidence:
The Sakai OSP course is designed to be clear and concise. We’ve strived to make the look/feel as simple and straightforward as possible. Students are only required to interact with the assignment tool and matrix (the main tools) to do their own work. The design enhances the presentation and communication of key information/content throughout the course. The design of EDUC 5780 mirrors the design of the other three courses in the program; this consistency enhances student learning throughout the ITADP program. Students also have the opportunity to build their own on-line portfolio at different points in the program which supports self-reflection and self-assessment.
When teaching staff were asked to elaborate on the strengths and weaknesses of Sakai, similar comments about its strength arose. Strengths included Sakai’s ability to track progress, its ability to store work over an entire semester (or longer) and its capability to “keep materials all in one place for easy viewing.” Additional comments included Sakai’s progressive feel and noted improvement over conventional e-mail. The instructor self-evaluation portion of this project/course found that instructors were generally satisfied with its innovative nature and especially liked the way Sakai was more accommodating to larger file sizes.
Additionally, when ITADP students were asked if they were satisfied with Sakai, a combined 62% of respondents replied "Agree" or "Strongly Agree". In the strength's section (open text field) of the online survey, 11 out of 15 students mentioned Sakai's effective matrix design and clear organization of information. Other strengths noted were ease of use, accessibility of files and ability to upload a variety of file types. In the weaknesses section (open text field), less than 1/3 felt the interface was complicated and the menu somewhat confusing.
Learner Support Self-Assessment:
Excellent
Learner Support Evidence:
Support for any innovative program is crucial. The ITAP has addressed this fact through a variety of support resources: a student-orientation session, a ticket-based software help system, a faculty training session and a support wiki. The Sakai OSP courses in this program contain a variety of resources and contact information and support documentation is created ‘on-the-fly’ when needed on a support wiki. EDUC 5780 users report great satisfaction with learner and instructional support.
The student-orientation session introduces students to the CLE as well as to the particular requirements for using the CLE throughout the semester. Courses also contain a web-link to a support wiki; this wiki contains step-step instructions on how to submit evidence to the matrix, trouble-shooting tips/tricks as well as extensive information about the OSP. The wiki is regularly updated with new information regarding the CLE. The ‘ticket-based’ help system allows us to track document fixes for future reference. In most cases, solutions to issues are also cross-posted to the support wiki for student/faculty reference. Students can e-mail support requests to this system at any time, and requests are usually responded to within a day.
Teaching Innovation:
Students in the ITA program are charged with making their teaching better. Reflective processes and features of this course enable students to attain that goal.
As illustrated in the ‘Learning Outcomes Self-Assessment’, we found that the new portfolios enable ITADP students to create, store, and manage audio and video recordings for their assignments. As the portfolios grow in size and quality, students document their progress in spoken English, as well as in teaching skills, over multiple semesters. The system also enables instructors and program administrators to review submitted materials.
The design of the courses in this program requires regular interactions between student and faculty. Administrative time savers are critical. At present, all sections and all courses within ITAP are using the online space. It has been the experience of program administrators and instructors that,that, compared to the “old way” of handling audio files through conventional e-mail, Sakai has “distinct advantages” and is more streamlined. No longer does the program have to concern itself with managing a large number of files. They now reside on an external server. Such self-assessment is clear evidence of ‘innovation’ making the teaching process more efficient and productive.
Site hosted by rSmart, a Sakai Commercial Affiliate.
Unless otherwise noted in copyright notes, all material published in the Opened Practices repository is licensed by its posting author under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License. Feel free to join the discussion about copyright and licensing at Opened Practices.