My first bite of AAPPL: logistics

 

1382316300d5kze
Image by: jmiltenburg

I have been tinkering with the idea of administering an outside performance/proficiency exam for a while. AAPPL (The ACTFL Assessment of Performance toward Proficiency in Languages) is the one I kept hearing about. So I’ve done my research, made sure AAPPL tech specs were met on our end at school, asked my students their opinions on which tests they preferred to take, and have pestered kind colleagues on Twitter with all sorts of questions about it. Carrie Toth had written a blog post about administering AAPPL and so did Bethanie Drew; you can find their wisdom here, here and here. Both provided me with much needed insights from “how to” to “how not to” which is equally important. Ladies, I thank you for your help! Je vous remercie! Спасибо!

This spring, when my department needed to spend some extra dollars by the end of the year (if you ever worked at school, you know that the money magically disappears if not used), I asked for permission to use some of it to pilot AAPPL with my senior class. With approval secured, the race to make it happen was on. And while I was checking the items off my to do list, I kept making another “what if” list in my head:

  • What if the technology fails?
  • What if the test includes prompts that we never discussed?
  • What if the difficulty level is way above their head?
  • What if they can’t figure out how to type accents?

…and that one…

  • What if the results are not where I would like them to be?

Yeah. That last one. It’s not really just my students that the test will be evaluating but also their teacher. It was my responsibility in the last four years to teach them vocabulary and structures, to develop their ability to use context, to organize and convey ideas in a comprehensible way without major patterns of errors, and most of all, to trust themselves to be able to interact and perform in French in a variety of situations. How did I measure up? I was definitely nervous. To the point that I had flashes and accelerated heart rate on the first day of the test administration. But before I can report to you about my takeaways from the results (you can read about them once speaking and written samples are scored), here’s my “To Do” list for the AAPPL first timers.

  • Learn about the test as much as possible. AAPPL site offers excellent resources: you can find tasks and topics descriptions according to grade levels here; tips for each section of the test here; AAPPL score descriptors here; and demo exams here.
  • Ask your IT team to review TECHNOLOGY section of the AAPPL FAQS (bottom of the page) to make sure that your school’s computers/laptops as well as Internet connection and filters comply with the requirements of the AAPPL test. You can test your own computer here; make sure to do it on school ground as your home connection may be quite different.
  • Establish an account with Language Testing International (you can find the link to this site on AAPPL’s site under “FAQS” and “Ordering”). It says that it may take up to two weeks to get your account approved but mine was ready to go in a few days.
  • Once your account is approved, you can easily order tests by downloading a template from Language Testing International and completing it with students’ information.
    • You can order all tests (Interpretive Reading, Interpretive Listening, Presentational Writing, Interpersonal Listening/Speaking) for every student, one per student or any combination of tests per student. You can add or cancel tests at any time.
    • You will not need to pay for tests upon ordering; you will be billed as the tests are completed and scored.
    • I recommend using students’ school ID numbers for the corresponding field on AAPPL test order to keep things simple.
    • I was able to negotiate for $10 per student from that “need to spend now” money for my pilot, and based on feedback from my students, I ordered Interpretive Reading, Interpretive Listening (both for the total of $5) and Presentational Writing ($5) for all of them. I also offered an option for those who wanted (and were willing to contribute $10) to take Interpersonal Listening/Speaking test; eight students volunteered.
  • Choose suitable date(s) for tests. AAPPL says that each of the components will take approximately 30 minutes but I found this not to be true. Most of my students took much longer for every test. The best part is that each test is composed of several tasks that are completed individually and each section can be ended and resumed at any point without compromising the test; all answers for tasks completed are saved automatically.
    • I allocated three class periods of 50 minutes on three consecutive days for three required tests (Interpretive Reading, Interpretive Listening, and Presentational Writing) but will never do it again because a considerable number of students were not able to finish the three tests and it was just too “standardized-testing” like. Next time, I will space the tests out (one or two per week), planning five of my class periods (5×50 minutes) for the entire test battery while making sure that fast finishers have something to do while others work at their own pace to finish.
    • Another unexpected obstacle came with the testing season as the district passed from one required standardized test to another using all available laptops which prevented me from completing all tests before AP exam season when random amount of students from your class may be missing due to an AP exam taking place that day. And then there were students absent due to sickness, vacations, college visits, scholarship interviews, etc. All of this dragged overall test completion way past my original intentions.
    • Take into consideration that it takes about two weeks to get speaking and written tests scored. For me, it means that these two tests must be done in April while Interpretive Reading and Listening can be completed at any time in May because the results are delivered instantly.
  • Before the first test: there are quite a few things to do.
    • Print labels with students’ log-in information. I taped them on 3×5 index cards and wrote a number 1-19 that helped me to establish a seating arrangement and keep track/pass out/collect these cards to be reused on the next testing day.
    • Go through EACH computer/laptop to be used for testing to make sure that it passes AAPPL Measure Systems Check. I spent my entire prep hour on this. BUT! Nothing is better than knowing that your kids will not encounter any technical issues. I also wrote the number of the laptops on each of the index cards with students’ log-in information to make sure that every student will get the same laptop that has been checked on each of the testing days.
    • Decide where you will administer the tests and secure that space. I wanted to have a quiet area away from the other classrooms but you can certainly do it in class too. Discuss all arrangements with your students so they know what to do on the first day of the test.
    • Prepare simple instruction sheet for your students; you can find mine here. I included the descriptions on how to type accents because we have 1:1 iPad program and students are not used to computer shortcuts.
  • On the day of the test: don’t be as nervous as I was. 🙂
    • Arrange the seating, set up the computers/laptops if you can. I wanted to use the time I had with students to the maximum, and having a prep hour just before testing time came in very handy.
    • I suggest that students watch one (or more) AAPPL tips videos that pertain to the test(s) they are taking, even if you may have watched them earlier in class. It helps to remind them how the test is structured and eases their anxiety.
    • Step back and supervise. If you did your homework, you can just walk around and be nervous like I was until you start seeing the results of the first test come in. I am certain that you will smile and be proud of what you and your students have accomplished. Just like I did. 

Have you administered AAPPL before or thinking of doing so? Do you have your own tips and tricks to share? And my main question to ponder after this experience: I would love to do this yearly but how do I fund this in the future?

9 comments

  1. Thank you so much for your detailed explanation of how to get ready for test administration. We are a week away from our first test and I am sooooo nervous. I am also very worried that I have not allowed enough time. I have an 85 minute period and plan for students to do both listening and reading tests during that class period. It is difficult to reserve computer lab time and I am not sure we will be able to get it all done. Your post is very timely and helpful for my thinking this out.

    Like

    • Rita,

      I am glad that you found this post useful. Your 85 minute period should be enough for most students to finish both tests. Listening is rather quick, it’s the reading that took a while. My students were taking a “Form B” test and it had several long passages at the end. Definitely plan on using the full 85 minutes for written portion if you plan to administer it and have some activities ready for the kids who finish early. As I mentioned in my post, it’s ok if the students do not complete the entire test. They can “quit” at any time they finish the task on a test and resume later from that point. If I were you, I would book an extra session in the lab just in case, for absent students and kids who didn’t finish in time.

      Like

    • Jim,
      I hope the first AAPPL test went well for your and your students. I am finally getting closer to analyzing the results as written scores are starting to come in and the speaking ones should hopefully be there in the next few days. Stay tuned for the Second Bite of AAPPL…

      Like

  2. We administer AAPPL at our school and our department chair is a wiz. We only do the listening and speaking tests and then we issue a writing test that was developed by team of teachers at school. We have a language lab and usually administer on a Saturday for our incoming freshman. We only use it as a placement exam, but, we are wondering if we should use it as a milestone test, in the same way that the National Spanish Exam is used. You seem like you have a great system here, keep it up!

    Like

  3. My experience was very similar to yours. I gave the AAPPL for the first time this year to my level 4 students (we have a 5 year program, year 5 is AP). I concur that it takes more time than you think it will. We too were caught out thinking it would take 3-4 days max and it took 5 to 6 days in the end. I would add to your feedback that the microphones for the speaking section should be carefully checked. We had some students who had recordings that were Unratable due to being too difficult to understand. They must have had the microphone too close to their mouths and it got all garbled and full of static. That was a disappointment.

    As fare as the results go, we will be giving the STAMP test next year because, while we were very proud of our students and very excited to see their results, we felt that some of the results were inflated and perhaps not totally realistic. I’ve since spoken with a couple of friends from others schools who also have given the AAPPL and learned that they had the same reaction to the results. Ultimately they were much happier with the STAMP test results and believe that the STAMP assessment was more accurate. In both cases, they have continued to use the STAMP. It is really hard to tell a kid who has an Advanced-low on the AAPPL that the AP exam is not going to be a cake walk and that they still have a great deal to learn.

    Like

    • Madame,

      Thank you for sharing your experiences. I have thought about giving STAMP to compare the two assessments and might just do it next year. Even though I can’t say that my students’ results were not realistic (I don’t have experience with many speaking scores); however, I did question scoring on writing section more than once. Having done the MOPI training, I can understand how the scores come about and at the same time I’ve had samples with the same score that were quite different in terms of both quality of the language and message communicated.

      You are correct about the microphones. We don’t have language lab or any special equipment aside from a few microphones that caused trouble during the first test run with poor quality recordings. I found that the best solution is to use regular commercial headphones with microphones, the ones that many teens use with their phones – not a single glitch.

      I also have to say that AAPPL team is very responsive: I posted a comment on ACTFL Community’s thread about AAPPL assessment with concerns about the language used in interpersonal speaking prompts and received a quick reply from Paul Sandrock who assured me that it will be passed along to the team to improve on next year’s assessment.

      As far as AP exam, the kids do have a great deal to learn but it is my opinion that College Board produces exam with unrealistic expectations, at least for French. Every single year, examinees come back with comments of background noise in recordings, strong regional accent speakers, readings on very narrow topics, or, like this year not having enough time after Calc to even eat lunch before starting the French exam. As you may know, it’s gotten to the point that many AP teachers complained on AP discussion board. Not the first time. Not sure if any changes will be made any time soon.

      In the end, we have to work with our students within restriction of the programs in our districts to help them achieve their best in terms of communicating in the language and viewing the world through a bit different lense than they did before embarking in language learning journey. AAPPL or STAMP of approval is a nice thing to have but the reward is really much more than that, no?

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s