Transcript
Video duration: 10:00
On-screen: [ETS? TOEIC?]
Speaker: Narrator 1, ETS? – The TOEIC? program presents Principles of Quality Assessments.
On-screen: [Why Assessment Matters.]
Why assessment matters. In today's competitive global market effectively communicating with colleagues, customers and partners around the world is essential. Now more than ever having an English proficient workforce is vital for business success. In fact English is spoken by one in four people globally and is the most widely used language in the world. How can you ensure that your workforce has the English language skills required to succeed?
In order to build a strong capable workforce decision makers such as human resources managers need to be able to make confident hiring, training and promotion decisions.
On-screen: [Confident Hiring, Training, Promotion Decisions.]
On-screen: [Ensure Graduates are Prepared.]
Additionally educational institutions need to ensure graduates are prepared for the global workforce. The consequences of making the wrong decision can be costly for your organization both monetarily and in terms of growth. When making informed decisions, it is important to gather the most reliable and accurate data. In some cases, like with English language proficiency, a measurement tool such as an assessment is required. The type of assessment you use determines the kind of information you will receive. This is why the assessment you choose matters.
On-screen: [English Assessments.]
You have many options for measuring English language proficiency, but how do you know which is the best assessment method? The key attributes of any quality assessment are validity, fairness and reliability.
On-screen: [English Assessments. Validity, Fairness, Reliability.]
On-screen: [Validity.]
Validity is the extent to which the test scores are appropriate for a particular purpose.
On-screen: [Fair.]
When we say fair, we mean assessments that are standardized, unbiased and accessible to anyone, anywhere regardless of gender, race, ethnicity or disability.
On-screen: [Reliable.]
An assessment is reliable when the test scores are consistent across administrations.
On-screen: [ETS? Over 70 Years Experience. Valid, Fair, Reliable.]
We at Educational Testing Service have over 70 years of experience and expertise developing valid, fair and reliable assessments for educators and learners around the world.
On-screen: [ETS? Over 70 Years Experience. Valid, Fair, Reliable. 50 Million Tests Annually. 180 Countries, Rigorous Research, 100% Quality, Equity in Education.]
Every year ETS? develops, administers and scores 50 million tests in over 180 countries. A commitment to rigorous research, 100% quality fairness and equity in education informs everything we do.
On-screen: [ETS? TOEIC?. TOEIC Test Created 1979. 14,000+ Organizations use the TOEIC? Tests. ~7 Million tests administered annually.]
In 1979 ETS? developed the TOEIC? program, now one of the world's leading English language assessments for the workplace. The TOEIC? tests include features that any quality language assessments should have.
On-screen: [ETS? TOEIC?. TOEIC Test Created 1979. 14,000+ Organizations use the TOEIC? Tests. ~7 Million tests administered annually. A Four-Skill Approach. Evidence-Centered Design. Test Administration. Reliable Scoring Process. Local Client Support.]
These include a four-skill approach, an evidence-centered test design, standardized and rigorous test security and administration, a consistent and reliable scoring process and client support services ensures score users understand test scores.
On-screen: [Four-Skill Approach]
Speaker: Narrator 2 – The cycle of communication in any language is made up of sending and receiving messages. When we send messages, we use our speaking and writing or productive skills. When we receive messages, we use our receptive skills: listening and reading.
In order to gain a full picture of one's language proficiency, you must measure all four of these skills.
On-screen: [Reading. Listening. Speaking. Writing.]
The TOEIC? program offers accurate measures of listening, reading, speaking and writing providing you with a complete assessment of English language skills. Listening and reading, the receptive skills, can easily be measured through multiple-choice questions. For the productive skills however, there are additional factors to consider. When we use our productive skills to communicate, we have an audience. In order to evaluate a person's speaking and writing skills, they must produce spoken and written messages.
Unlike multiple-choice responses that have a single correct answer, speaking and writing test questions are open-ended. This means each question could have several right answers. These open-ended questions are evaluated on several different kinds of criteria such as grammar, vocabulary, task fulfillment and appropriateness. In a professional work environment an individual may use productive skills to write emails, give presentations, or exchange information with customers or colleagues. We want to help you make sure your employees have the skills necessary to perform these tasks at work.
On-screen: [Test Designs.]
Speaker: Narrator 1 – One of the most important factors when choosing an assessment is test design. Here at ETS? we use a methodology called evidence-centered design for developing the TOEIC assessments.
On-screen: [Evidence-Centered Design. ETS?]
This methodology ensures that the information gathered from the assessment is consistent with the underlying purpose. In other words, using evidence found through research to design a test helps ensure the test will measure what it is intended to measure.
On-screen: [Consistent, Reliable, Fair, Valid.]
This process helps make certain that test results are consistent, reliable, fair and valid, which is exactly what you need to make confident decisions. Test questions or items are carefully written by assessment developers who are experts in their given subject. Every item goes through a rigorous content and fairness review. Reviews are completed by multiple assessment developers before the test is assembled. After the test is administered, each item undergoes a lengthy statistical analysis to ensure that it functions as intended. Each question on a test provides data that can be used to evaluate a test taker's skills. In general the more data collected on an assessment, the more accurate the results.
On-screen: [More Questions Needed?]
When choosing a test, it is important to consider not only the number of questions, but also the different question types.
On-screen: [Question Types. Questions for Reading Ability. Questions for Listening and Reading Ability. Questions for Speaking and Writing Ability.]
The TOEIC? program takes both into consideration. It uses an appropriate number of carefully chosen questions on each assessment to collect the right amount of data for an accurate result.
On-screen: [Evidence-Centered Design. Carefully chosen question types. Measures needed skills.]
ETS? assessment developers use evidence-centered design to carefully choose question types that directly measure the skills and abilities that are needed for English in the global workplace.
On-screen: [Research for the appropriate test use for: Hiring Training Promotion.]
In addition, ETS? researchers continue to support the use of TOEIC? assessments through ongoing research.
On-screen: [Standardized Administration and Security.]
Speaker: Narrator 2 – But test design is only one of the factors that ensures consistent, reliable, fair and valid data. Test administration and security are also very important factors. Standardized testing procedures ensure that test takers have the same testing experience no matter the place and time. Rigorous test security guidelines ensure that no test takers are unfairly advantaged or disadvantaged and that test scores remain fair and comparable across administrations. ETS? is a leader in test security and continues to refine protocols for detecting and monitoring security threats. In fact many of the test security practices pioneered by ETS? have become the industry standard and have been adopted by other companies and organizations around the globe.
On-screen: [Scoring Process.]
Speaker: Narrator 1 – Once test takers are finished taking the test, those tests need to be scored. The scoring process is a critical factor when evaluating a quality assessment. Listening and reading skills are measured using multiple-choice questions with which the objectivity of scores is certain.
On-screen: [Test Scoring Machine. Multiple-Choice Questions. Objectivity. Accuracy.]
The scoring of multiple-choice answer sheets is automated and quality control procedures ensure accuracy. Speaking and writing skills however are measured with open-ended questions.
On-screen: [Open-Ended Questions.]
This means that test takers are asked to produce spoken and written responses that must then be evaluated.
On-screen: [Open-Ended Questions. Qualified Human Raters. Rigorous Scoring Process.]
The TOEIC? program employs qualified human raters to evaluate TOEIC? speaking and writing responses. TOEIC? speaking and writing test scoring is a rigorous process built to ensure the most fair and valid results for every response. Unlike an interview test for example, where evaluators may exhibit unfair biases, TOEIC? test taker responses are randomly and anonymously assigned to raters effectively minimizing bias. Each TOEIC? speaking and writing test is evaluated by no less than six highly qualified raters. All TOEIC? raters are required to have specific experiences and education. In order to qualify as a TOEIC?-certified rater, all applicants must complete a certification test.
On-screen: [Qualified, Certified, Trained, Calibrated.]
Each rater is qualified, certified, trained and calibrated to each item type before they are able to score. For each scoring session, at least once per day, a rater must pass a scoring accuracy test referred to as calibration to ensure they are aligned with the scoring guidelines. Raters who do not pass calibration on any given scoring session are not allowed to score. TOEIC?-certified raters are also regularly monitored by scoring leaders to help assure complete score accuracy.
On-screen: [The Complete Solution.]
On-screen: [TOEIC? Program. Four Skill Approach. Test Design. Test Administration. Test Security. Scoring Process. Customer Service.]
Speaker: Narrator 2 – The complete solution. The TOEIC? test design, test administration, security and scoring processes give you the complete solution by ensuring that test results are truly valid, fair and reliable.
On-screen: [What do the test scores mean?]
It is important to note that test results are only valid, fair and reliable when they are used appropriately.
On-screen: [EPN. ETS? Preferred Network?. Interpreting test scores. Right tools. Local customer support. Setting cut scores. Benchmarks.]
The TOEIC? program is dedicated to assisting score users with understanding and interpreting test scores through tools and local customer support including assistance on setting cut scores or benchmarks. TOEIC? test scores can be used to prepare students for the global workplace and support informed hiring, training or promotion decisions. Your choice of assessment not only impacts your organization, but also builds the workforce of the future. 68% of global executives say that English is necessary to succeed in global markets. Prepare your organization for success by using the TOEIC? program, the world's leading English language assessment for the workplace.
On-screen: [EPN. ETS? Preferred Network?. For decisions that matter, the assessment you choose matters.]
To find out more about the TOEIC? program, contact your local ETS? preferred network member because for decisions that matter to you, the assessment you choose matters. Contact your local EPN member by visiting www.TOEICglobal.com.
On-screen: [www.TOEICglobal.com]
End of Principles of Quality Assessments Video