The Amazon Web Services (
AWS) Certification team is seeking a Psychometrician with experience working with criterion-referenced assessment programs to support a large global
AWS Certification and Credentialing program. You will work closely with a team of psychometricians, subject matter experts, certification exam program managers, publishing, delivery, security, and product management teams to support ongoing analyses of exam and credential data. To be successful in this position, you must be highly motivated, creative, detail oriented, and a self-starter who is able to think big, execute, ensure high quality, yet stay focused on the details.
Key job responsibilities
Perform and support the main psychometric aspects of exam development and operations, including but not limited to automated test
assembly, item and test analyses, optimal item bank design, job task analysis, standard setting, quality assurance, and project planning. Conduct main aspects of psychometric analysis in operational work including performing item analysis using psychometric methods, building optimal test forms and pools via optimization techniques, analyzing and monitoring item bank health, setting pass standards via standard setting studies, and supporting Job Task Analysis (JTA) to define and refresh test blueprints. Conduct main aspects of psychometric analysis in developing and applying statistical and psychometric modeling to evaluate and ensure
AWS certification exams’ validity, reliability, applicability, efficiency, and accuracy. Participate in research projects to improve existing operational processes and quality using advanced techniques such as
Machine Learning (ML), statistical modeling,
Natural Language Processing (
NLP), Generative Artificial Intelligence (GenAI), etc. Develop automation code using
R or
Python for psychometric workflow pipeline and other tasks to improve operational efficiencies. Present, interpret, and communicate the results of analyses to stakeholders through written and oral reports. Follow the accreditation standards set by ISO/IEC:2012 17024 and the National Council for Certifying Agencies (NCCA) as they relate to valid psychometric practices. Engage with the professional community through conferences and publications.
- PhD
- Degree held in Statistics, Psychometrics, Educational Measurement, Quantitative Psychology, Data Science, Industrial-Organizational (I/O) Psychology, or a related field and one year of research or work experience in the job offered, or as a Research Scientist, Research Assistant, Software Engineer, or a related occupation.
- Must have 1 year of experience in the following skill(s): 1. large-scale education, licensure, or certification assessment programs. 2. operational psychometric tasks on large-scale education, licensure, or certification assessment programs including item analysis, equating and scaling, item response theory, classical test theory, form and pool
assembly, item bank health analysis, standard setting, and job task analysis. 3. at least one of the complex test designs such as
linear-on-the-fly testing (LOFT), computerized adaptive testing (CAT). 4. at least one of the following areas including
machine learning (ML) or
natural language processing (
NLP). 5. Programming skills in at least one script-based programming language (
R,
Python).
- Experience in testing for global audiences.
- History of conference presentations and publications.
- Experience with data forensics methods.
Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit
https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.
USA, OR, Portland - 136,000.00 - 184,000.00
USD annually
USA, TX, Austin - 136,000.00 - 184,000.00
USD annually
USA, VA, Arlington - 136,000.00 - 184,000.00
USD annually
USA, VA, Herndon - 136,000.00 - 184,000.00
USD annually
USA, WA, Bellevue - 136,000.00 - 184,000.00
USD annually
USA, WA, Seattle - 136,000.00 - 184,000.00
USD annually