Developer Test Items
Universal Consulting Services, Inc.
Pensacola, FL
NMOTC provides professional, technical and consultative services in operationally related Fleet and Fleet Marine Force medical matters worldwide. The team supports technology management programs across the NMOTC Enterprise (Naval Aerospace Medical Institute [NAMI], Pensacola, FL; Naval Survival Training Institute [NSTI]; Naval Undersea Medical Institute [NUMI], Groton, CT; Surface Warfare Medical Institute [SWMI], San Diego, CA; Naval Expeditionary Medical Training Institute [NEMTI], Camp Pendleton, CA; Naval Special Operations Medical Institute [NSOMI], Fort Bragg, NC). The Developer Test Items position supports aviation psychological test item management and development. The position provides data collection protocol design and oversight, test item development, test bank management, and advanced data analytic support. Active Secret Clearance Required


Troubleshoot and continually evaluate functionality and design of adaptive test engines and test library usage/exposure over time using GOTS and COTS software systems;
Evaluate and maintain data on test library content coverage for 4,000 cognitive and non-cognitive items measuring 5 different aptitudes in 3 – 10 content areas each, as well as 11 non-cognitive trait facet categories;
Develop scales and test forms to capture traits, facets, or aptitudes related to effective job performance (scale and form development typically involves simultaneous design of numerous measures to be used for adaptive test library expansion, requiring focus on psychometric equivalence, content redundancy, establishment of linking items, and grammatical and stylistic quality);
Perform data management and cleaning for transfer of trait and aptitude measures data into formats and datasets amendable to analysis using item-response theory and factor analysis;
Perform parameter estimation and scale linkage analyses across forms to ensure comparability of scores generated using different forms targeting the same construct;
Oversee data collection protocol requirements for delivery of up to 30 different test forms to participant samples ranging in size from 10 – 300 people, to include maintenance of materials requirements and test form requirements by participant; and,
Contribute to or draft technical documentation summarizing analyses of test bank development work, scale validation, scale bias estimation, and data-model fit for customized measurement scales.

  • Knowledge and experience in developing psychological test questions in a military aviation environment
  • Effective communication and customer service skills;
  • Experience in analyzing and documenting information;
  • Experience in applying and deploying testing instruments;
  • Experience in using BILOG-MG3 & SPSS software;
  • Possess a Bachelor's Degree in Psychology with 2 years of experience in Psychology; and,
  • SECRET Security Clearance